AbstractObjectiveDetermination of the sunburn protection provided by a sunscreen product is required globally for sales of these products. Over the past 80 years, many aspects of determining the protection ‘factor’ have evolved and been modified, with varying levels of impact on the sunburn protection factor (SPF) value. In order to compare new non‐invasive and in vitro methods against traditional SPF test protocols, a large, multi‐center clinical trial was conducted to establish the level of equivalence of these new methods with the current codified testing standard ISO24444 ([1]: Cosmetics – sun protection methods – in vivo determination of the sun protection factor (SPF), 2019). This report reports the variability found in the in vivo determination of SPF values.MethodsThirty‐two products of varying levels of sunburn and UVA protection, in a variety of formulation vehicles and ultraviolet (UV) filter combinations and concentrations, were coded and sent to pre‐qualified in vivo SPF testing laboratories. The products were divided into eight product groups (four products per product group). For each product group, samples were sent to four laboratories (across product groups, a total of 12 laboratories participated). Precision and trueness estimates were calculated separately for each product group. ‘Expected’ SPF values were not provided to the test laboratories. However, laboratories were informed as to whether the ‘true’ SPF was less than or greater than 25.ResultsInterlaboratory variability for samples was proportional to the SPF of the products, with high SPF products having higher variability. Intra‐laboratory variability (repeatability) was much lower than the interlaboratory variability.ConclusionsThis study highlights the fact that the interlaboratory variability of SPF results is considerable and is likely greater than expected by the public and regulatory bodies.