BackgroundDetecting new and enlarged lesions in multiple sclerosis (MS) patients is needed to determine their disease activity. LeMan‐PV is a software embedded in the scanner reconstruction system of one vendor, which automatically assesses new and enlarged white matter lesions (NELs) in the follow‐up of MS patients; however, multicenter validation studies are lacking.PurposeTo assess the accuracy of LeMan‐PV for the longitudinal detection NEL white‐matter MS lesions in a multicenter clinical setting.Study TypeRetrospective, longitudinal.SubjectsA total of 206 patients with a definitive MS diagnosis and at least two follow‐up MRI studies from five centers participating in the Swiss Multiple Sclerosis Cohort study. Mean age at first follow‐up = 45.2 years (range: 36.9–52.8 years); 70 males.Field Strength/SequenceFluid attenuated inversion recovery (FLAIR) and T1‐weighted magnetization prepared rapid gradient echo (T1‐MPRAGE) sequences at 1.5 T and 3 T.AssessmentThe study included 313 MRI pairs of datasets. Data were analyzed with LeMan‐PV and compared with a manual “reference standard” provided by a neuroradiologist. A second rater (neurologist) performed the same analysis in a subset of MRI pairs to evaluate the rating‐accuracy. The Sensitivity (Se), Specificity (Sp), Accuracy (Acc), F1‐score, lesion‐wise False‐Positive‐Rate (aFPR), and other measures were used to assess LeMan‐PV performance for the detection of NEL at 1.5 T and 3 T. The performance was also evaluated in the subgroup of 123 MRI pairs at 3 T.Statistical TestsIntraclass correlation coefficient (ICC) and Cohen's kappa (CK) were used to evaluate the agreement between readers.ResultsThe interreader agreement was high for detecting new lesions (ICC = 0.97, Pvalue < 10−20, CK = 0.82, P value = 0) and good (ICC = 0.75, P value < 10−12, CK = 0.68, P value = 0) for detecting enlarged lesions. Across all centers, scanner field strengths (1.5 T, 3 T), and for NEL, LeMan‐PV achieved: Acc = 61%, Se = 65%, Sp = 60%, F1‐score = 0.44, aFPR = 1.31. When both follow‐ups were acquired at 3 T, LeMan‐PV accuracy was higher (Acc = 66%, Se = 66%, Sp = 66%, F1‐score = 0.28, aFPR = 3.03).Data ConclusionIn this multicenter study using clinical data settings acquired at 1.5 T and 3 T, and variations in MRI protocols, LeMan‐PV showed similar sensitivity in detecting NEL with respect to other recent 3 T multicentric studies based on neural networks. While LeMan‐PV performance is not optimal, its main advantage is that it provides automated clinical decision support integrated into the radiological‐routine flow.Evidence Level4Technical EfficacyStage 2