AbstractBackgroundThe multicenter SPIDY trial (pallidal stimulation for generalized, idiopathic dystonia) recently reported a marked improvement in dystonia which was assessed by the Burke‐Fahn‐Marsden (BFM) scale. However, the reliability of this tool has rarely been evaluated and its use in a multicenter study has never been assessed prospectively.PurposeTo evaluate the concordance between three unblinded clinical raters and one single‐blinded rater for 10 prospective series of ratings on the BFM scale in 22 dystonic patients of the SPIDY study.MethodsTen assessments on the BFM scale were performed under various stimulation conditions at different time points (before surgery and 1, 3, 6, and 12 months afterwards). Patients were first evaluated by three unblinded clinical raters (one per center). All assessments were videotaped and sent to a blinded rater. Intra‐ and inter‐rater reliability was assessed using intraclass correlation coefficients.ResultsThe intra‐rater reliability at inclusion was better for the blinded rater than for the clinical raters. The inter‐rater reliability (comparing the blinded rater with each clinical rater) was “very good” at inclusion, “fair” at month 1 and was “good” at month 3, month 6, and month 12.ConclusionBlinding (rather than video) is probably the key factor in better intra‐rater reliability and can produce more accurate rating than clinical rating. Consequently, a blind procedure should be performed systematically in multicenter studies. As inter‐rater reliability is good in trained unblinded raters, the BFM scale may also be used in the follow up of dystonic patients in movement disorders centers, in clinical practice. © 2007 Movement Disorder Society