Indirect Comparisons: A Review of Reporting and Methodological Quality

Article English OPEN
Donegan, Sarah ; Williamson, Paula ; Gamble, Carrol ; Tudur-Smith, Catrin (2010)
  • Publisher: Public Library of Science
  • Journal: PLoS ONE (vol: 5)
  • Related identifiers: doi: 10.1371/journal.pone.0011054, pmc: PMC2978085
  • Subject: Evidence-Based Healthcare/Clinical Decision-Making | Research Article | Mathematics/Statistics | Evidence-Based Healthcare/Statistical Methodologies and Health Informatics | wa_20_5

Background: The indirect comparison of two interventions can be valuable in many situations. However, the quality of an indirect comparison will depend on several factors including the chosen methodology and validity of underlying\ud assumptions. Published indirect comparisons are increasingly more common in the medical literature, but as yet, there are no published recommendations of how they should be reported. Our aim is to systematically review the quality of published indirect comparisons to add to existing empirical data suggesting that improvements can be made when reporting and applying indirect comparisons.\ud \ud Methodology/Findings: Reviews applying statistical methods to indirectly compare the clinical effectiveness of two\ud interventions using randomised controlled trials were eligible. We searched (1966–2008) Database of Abstracts and Reviews of Effects, The Cochrane library, and Medline. Full review publications were assessed for eligibility. Specific criteria to assess quality were developed and applied. Forty-three reviews were included. Adequate methodology was used to calculate the indirect comparison in 41 reviews. Nineteen reviews assessed the similarity assumption using sensitivity analysis, subgroup analysis, or meta-regression. Eleven reviews compared trial-level characteristics. Twenty-four reviews assessed statistical\ud homogeneity. Twelve reviews investigated causes of heterogeneity. Seventeen reviews included direct and indirect evidence for the same comparison; six reviews assessed consistency. One review combined both evidence types. Twentyfive reviews urged caution in interpretation of results, and 24 reviews indicated when results were from indirect evidence by stating this term with the result.\ud Conclusions: This review shows that the underlying assumptions are not routinely explored or reported when undertaking indirect comparisons. We recommend, therefore, that the quality of indirect comparisons should be improved, in particular, by assessing assumptions and reporting the assessment methods applied. We propose that the quality criteria applied in this article may provide a basis to help review authors carry out indirect comparisons and to aid appropriate interpretation.
  • References (58)
    58 references, page 1 of 6

    1. Song F, Harvey I, Lilford R (2008) Adjusted indirect comparison may be less biased than direct comparison for evaluating new pharmaceutical interventions. Journal of Clinical Epidemiology 61(5): 455-463.

    2. Glenny AM, Altman DG, Song F, Sakarovitch C, Deeks JJ, et al. (2005) Indirect comparisons of competing interventions. Health Technology Assessment 9(26): 1-134.

    3. Bucher HC, Guyatt GH, Griffith LE, Walter SD (1997) The results of direct and indirect treatment comparisons in meta-analysis of randomized controlled trials. Journal of Clinical Epidemiology 50(6): 683-91.

    4. Higgins J, Green S, eds. Cochrane Handbook for Systematic Reviews of Interventions. Version 5.0.0 [updated February 2008]. The Cochrane Collaboration. Available:

    5. Altman DG, Bland JM (2003) Statistics Notes: Interaction revisited: the difference between two estimates. Bmj 326(7382): 219.

    6. Song F, Loke YK, Walsh T, Glenny A-M, Eastwood AJ, et al. (2009) Methodological problems in the use of indirect comparisons for evaluating healthcare interventions: survey of published systematic reviews. Bmj 338(apr03_1): b1147-.

    7. Cooper N, Sutton A, Morris D, Ades A, Welton N (2009) Addressing betweenstudy heterogeneity and inconsistency in mixed treatment comparisons: Application to stroke prevention treatments in individuals with non-rheumatic atrial fibrillation. Statistics in Medicine 28(14): 1861-1881.

    8. Song F, Altman DG, Glenny A-M, Deeks JJ (2003) Validity of indirect comparison for estimating efficacy of competing interventions: empirical evidence from published meta-analyses. BMJ 326(7387): 472.

    9. Song F, Glenny AM, Altman DG (2000) Indirect comparison in evaluating relative efficacy illustrated by antimicrobial prophylaxis in colorectal surgery. Controlled Clinical Trials 21(5): 488-497.

    10. Edwards SJ, Clarke MJ, Wordsworth S, Borrill J (2009) Indirect comparisons of treatments based on systematic reviews of randomised controlled trials. International Journal of Clinical Practice 63(6): 841-854.

  • Related Research Results (3)
  • Metrics
    No metrics available
Share - Bookmark