Heavy Tailed Distributions of Effect Sizes in Systematic Reviews of Complex Interventions

Article English OPEN
Burton, Christopher (2012)
  • Publisher: Public Library of Science
  • Journal: PLoS ONE, volume 7, issue 3 (eissn: 1932-6203)
  • Related identifiers: pmc: PMC3313970, doi: 10.1371/journal.pone.0034222
  • Subject: Agricultural and Biological Sciences(all) | Applied Mathematics | Research Article | Mathematics | Clinical Research Design | Medicine | Health Care Policy | /dk/atira/pure/subjectarea/asjc/1100 | Biochemistry, Genetics and Molecular Biology(all) | /dk/atira/pure/subjectarea/asjc/2700 | /dk/atira/pure/subjectarea/asjc/1300 | Medicine(all) | Statistics | Non-Clinical Medicine

Background<br/><br/>Systematic reviews of complex interventions commonly find heterogeneity of effect sizes among similar interventions which cannot be explained. Commentators have suggested that complex interventions should be viewed as interventions in complex systems. We hypothesised that if this is the case, the distribution of effect sizes from complex interventions should be heavy tailed, as in other complex systems. Thus, apparent heterogeneity may be a feature of the complex systems in which such interventions operate.<br/><br/>Methodology/Principal Findings<br/><br/>We specified three levels of complexity and identified systematic reviews which reported effect sizes of healthcare interventions at two of these levels (interventions to change professional practice and personal interventions to help smoking cessation). These were compared with each other and with simulated data representing the lowest level of complexity. Effect size data were rescaled across reviews at each level using log-normal parameters and pooled. Distributions were plotted and fitted against the inverse power law (Pareto) and stretched exponential (Weibull) distributions, heavy tailed distributions which are commonly reported in the literature, using maximum likelihood fitting. The dataset included 155 studies of interventions to change practice and 98 studies of helping smoking cessation. Both distributions showed a heavy tailed distribution which fitted best to the inverse power law for practice interventions (exponent = 3.9, loglikelihood = −35.3) and to the stretched exponential for smoking cessation (loglikelihood = −75.2). Bootstrap sensitivity analysis to adjust for possible publication bias against weak results did not diminish the goodness of fit.<br/><br/>Conclusions/Significance<br/><br/>The distribution of effect sizes from complex interventions includes heavy tails as typically seen in both theoretical and empirical complex systems. This is in keeping with the idea of complex interventions as interventions in complex systems.
  • References (32)
    32 references, page 1 of 4

    1. Campbell M, Fitzpatrick R, Haines A, Kinmonth AL, Sandercock P, et al. (2000) Framework for design and evaluation of complex interventions to improve health. BMJ 321(7262): 694-696.

    2. Craig P, Dieppe P, Macintyre S, Michie S, Nazareth I, et al. (2008) Developing and evaluating complex interventions: the new Medical Research Council guidance. BMJ 337: a1655.

    3. Oxman AD, Thomson MA, Davis DA, Haynes RB (1995) No magic bullets: a systematic review of 102 trials of interventions to improve professional practice. CMAJ 153(10): 1423-1431.

    4. Coiera E (2011) Why system inertia makes health reform so difficult. BMJ 342.

    5. Rickles D, Hawe P, Shiell A (2007) A simple guide to chaos and complexity. J Epidemiol Community Health 61(11): 933-937.

    6. Rickles D (2009) Causality in complex interventions. Med Health Care Philos 12(1): 77-90.

    7. Hawe P, Shiell A, Riley T (2009) Theorising interventions as events in systems. Am J Community Psychol 43(3-4): 267-276.

    8. Shiell A, Hawe P, Gold L (2008) Complex interventions or complex systems? Implications for health economic evaluation. BMJ 336(7656): 1281-1283.

    9. Paley J (2010) The appropriation of complexity theory in health care. J Health Serv Res Policy 15(1): 59-61.

    10. Hammond RA (2009) Complex systems modeling for obesity research. Prev Chronic Dis 6(3).

  • Related Research Results (2)
  • Metrics
    No metrics available
Share - Bookmark