
doi: 10.1086/382053
To the Editor:Our study (Bugawan et al. 2003xAssociation and interaction of the IL4R, IL4, and IL13 loci with type 1 diabetes among Filipinos. Bugawan, TL, Mirel, DB, Valdes, AM, Panelo, A, Pozzilli, P, and Erlich, HA. Am J Hum Genet. 2003; 72: 1505–1514Abstract | Full Text | Full Text PDF | PubMed | Scopus (48)See all References2003) reported a negative association of a specific IL4-524 haplotype with type 1 diabetes (T1D), consistent with a previous report (Mirel et al. 2002xAssociation of IL4R haplotypes with type 1 diabetes. Mirel, DB, Valdes, AM, Lazzeroni, LC, Reynolds, RL, Erlich, HA, and Noble, JA. Diabetes. 2002; 51: 3336–3341Crossref | PubMedSee all References2002), and presented evidence for a genetic interaction between IL4-524 and IL4R SNPs. To test the latter, we computed relevant P values by permuting multilocus genotypes separately in case and control groups.The criticism raised by Kraft (2004xMultiple comparisons in studies of gene × gene, gene × environment interaction. Kraft, P. Am J Hum Genet. 2004; 74: 582–584Abstract | Full Text | Full Text PDF | PubMed | Scopus (9)See all References2004 [in this issue]) is not directed at our implementation of permutation testing, per se, but at permutation testing in general. His argument is that permutation testing does not properly account for multiple comparisons, resulting in an increase in false claims of significance, or type I familywise error (FWE). In the place of permutation testing, Kraft advocates the use of the Simes method—an elaboration of the classic Bonferroni procedure. In response, we wish to show that permutation testing can be used to obtain a desired false-positive error rate (as, indeed, can be demonstrated using Kraft’s example) and, moreover, that such an approach has the added advantage of providing additional protection against false claims of nonsignificance, or type II error.It should be noted that permutation methods are well established as a robust approach for obtaining overall significance levels while minimizing type II error (e.g., Good 1994xGood, P. CrossrefSee all References1994; Doerge and Churchill 1996xPermutation tests for multiple loci affecting a quantitative character. Doerge, RW and Churchill, GA. Genetics. 1996; 142: 285–294PubMedSee all References1996; Lynch and Walsh 1998xLynch, M and Walsh, B. : 441–442See all References1998), that such methods are extensible to multiple-testing scenarios (Westfall and Young 1993xWestfall, PH and Young, SS. See all References1993), and that examples of their application to human genetics are not uncommon (e.g., Lewis et al. 2003xGenome scan meta-analysis of schizophrenia and bipolar disorder, part II: schizophrenia. Lewis, CM, Levinson, DF, Wise, LH, DeLisi, LE, Straub, RE, Hovatta, I, Williams, NM et al. Am J Hum Genet. 2003; 73: 34–48Abstract | Full Text | Full Text PDF | PubMed | Scopus (807)See all References2003). However, as with any statistical method, the validity is dependent on correct application. Kraft provides an analysis of the permutation testing by discussing the distribution of two P values obtained from hypothetically permuted distributions (i.e., independent and uniformly distributed under the null hypothesis). The joint cumulative distribution function (CDF) for these two P values is given as F(P(1),P(2))=P(1)(2P(2)−P(1)), where P(1) and P(2) are, respectively, the first- and second-ordered P values. As such, Kraft notes that the Pr(P<.05) for this joint distribution is ∼0.1, indicating that we would expect to see the smaller P value, or P(1)<.05, about 10% of the time. Kraft’s argument, therefore, is that for independent tests, use of a critical value of .05 leads to a type I error rate of 10%.In fact, the proper approach for permutation testing—adjusted or unadjusted for multiple comparisons—is to find the critical value corresponding to the desired type I error rate. Specifically, if we consider the simulations presented by Kraft as equivalent to the result of a permutation test, we would seek the value of x in the permuted distribution for which Pr(P
Genetics, Genetics(clinical)
Genetics, Genetics(clinical)
| selected citations These citations are derived from selected sources. This is an alternative to the "Influence" indicator, which also reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically). | 1 | |
| popularity This indicator reflects the "current" impact/attention (the "hype") of an article in the research community at large, based on the underlying citation network. | Average | |
| influence This indicator reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically). | Average | |
| impulse This indicator reflects the initial momentum of an article directly after its publication, based on the underlying citation network. | Average |
