
arXiv: 1711.04155
SummaryFactor analysis and principal component analysis are used in many application areas. The first step, choosing the number of components, remains a serious challenge. Our work proposes improved methods for this important problem. One of the most popular state of the art methods is parallel analysis (PA), which compares the observed factor strengths with simulated strengths under a noise-only model. The paper proposes improvements to PA. We first derandomize it, proposing deterministic PA, which is faster and more reproducible than PA. Both PA and deterministic PA are prone to a shadowing phenomenon in which a strong factor makes it difficult to detect smaller but more interesting factors. We propose deflation to counter shadowing. We also propose to raise the decision threshold to improve estimation accuracy. We prove several consistency results for our methods, and test them in simulations. We also illustrate our methods on data from the human genome diversity project, where they significantly improve the accuracy.
Methodology (stat.ME), FOS: Computer and information sciences, permutation methods, principal component analysis, factor analysis, Factor analysis and principal components; correspondence analysis, random matrix theory, parallel analysis, Statistics - Methodology, Applications of statistics to biology and medical sciences; meta analysis
Methodology (stat.ME), FOS: Computer and information sciences, permutation methods, principal component analysis, factor analysis, Factor analysis and principal components; correspondence analysis, random matrix theory, parallel analysis, Statistics - Methodology, Applications of statistics to biology and medical sciences; meta analysis
| selected citations These citations are derived from selected sources. This is an alternative to the "Influence" indicator, which also reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically). | 61 | |
| popularity This indicator reflects the "current" impact/attention (the "hype") of an article in the research community at large, based on the underlying citation network. | Top 1% | |
| influence This indicator reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically). | Top 10% | |
| impulse This indicator reflects the initial momentum of an article directly after its publication, based on the underlying citation network. | Top 10% |
