
The partial information decomposition (PID) aims to quantify the amount of redundant information that a set of sources provides about a target. Here, we show that this goal can be formulated as a type of information bottleneck (IB) problem, termed the “redundancy bottleneck” (RB). The RB formalizes a tradeoff between prediction and compression: it extracts information from the sources that best predict the target, without revealing which source provided the information. It can be understood as a generalization of “Blackwell redundancy”, which we previously proposed as a principled measure of PID redundancy. The “RB curve” quantifies the prediction–compression tradeoff at multiple scales. This curve can also be quantified for individual sources, allowing subsets of redundant sources to be identified without combinatorial optimization. We provide an efficient iterative algorithm for computing the RB curve.
FOS: Computer and information sciences, partial information decomposition, redundancy, Science, Physics, QC1-999, Computer Science - Information Theory, Information Theory (cs.IT), Q, information bottleneck, rate distortion, Machine Learning (stat.ML), Astrophysics, Article, QB460-466, Statistics - Machine Learning
FOS: Computer and information sciences, partial information decomposition, redundancy, Science, Physics, QC1-999, Computer Science - Information Theory, Information Theory (cs.IT), Q, information bottleneck, rate distortion, Machine Learning (stat.ML), Astrophysics, Article, QB460-466, Statistics - Machine Learning
| selected citations These citations are derived from selected sources. This is an alternative to the "Influence" indicator, which also reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically). | 2 | |
| popularity This indicator reflects the "current" impact/attention (the "hype") of an article in the research community at large, based on the underlying citation network. | Top 10% | |
| influence This indicator reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically). | Average | |
| impulse This indicator reflects the initial momentum of an article directly after its publication, based on the underlying citation network. | Average |
