
arXiv: 1007.2827
We draw relationships between the generalized data processing theorems of Zakai and Ziv (1973 and 1975) and the dynamical version of the second law of thermodynamics, a.k.a. the Boltzmann H-Theorem, which asserts that the Shannon entropy, $H(X_t)$, pertaining to a finite--state Markov process $\{X_t\}$, is monotonically non-decreasing as a function of time $t$, provided that the steady-state distribution of this process is uniform across the state space (which is the case when the process designates an isolated system). It turns out that both the generalized data processing theorems and the Boltzmann H-Theorem can be viewed as special cases of a more general principle concerning the monotonicity (in time) of a certain generalized information measure applied to a Markov process. This gives rise to a new look at the generalized data processing theorem, which suggests to exploit certain degrees of freedom that may lead to better bounds, for a given choice of the convex function that defines the generalized mutual information.
26 pages, 2 figures, submitted to IEEE Transactions on Information Theory
FOS: Computer and information sciences, Statistical Mechanics (cond-mat.stat-mech), Computer Science - Information Theory, Information Theory (cs.IT), FOS: Physical sciences, Condensed Matter - Statistical Mechanics
FOS: Computer and information sciences, Statistical Mechanics (cond-mat.stat-mech), Computer Science - Information Theory, Information Theory (cs.IT), FOS: Physical sciences, Condensed Matter - Statistical Mechanics
| selected citations These citations are derived from selected sources. This is an alternative to the "Influence" indicator, which also reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically). | 14 | |
| popularity This indicator reflects the "current" impact/attention (the "hype") of an article in the research community at large, based on the underlying citation network. | Top 10% | |
| influence This indicator reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically). | Top 10% | |
| impulse This indicator reflects the initial momentum of an article directly after its publication, based on the underlying citation network. | Top 10% |
