Parametric Bayesian Estimation of Differential Entropy and Relative Entropy

Other literature type, Article English OPEN
Maya Gupta; Santosh Srivastava;
  • Publisher: Molecular Diversity Preservation International
  • Journal: Entropy (issn: 1099-4300)
  • Publisher copyright policies & self-archiving
  • Related identifiers: doi: 10.3390/e12040818
  • Subject: Astrophysics | Wishart | Kullback-Leibler | QB460-466 | Q | differential entropy | Pareto | relative entropy | Science | Physics | QC1-999
    arxiv: Statistics::Methodology | Statistics::Theory | Statistics::Other Statistics

Given iid samples drawn from a distribution with known parametric form, we propose the minimization of expected Bregman divergence to form Bayesian estimates of differential entropy and relative entropy, and derive such estimators for the uniform, Gaussian, Wishart, and... View more
  • References (31)
    31 references, page 1 of 4

    1. El Saddik, A.; Orozco, M.; Asfaw, Y.; Shirmohammadi, S.; Adler, A. A novel biometric system for identification and verification of haptic users. IEEE Trans. Instrum. Meas. 2007, 56, 895-906.

    2. Choi, H. Adaptive Sampling and Forecasting with Mobile Sensor Networks; PhD Dissertation; MIT: Cambridge, MA, USA, 2009.

    3. Moddemeijer, R. On estimation of entropy and mutual information of continuous distributions. Signal Process. 1989, 16, 233-246.

    4. Misra, N.; Singh, H.; Demchuk, E. Estimation of the entropy of a multivariate normal distribution. J. Multivariate Anal. 2005, 92, 324-342.

    5. Wang, Q.; Kulkarni, S.R.; Verd u´, S. Divergence estimation for multi-dimensional densities via k nearest-neighbor distances. IEEE Trans. Inform. Theory 2009, 55, 2392-2405.

    6. Ahmed, N.A.; Gokhale, D.V. Entropy expressions and their estimators for multivariate distributions. IEEE Trans. Inform. Theory 1989, pp. 688-692.

    7. Beirlant, J.; Dudewicz, E.; Gy o¨rfi, L.; Meulen, E.V.D. Nonparametric entropy estimation: An overview. Intl. J. Math. Stat. Sci. 1997, 6, 17-39.

    8. Nilsson, M.; Kleijn, W.B. On the estimation of differential entropy from data located on embedded manifolds. IEEE Trans. Inform. Theory 2007, 53, 2330-2341.

    9. Kozachenko, L.F.; Leonenko, N.N. Sample estimate of entropy of a random vector. Probl. Inform. Transm. 1987, 23, 95-101.

    10. Goria, M.N.; Leonenko, N.N.; Mergel, V.V.; Inverardi, P.L. A new class of random vector entropy estimators and its applications in testing statistical hypotheses. J. Nonparametric Stat. 2005, 17, 277-297.

  • Related Organizations (8)
  • Metrics
Share - Bookmark