Measuring information content from observations for data assimilation: relative entropy versus shannon entropy difference

Article English OPEN
Xu, Qin (2007)

The relative entropy is compared with the previously used Shannon entropy difference as a measure of the amount of information extracted from observations by an optimal analysis in terms of the changes in the probability density function (pdf) produced by the analysis with respect to the background pdf. It is shown that the relative entropy measures both the signal and dispersion parts of the information content from observations, while the Shannon entropy difference measures only the dispersion part. When the pdfs are Gaussian or transformed to Gaussian, the signal part of the information content is given by a weighted inner-product of the analysis increment vector and the dispersion part is given by a non-negative definite function of the analysis and background covariance matrices. When the observation space is transformed based on the singular value decomposition of the scaled observation operator, the information content becomes separable between components associated with different singular values. Densely distributed observations can be then compressed with minimum information loss by truncating the components associate with the smallest singular values. The differences between the relative entropy and Shannon entropy difference in measuring information content and information loss are analysed in details and illustrated by examples.
  • References (30)
    30 references, page 1 of 3

    Bennett, A. F. 1992. Inverse Method in Physical Oceanography. Cambridge University Press, New York, 346 pp.

    Bernardo, J. M. and Smith, A. F. M. 1994. Bayesian Theory. John Wiley and Sons, New York, 586 pp.

    Daley, R. 1991. Atmospheric Data Analysis. Cambridge University Press, New York, 457 pp.

    Doviak, J. D. and Zrnic, D. S. 1993. Doppler Radar and Weather Observations. 2nd Edition. Academic Press, New York, 562 pp.

    Eyre, J. R. 1990. The information content of data from satellite sounding systems. A simulation study. Q. J. R. Meteor. Soc. 116, 401-434.

    Golub, G. H. and Van Loan, C. F. 1983. Matrix Computations. Johns Hopkins University Press, Baltimore, 476 pp.

    Gong, J., Wang, L. and Xu, Q. 2003. A three-step dealiasing method for Doppler velocity data quality control. J. Atmos. Ocean. Technol. 20, 1738-1748.

    Haven, K., Majda, A. J. and Abramov, R. 2005. Quantifying predictability through information theory. small sample estimation in a nonGaussian framework. J. Comp. Phys. 206, 334-362.

    Hodur, R. M. 1997. The Naval Research Laboratory's coupled ocean/atmosphere mesoscale prediction system (COAMPS). Mon. Wea. Rev. 125, 1414-1430.

    Houtekamer, P. L. and Mitchell, H. L. 2001. A sequential ensemble Kalman filter for atmospheric data assimilation. Mon. Wea. Rev. 129, 123-137.

  • Metrics
    No metrics available
Share - Bookmark