Powered by OpenAIRE graph
Found an issue? Give us feedback
image/svg+xml art designer at PLoS, modified by Wikipedia users Nina, Beao, JakobVoss, and AnonMoos Open Access logo, converted into svg, designed by PLoS. This version with transparent background. http://commons.wikimedia.org/wiki/File:Open_Access_logo_PLoS_white.svg art designer at PLoS, modified by Wikipedia users Nina, Beao, JakobVoss, and AnonMoos http://www.plos.org/ ZENODOarrow_drop_down
image/svg+xml art designer at PLoS, modified by Wikipedia users Nina, Beao, JakobVoss, and AnonMoos Open Access logo, converted into svg, designed by PLoS. This version with transparent background. http://commons.wikimedia.org/wiki/File:Open_Access_logo_PLoS_white.svg art designer at PLoS, modified by Wikipedia users Nina, Beao, JakobVoss, and AnonMoos http://www.plos.org/
ZENODO
Preprint . 2022
License: CC BY
Data sources: Datacite
image/svg+xml art designer at PLoS, modified by Wikipedia users Nina, Beao, JakobVoss, and AnonMoos Open Access logo, converted into svg, designed by PLoS. This version with transparent background. http://commons.wikimedia.org/wiki/File:Open_Access_logo_PLoS_white.svg art designer at PLoS, modified by Wikipedia users Nina, Beao, JakobVoss, and AnonMoos http://www.plos.org/
ZENODO
Preprint . 2022
License: CC BY
Data sources: ZENODO
image/svg+xml art designer at PLoS, modified by Wikipedia users Nina, Beao, JakobVoss, and AnonMoos Open Access logo, converted into svg, designed by PLoS. This version with transparent background. http://commons.wikimedia.org/wiki/File:Open_Access_logo_PLoS_white.svg art designer at PLoS, modified by Wikipedia users Nina, Beao, JakobVoss, and AnonMoos http://www.plos.org/
ZENODO
Preprint . 2022
License: CC BY
Data sources: Datacite
versions View all 2 versions
addClaim

Information, Shannon's Entropy and Violating Initial Constraints

Authors: Ruggeri, Francesco R.;

Information, Shannon's Entropy and Violating Initial Constraints

Abstract

One may maximize Shannon’s entropy -Sum over i f(ei) ln(f(ei)) subject to the constraint Sum over i ei f(ei) where f(ei) is the probability to have a gas particle with energy ei. This yields f(ei)= C(T)exp(-ei/T) from which one may calculate Eave. The result, however, is consistent with a canonical distribution, not a microcanonical one for which total energy = Eave is a strict constraint. For the canonical approach one considers weights exp(-Ei/T) where Ei may be larger than Eave. (Ei is the sum of ei’s.) In other words, one violates the original microcanonical constraint on total energy (for ease of calculation). This is consistent with Shannon’s entropy for which “information” is ln(f(ei)). If one thinks in terms of two body elastic scattering then f(e1)f(e2) = f(e3)f(e4) and e1+e2 = e3+e4. One may use any e1 and e2 even if their sum is greater than Eave the microcanonical constraint. Thus the use of information or Shannon’s entropy may violate original constraints in a problem. In this note we wish to examine an example (1) from information theory involving the compression of bits used to represent six letters A-F which have different probabilities of occurrence. Six letters may be encoded using 2x2x2 bits. This is an initial kind of constraint or bound i.e. one needs 3 bits for each letter. Given the probabilities one may calculate ln(probability) = information of each letter which represents the number of bits needed for each. The idea of information as ln(probability) is part of Shannon’s entropy. The problem is that like in the case of the canonical distribution violating the microcanonical constraint on total energy =Eave (which is done for ease of calculation), using information may violate an existing constraint or bound already in the problem. For example, for a low probability, ln(probability) may be larger than 3 the original bound or constraint used for the uniqueness of 6 letters. Using Shannon-Fano coding, a letter is coded with a number of bits close to ln(probability) which means that for ln(probability)>3, the number of bits needed is larger than the original bound or constraint. The idea, however, in information theory is to compress the number of bits. Furthermore there is no “ease of calculation” as in the case of the canonical distribution. We investigate this idea in this note.

Keywords

violation of constraints, information

  • BIP!
    Impact byBIP!
    selected citations
    These citations are derived from selected sources.
    This is an alternative to the "Influence" indicator, which also reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically).
    0
    popularity
    This indicator reflects the "current" impact/attention (the "hype") of an article in the research community at large, based on the underlying citation network.
    Average
    influence
    This indicator reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically).
    Average
    impulse
    This indicator reflects the initial momentum of an article directly after its publication, based on the underlying citation network.
    Average
    OpenAIRE UsageCounts
    Usage byUsageCounts
    visibility views 10
    download downloads 13
  • 10
    views
    13
    downloads
    Powered byOpenAIRE UsageCounts
Powered by OpenAIRE graph
Found an issue? Give us feedback
visibility
download
selected citations
These citations are derived from selected sources.
This is an alternative to the "Influence" indicator, which also reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically).
BIP!Citations provided by BIP!
popularity
This indicator reflects the "current" impact/attention (the "hype") of an article in the research community at large, based on the underlying citation network.
BIP!Popularity provided by BIP!
influence
This indicator reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically).
BIP!Influence provided by BIP!
impulse
This indicator reflects the initial momentum of an article directly after its publication, based on the underlying citation network.
BIP!Impulse provided by BIP!
views
OpenAIRE UsageCountsViews provided by UsageCounts
downloads
OpenAIRE UsageCountsDownloads provided by UsageCounts
0
Average
Average
Average
10
13
Green