Downloads provided by UsageCounts
One may maximize Shannon’s entropy -Sum over i f(ei) ln(f(ei)) subject to the constraint Sum over i ei f(ei) where f(ei) is the probability to have a gas particle with energy ei. This yields f(ei)= C(T)exp(-ei/T) from which one may calculate Eave. The result, however, is consistent with a canonical distribution, not a microcanonical one for which total energy = Eave is a strict constraint. For the canonical approach one considers weights exp(-Ei/T) where Ei may be larger than Eave. (Ei is the sum of ei’s.) In other words, one violates the original microcanonical constraint on total energy (for ease of calculation). This is consistent with Shannon’s entropy for which “information” is ln(f(ei)). If one thinks in terms of two body elastic scattering then f(e1)f(e2) = f(e3)f(e4) and e1+e2 = e3+e4. One may use any e1 and e2 even if their sum is greater than Eave the microcanonical constraint. Thus the use of information or Shannon’s entropy may violate original constraints in a problem. In this note we wish to examine an example (1) from information theory involving the compression of bits used to represent six letters A-F which have different probabilities of occurrence. Six letters may be encoded using 2x2x2 bits. This is an initial kind of constraint or bound i.e. one needs 3 bits for each letter. Given the probabilities one may calculate ln(probability) = information of each letter which represents the number of bits needed for each. The idea of information as ln(probability) is part of Shannon’s entropy. The problem is that like in the case of the canonical distribution violating the microcanonical constraint on total energy =Eave (which is done for ease of calculation), using information may violate an existing constraint or bound already in the problem. For example, for a low probability, ln(probability) may be larger than 3 the original bound or constraint used for the uniqueness of 6 letters. Using Shannon-Fano coding, a letter is coded with a number of bits close to ln(probability) which means that for ln(probability)>3, the number of bits needed is larger than the original bound or constraint. The idea, however, in information theory is to compress the number of bits. Furthermore there is no “ease of calculation” as in the case of the canonical distribution. We investigate this idea in this note.
violation of constraints, information
violation of constraints, information
| selected citations These citations are derived from selected sources. This is an alternative to the "Influence" indicator, which also reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically). | 0 | |
| popularity This indicator reflects the "current" impact/attention (the "hype") of an article in the research community at large, based on the underlying citation network. | Average | |
| influence This indicator reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically). | Average | |
| impulse This indicator reflects the initial momentum of an article directly after its publication, based on the underlying citation network. | Average |
| views | 10 | |
| downloads | 13 |

Views provided by UsageCounts
Downloads provided by UsageCounts