Powered by OpenAIRE graph
Found an issue? Give us feedback
image/svg+xml art designer at PLoS, modified by Wikipedia users Nina, Beao, JakobVoss, and AnonMoos Open Access logo, converted into svg, designed by PLoS. This version with transparent background. http://commons.wikimedia.org/wiki/File:Open_Access_logo_PLoS_white.svg art designer at PLoS, modified by Wikipedia users Nina, Beao, JakobVoss, and AnonMoos http://www.plos.org/ ZENODOarrow_drop_down
image/svg+xml art designer at PLoS, modified by Wikipedia users Nina, Beao, JakobVoss, and AnonMoos Open Access logo, converted into svg, designed by PLoS. This version with transparent background. http://commons.wikimedia.org/wiki/File:Open_Access_logo_PLoS_white.svg art designer at PLoS, modified by Wikipedia users Nina, Beao, JakobVoss, and AnonMoos http://www.plos.org/
ZENODO
Preprint . 2026
License: CC BY
Data sources: ZENODO
ZENODO
Preprint . 2026
License: CC BY
Data sources: Datacite
ZENODO
Preprint . 2026
License: CC BY
Data sources: Datacite
versions View all 2 versions
addClaim

Semantic Entropy and Structural Invariance in LLM-Mediated ExpansionCompression Loops

Authors: Han, Huiwen;

Semantic Entropy and Structural Invariance in LLM-Mediated ExpansionCompression Loops

Abstract

We develop a quantitative information-theoretic account of semantic decay in large-language-model (LLM) mediated Expansion--Compression (EC) loops.Building on the unified framework of DECO Paper~0 \citep{deco_paper0},we introduce semantic entropy $\HS(X)$ as the differential entropy of a random variable distributed over a semantic manifold, and prove that each application of the EC-transform $T = \C \circ \E$ is a strictly entropy-reducing operation in expectation (Semantic Entropy CollapseTheorem). We derive closed-form bounds on the mutual information $I\!\left(K_A;\,\hat{K}_B\right)$ between the originating ideas $K_A$ and the recipient's extraction $\hat{K}_B$ as functions of expansion ratio $\rho_E$, compression ratio $\rho_C$, LLM temperature $\tau$, and iteration count $n$. We introduce the \emph{Semantic Gravity Well} model: a potential landscape on the semantic manifold in which the LLM training centroid acts as an attractor, and original content resides at unstable high-curvature saddle points. Under this model, we prove the Differential Decay Theorem: propositional fidelity $F_{\mathrm{prop}}$ decays exponentially in $n$, affective fidelity $F_{\mathrm{aff}}$ decays sub-exponentially, and structural fidelity $F_{\mathrm{str}}$ converges to a strictly positive constant (the causal skeleton invariant) under mild conditions.Finally, we characterise the Semantic Channel Capacity of the EC-loop and show it is strictly less than the Shannon capacity of the raw linguistic channel, with the gap determined by the curvature of the semantic manifold at the seed point. These results quantify the information-theoretic cost of AI-mediated cognitive decoupling and provide empirically testable predictions for the subsequent papers in the series.

Keywords

semantic entropy, semantic manifold, channel capacity, LLM-mediated communication, information decay, cognitive decoupling, expansion--compression loop, structural invariance, mutual information

  • BIP!
    Impact byBIP!
    selected citations
    These citations are derived from selected sources.
    This is an alternative to the "Influence" indicator, which also reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically).
    0
    popularity
    This indicator reflects the "current" impact/attention (the "hype") of an article in the research community at large, based on the underlying citation network.
    Average
    influence
    This indicator reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically).
    Average
    impulse
    This indicator reflects the initial momentum of an article directly after its publication, based on the underlying citation network.
    Average
Powered by OpenAIRE graph
Found an issue? Give us feedback
selected citations
These citations are derived from selected sources.
This is an alternative to the "Influence" indicator, which also reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically).
BIP!Citations provided by BIP!
popularity
This indicator reflects the "current" impact/attention (the "hype") of an article in the research community at large, based on the underlying citation network.
BIP!Popularity provided by BIP!
influence
This indicator reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically).
BIP!Influence provided by BIP!
impulse
This indicator reflects the initial momentum of an article directly after its publication, based on the underlying citation network.
BIP!Impulse provided by BIP!
0
Average
Average
Average