Powered by OpenAIRE graph
Found an issue? Give us feedback
image/svg+xml art designer at PLoS, modified by Wikipedia users Nina, Beao, JakobVoss, and AnonMoos Open Access logo, converted into svg, designed by PLoS. This version with transparent background. http://commons.wikimedia.org/wiki/File:Open_Access_logo_PLoS_white.svg art designer at PLoS, modified by Wikipedia users Nina, Beao, JakobVoss, and AnonMoos http://www.plos.org/ ZENODOarrow_drop_down
image/svg+xml art designer at PLoS, modified by Wikipedia users Nina, Beao, JakobVoss, and AnonMoos Open Access logo, converted into svg, designed by PLoS. This version with transparent background. http://commons.wikimedia.org/wiki/File:Open_Access_logo_PLoS_white.svg art designer at PLoS, modified by Wikipedia users Nina, Beao, JakobVoss, and AnonMoos http://www.plos.org/
ZENODO
External research report
Data sources: ZENODO
addClaim

Semantic Compression Work

Authors: Wu, Jifeng;

Semantic Compression Work

Abstract

We introduce Semantic Compression Work: the drop in the next-token Shannon entropy of a language model across a token sequence. Each sequence either compresses the model's uncertainty (a funnel), leaves it approximately unchanged (equilibrium), or expands it (a fan-out). The framework treats the model's probability space not as a black box producing outputs, but as a measurable system whose uncertainty evolves sequence by sequence.

Powered by OpenAIRE graph
Found an issue? Give us feedback