Powered by OpenAIRE graph
Found an issue? Give us feedback
image/svg+xml art designer at PLoS, modified by Wikipedia users Nina, Beao, JakobVoss, and AnonMoos Open Access logo, converted into svg, designed by PLoS. This version with transparent background. http://commons.wikimedia.org/wiki/File:Open_Access_logo_PLoS_white.svg art designer at PLoS, modified by Wikipedia users Nina, Beao, JakobVoss, and AnonMoos http://www.plos.org/ ZENODOarrow_drop_down
image/svg+xml art designer at PLoS, modified by Wikipedia users Nina, Beao, JakobVoss, and AnonMoos Open Access logo, converted into svg, designed by PLoS. This version with transparent background. http://commons.wikimedia.org/wiki/File:Open_Access_logo_PLoS_white.svg art designer at PLoS, modified by Wikipedia users Nina, Beao, JakobVoss, and AnonMoos http://www.plos.org/
ZENODO
Software
Data sources: ZENODO
addClaim

Transport, Not Computation: A Cross-Model Falsification Program on How 1B-3B Language Models Use Long Context

Authors: Vyaltsev, Evgenii; Vyaltsev, Daniil;

Transport, Not Computation: A Cross-Model Falsification Program on How 1B-3B Language Models Use Long Context

Abstract

An umbrella review of a program of pre-registered, matched-control causal experiments on small open language models (Llama-3.2-1B/3B, Qwen2.5-3B). After establishing that long-context retention has no critical key budget (the sparse compressor is not the limit), four independent causal gates - on attention, on the repair of retrieval-disambiguation failure, on multi-hop synthesis, and on a directly-optimized latent state - converge on one regime-bounded thesis: the network delivers and composes only through an external buffer; it does not latently compute, and a latent computation state cannot be optimized into existence. The contribution is a map of the boundary of internal competence, a training-free long-context engine plus a 27-byte micro-hint memory architecture, and a falsification methodology that reversed its own false positives.

Powered by OpenAIRE graph
Found an issue? Give us feedback