Powered by OpenAIRE graph
Found an issue? Give us feedback
ZENODOarrow_drop_down
ZENODO
Preprint . 2026
License: CC BY
Data sources: Datacite
ZENODO
Preprint . 2026
License: CC BY
Data sources: Datacite
ZENODO
Preprint . 2026
License: CC BY
Data sources: Datacite
addClaim

Curriculum Order and the Geometry of Learned Representations: The Basque First Hypothesis

Authors: Eichler, David;

Curriculum Order and the Geometry of Learned Representations: The Basque First Hypothesis

Abstract

The field of large language model development has optimized aggressively along two axes: scale of compute and volume of training data. This paper proposes a third axis, almost entirely unexplored at frontier model scale: training language curriculum order. We argue that English-first pretraining — the de facto standard — builds reasoning capacity on a morphologically impoverished foundation, encoding argument structure through positional heuristics rather than explicit grammatical representation. We introduce the Basque First hypothesis as a proof of concept: that beginning pretraining with typologically distant, morphologically rich languages forces the construction of richer internal representations of predicate-argument structure before any English token is encountered. We then extend this argument to its logical conclusion — a purpose-built synthetic training language designed to encode logical structure, evidentiality, quantifier scope, and predicate-argument relations without the historical accidents and ambiguities of natural language. This synthetic substrate, invisible in the model’s outputs but structural in its weights, functions as what we term the dark energy of the model: inferred not from observation but from the coherence and reasoning quality it produces. Recent interpretability work — Anthropic’s finding that verbalizable representations form a global workspace (the J-space) within language models — supplies both a coordinate system for this claim, since the substrate’s influence is predicted to reside in the non-verbalizable complement of that workspace, and a measurement instrument for testing it. Finally, we identify an emergent flywheel: AI systems trained on such substrates will generate formal proofs, structured arguments, and logical derivations that become the highest-quality training corpora for the next generation of models — compounding reasoning quality independent of raw scale.

Keywords

computational linguistics, Basque, morphology, cs.CL, language model pretraining, NLP

  • BIP!
    Impact byBIP!
    selected citations
    These citations are derived from selected sources.
    This is an alternative to the "Influence" indicator, which also reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically).
    0
    popularity
    This indicator reflects the "current" impact/attention (the "hype") of an article in the research community at large, based on the underlying citation network.
    Average
    influence
    This indicator reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically).
    Average
    impulse
    This indicator reflects the initial momentum of an article directly after its publication, based on the underlying citation network.
    Average
Powered by OpenAIRE graph
Found an issue? Give us feedback
selected citations
These citations are derived from selected sources.
This is an alternative to the "Influence" indicator, which also reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically).
BIP!Citations provided by BIP!
popularity
This indicator reflects the "current" impact/attention (the "hype") of an article in the research community at large, based on the underlying citation network.
BIP!Popularity provided by BIP!
influence
This indicator reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically).
BIP!Influence provided by BIP!
impulse
This indicator reflects the initial momentum of an article directly after its publication, based on the underlying citation network.
BIP!Impulse provided by BIP!
0
Average
Average
Average
Upload OA version
Are you the author of this publication? Upload your Open Access version to Zenodo!
It’s fast and easy, just two clicks!