
Byte-Pair Encoding tokenization is statistically efficient for vocabulary compression, but semantically blind to structured technical entities, fragmenting physical quantities, numbers, units, and symbolic expressions into lexically arbitrary subwords. We present TOTEN, a knowledge-based ontological tokenization framework that replaces statistical derivation with declarative classification grounded in a formal ontology of engineering entities (OEE). We formalize TOTEN as the triple ⟨O, classify, {instτ }⟩: the ontology gathers types, structural principles, composition relations, and preservable invariants; the classification function maps raw text into typed regions; and the indexed family of instantiators produces a self-descriptive structured representation. Robustness derives from deterministic coupling with three consolidated external oracles — Pint (dimensional), Unicode Character Database (typographic), and RSLP (Portuguese morphology). The intrinsic evaluation covers four properties verifiable by construction—ontological atomicity, dimensional equivalence, typographic robustness, and numerical reconstruction — over an internally generated and physically validated benchmark (EngQuant, N = 800) and four external corpora in Brazilian Portuguese (N = 1 771 cases eligible for numerical reconstruction). We additionally report detection recall, distinguishing coverage from conditional atomicity. Compared to eight representative state-of-the-art systems, TOTEN achieves unit ontological atomicity in all contrasts and numerical reconstruction of 0.775 to 0.904 on external corpora, against 0.627 0.703 for the best baseline (Quantulum3); on the internal benchmark, 0.780 against 0.340. Differences in atomicity and reconstruction are statistically significant (McNemar with Holm correction). The Spearman rank correlation between internal and external corpus rankings confirms the concurrent validity of the control benchmark. Dimensional equivalence shows statistical parity with Pint, the oracle from which the system inherits dimensional authority.
| selected citations These citations are derived from selected sources. This is an alternative to the "Influence" indicator, which also reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically). | 0 | |
| popularity This indicator reflects the "current" impact/attention (the "hype") of an article in the research community at large, based on the underlying citation network. | Average | |
| influence This indicator reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically). | Average | |
| impulse This indicator reflects the initial momentum of an article directly after its publication, based on the underlying citation network. | Average |
