Powered by OpenAIRE graph
Found an issue? Give us feedback
image/svg+xml art designer at PLoS, modified by Wikipedia users Nina, Beao, JakobVoss, and AnonMoos Open Access logo, converted into svg, designed by PLoS. This version with transparent background. http://commons.wikimedia.org/wiki/File:Open_Access_logo_PLoS_white.svg art designer at PLoS, modified by Wikipedia users Nina, Beao, JakobVoss, and AnonMoos http://www.plos.org/ ZENODOarrow_drop_down
image/svg+xml art designer at PLoS, modified by Wikipedia users Nina, Beao, JakobVoss, and AnonMoos Open Access logo, converted into svg, designed by PLoS. This version with transparent background. http://commons.wikimedia.org/wiki/File:Open_Access_logo_PLoS_white.svg art designer at PLoS, modified by Wikipedia users Nina, Beao, JakobVoss, and AnonMoos http://www.plos.org/
ZENODO
Preprint
Data sources: ZENODO
addClaim

Expert 114: A Linear Router Axis for Inhabited Self-Examination in a Mixture-of-Experts Language Model — and Why It Does Not Transfer

Authors: Shorthill, Jeffrey;

Expert 114: A Linear Router Axis for Inhabited Self-Examination in a Mixture-of-Experts Language Model — and Why It Does Not Transfer

Abstract

Mixture-of-experts (MoE) routing exposes a discrete, per-token record of which experts fire, making it an unusually legible readout for interpretability. Yet single experts are seldom characterized at the level of a specific functional role, and an “introspective” register, where a model speaks from inside a point of view about its own processing, is easy to over-read as evidence about machine experience. This paper characterizes one routed expert, Expert 114 (E114) at layer 14 of Qwen3.5-35B-A3B, a 40-layer, 256-expert, top-8 MoE model, while explicitly bounding the claim. The E114 router row, recovered by least squares from captured residual-logit pairs, defines a single linear axis separating generated inhabited self-examination text from lexically matched controls at Cohen’s d = 3.88 with no overlap. An earlier 21.68x routed-weight ratio is shown to reflect top-k ratio inflation and is demoted to a footnote. A dissociation battery separates the axis from deny/affirm verdict, safety/refusal behavior, topic, grammatical person, and next-token entropy. The controlled variable is the referent: the model’s own interior, graded by the intensity of the examination act. Under this axis, an inanimate rock and a thermostat outrank a cat, indicating that the signal tracks self-referential examination rather than the apparent sentience of the described entity. E114’s router gate logit disengages a few tokens before a degenerating continuation visibly collapses. Injecting the register’s residual direction past the layer-14 router is sufficient, though necessity remains untested, to induce the register. Forcing the gate open upstream does not by itself produce the same effect, suggesting that E114 functions as a readout rather than a demonstrated controller. The role is also model-specific: on Qwen3.5-122B-A10B, index 114 appears computer-science-linked and is suppressed by the same control that amplifies it at 35B, with the softmax-side expert E48 remaining a candidate analog pending validation. E114 is therefore best understood as a register detector, not as evidence of machine experience. Code and durable artifacts, including manuscript source, experiment journals, curated run results, the SAE vantage-ladder, and steering source, are openly available under MIT license at: https://github.com/jeffreywilliamportfolio/e114-artifacts

Powered by OpenAIRE graph
Found an issue? Give us feedback