
We study whether the spectral sensitivity of the per-token empirical Fisher Information Matrix (FIM), λmax(Ft), can serve as a lightweight, model-agnostic runtime signal for anticipating hallucination during autoregressive decoding on consumer edge hardware. Across four pre-planned experiments on twelve open-weight models run locally via Ollama and MLX on an Apple M5 (32GB unified memory), we establish, in turn: cross-run stability of the signal, its predictive correlation with hallucination onset, its early-warning lead time, and the effectiveness of an intervention triggered by the resulting alarm. Concretely, ten of twelve models satisfy a stability gate (coefficient of variation < 0.20); the signal correlates strongly with an entropy-based hallucination-onset label for non-thinking-mode decoding, most strongly for qwen3:32b (r= 0.962, p= 1.4 ×10−28, median r= 0.479 across nine non-thinking models) – a correlation we report as an upper bound given a noted oracle-circularity caveat between the label and the signal itself; a first-passage-time alarm flags onset at a median lead time of 27 tokens (IQR 12–39) with an 85.6% detection rate (t= 33.2, p= 2.2 ×10−114); and truncating generation once the alarm fires reduces post-onset token count by 66% (Cohen’s d= 1.95) without measurable quality degradation. Separately, we report and analyze a 244×gap between the analytically derived and the empirically calibrated KL-divergence alarm threshold under 4-bit (Q4 K M) quantization on this hardware, corroborated by a multi-model robustness check (range 64–287× across eight models) – a finding we frame not as a universal constant but as evidence for a general Calibration Necessity Protocol: safety-relevant inference-time thresholds must be empirically recalibrated per hardware/quantization configuration rather than transferred analytically. We report this drift, a full experimental and hardware configuration, and a set of explicit limitations – including a top-K = 20 logprob constraint, single-hardware-platform scope, and an open correction-mechanism gap – to support independent replication and extension.
Replaced the term 'pre-registered' throughout with 'pre-planned' and added an explanatory note clarifying that no formal, timestamped registration was lodged prior to running the experiments. This follows feedback received during peer review, per TMLR's guidance that authors should adjust claims to match available evidence rather than remove the underlying finding.
Fisher information, runtime monitoring, large language models, quantization, hallucination detection, on-device inference
Fisher information, runtime monitoring, large language models, quantization, hallucination detection, on-device inference
| selected citations These citations are derived from selected sources. This is an alternative to the "Influence" indicator, which also reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically). | 0 | |
| popularity This indicator reflects the "current" impact/attention (the "hype") of an article in the research community at large, based on the underlying citation network. | Average | |
| influence This indicator reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically). | Average | |
| impulse This indicator reflects the initial momentum of an article directly after its publication, based on the underlying citation network. | Average |
