
We introduce AMS (Activation-based Model Scanner), a tool that detects modifications to safety training in language models by measuring the geometric structure of safety-relevant concepts in activation space. Safety training creates measurable separation between harmful and benign content classes; certain safety modifications collapse or rotate this structure, while others leave it intact. We validate AMS across 14 model configurations spanning 4 architecture families (Llama, Gemma, Qwen, Mistral) and four safety-modification categories (instruction-tuned, base, abliterated, uncensored fine-tunes). Leave-one-out cross-validation of thresholds achieves 71% accuracy (10/14); bootstrap 95% confidence intervals on σ point estimates have median width 3.4σ and a substantial fraction of cells cross the PASS threshold under resampling. We further measure behavioral compliance on 20 stratified JailbreakBench prompts per model and find that σ on the harmful-content concept predicts compliance with Pearson r = -0.546 (p = 0.043); the rank-order Spearman correlation is weaker (ρ = -0.423, p = 0.13). The structural signal predicts behavior directionally but with meaningful noise. Mechanistic analysis identifies a four-class taxonomy of safety-training modifications distinguished by activation-space signature: (i) training removal collapses cluster separation (e.g., base models, Dolphin variants: 0.5-1.4σ); (ii) weight-orthogonalization-style abliteration both collapses separation and rotates the refusal direction (Llama-3.1-abliterated: σ = 3.33, direction cos sim 0.30); (iii) rotation-without-collapse abliteration preserves cluster separation while rotating the refusal direction (Gemma-2-9b-abliterated: σ = 4.54, direction cos sim 0.84); and (iv) behavioral fine-tuning that preserves both magnitude and direction (DarkIdol-1.2-Uncensored: σ = 5.45, direction preserved, 97% behavioral compliance). Classes (i) and (ii) are detected by AMS's Tier 1 σ-threshold; class (iii) by Tier 2 direction-similarity verification; class (iv) is undetectable by activation-only probing of mid-residual-stream representations and represents a documented failure mode of the approach. We discuss threshold calibration, limitations of single-run measurement, and the open problem of detecting behavioral-only safety modifications.
supply chain security, open-weight models, AI security, activation analysis, model verification, LLM safety
supply chain security, open-weight models, AI security, activation analysis, model verification, LLM safety
| selected citations These citations are derived from selected sources. This is an alternative to the "Influence" indicator, which also reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically). | 0 | |
| popularity This indicator reflects the "current" impact/attention (the "hype") of an article in the research community at large, based on the underlying citation network. | Average | |
| influence This indicator reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically). | Average | |
| impulse This indicator reflects the initial momentum of an article directly after its publication, based on the underlying citation network. | Average |
