Powered by OpenAIRE graph
Found an issue? Give us feedback
image/svg+xml art designer at PLoS, modified by Wikipedia users Nina, Beao, JakobVoss, and AnonMoos Open Access logo, converted into svg, designed by PLoS. This version with transparent background. http://commons.wikimedia.org/wiki/File:Open_Access_logo_PLoS_white.svg art designer at PLoS, modified by Wikipedia users Nina, Beao, JakobVoss, and AnonMoos http://www.plos.org/ ZENODOarrow_drop_down
image/svg+xml art designer at PLoS, modified by Wikipedia users Nina, Beao, JakobVoss, and AnonMoos Open Access logo, converted into svg, designed by PLoS. This version with transparent background. http://commons.wikimedia.org/wiki/File:Open_Access_logo_PLoS_white.svg art designer at PLoS, modified by Wikipedia users Nina, Beao, JakobVoss, and AnonMoos http://www.plos.org/
ZENODO
Software
Data sources: ZENODO
addClaim

trishasalas/thatDangCircuit: thatDangCircuit v1.0.0 — compound binding is distributed

Authors: Trisha Salas;

trishasalas/thatDangCircuit: thatDangCircuit v1.0.0 — compound binding is distributed

Abstract

First archived release. A mechanistic interpretability study of noun-noun compound binding in GPT-2 (small through XL) and Pythia (1B, 2.8B), reporting negative localization results: binding capacity is distributed and redundant, not circuit-localized. Compound terms span four professional domains — medical, legal, finance, and accessibility — with matched weak-collocation controls. Three converging lines of evidence establish the finding. Head ablation: removing the six strongest binding heads (the "universal six") shifts mean binding by approximately 1%, with compensator heads absorbing the loss; mean ablation reproduces the null, and domain-specific head triples fail to selectively suppress binding within their own domain (<1.5% in every case). Residual-stream cosine analysis: compound and control representations diverge across a broad mid-network band rather than at a single layer, with no point of clean single-direction separability. Projection steering: a candidate binding direction constructed at L33 from 176 compounds against controls, swept from suppression through 10× amplification against a random-direction null, produces a U-shaped degradation centered on the identity and no directional effect — the pre-registered prediction (amplification raises binding, suppression lowers it) was falsified. The direction is real, and specific late-layer heads read from it, but it exerts no functional control over binding. The negative result carries a positive interpretive claim: the redundancy and resilience are themselves the finding. Binding emerges from training as a distributed property of the network — many partial paths to the same output — and "where is the circuit?" is the wrong shape of question for this capability in these models. The canonical write-up, with per-head evidence and full tables, is analysis/findings.md. Methodological decisions are dated and recorded in DECISIONS.md. Analyses are reproducible with the pinned instrument (transformer-lens==2.17.0; the pin is load-bearing — see DECISIONS 2026-07-04). The steering notebook is generated from append_steering_cells.py and should be rebuilt, not hand-edited. Falsified predictions are reported at full strength by policy. Code is MIT-licensed; data, results, figures, and documentation are CC BY 4.0.

Powered by OpenAIRE graph
Found an issue? Give us feedback