
Standard transformer attention treats all tokens as epistemically interchangeable, computing relevance purely through semantic similarity of query and key vectors. This architectural design discards a critical signal in source-critical domains: the provenance type of the information being attended to. We introduce Provenance-Conditioned Attention (PCA), a mechanism that augments scaled dot-product attention with a learned, low-dimensional source-type gating channel. Each token carries a provenance embedding indicating its epistemic category (e.g., testimony, archival record, scholarly analysis), and a compatibility function over these embeddings multiplicatively gates attention flow. We present three architectural variants: multiplicative gating, additive score fusion, and head-partitioned attention. PCA adds approximately 0.01% additional parameters at target model scale and recovers standard attention as a special case. We validate PCA on three synthetic benchmarks of increasing difficulty across six training scales (500–20,000 examples) with five random seeds per configuration. On single-source retrieval, PCA matches a segment-embedding baseline from 2,000 training examples onward. On a compositional multi-hop task requiring simultaneous attention to two source types, all PCA variants achieve perfect accuracy (1.000 ± 0.000), compared to 0.921 ± 0.158 for segment embeddings and 0.181 ± 0.010 for standard attention. PCA addresses a gap orthogonal to recent work on epistemic uncertainty in attention: not "how certain am I?" but "what kind of knowing is this?"
Code for all experiments is available upon request. Experiments were conducted on a single NVIDIA T4 GPU via Kaggle with a total runtime of approximately 71 minutes.
attention mechanism transformer architecture epistemic provenance source attribution inductive bias multi-document reasoning natural language processing digital humanities genocide studies
attention mechanism transformer architecture epistemic provenance source attribution inductive bias multi-document reasoning natural language processing digital humanities genocide studies
| selected citations These citations are derived from selected sources. This is an alternative to the "Influence" indicator, which also reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically). | 0 | |
| popularity This indicator reflects the "current" impact/attention (the "hype") of an article in the research community at large, based on the underlying citation network. | Average | |
| influence This indicator reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically). | Average | |
| impulse This indicator reflects the initial momentum of an article directly after its publication, based on the underlying citation network. | Average |
