Semantic Attention(SMAT): Attention Gated Through Learned Similarity and Hidden State Centrality
Log, Any;
Semantic Attention(SMAT): Attention Gated Through Learned Similarity and Hidden State Centrality
SMAT adds a learned semantic similarity matrix and a centrality-derived value gate to transformer attention. The gate is load-bearing, removing it collapses performance. Semantic information acts through gating, not direct attention biasing.