Attention Memory Patterns — What Models Actually Store in KV-Cache

Systematic analysis of attention memory patterns in transformer-based large language models, examining head specialization, attention sink phenomena, information density gradients across layers, and key-value redundancy patterns that inform cache compression strategies.

Related Organizations

Odessa National Polytechnic University
Ukraine

Keywords

transformer memory, attention heads, machine learning, research, KV-cache, AI, cache compression, attention patterns, large language models

Impact byBIP!

	selected citations These citations are derived from selected sources. This is an alternative to the "Influence" indicator, which also reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically).	0
	popularity This indicator reflects the "current" impact/attention (the "hype") of an article in the research community at large, based on the underlying citation network.	Average
	influence This indicator reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically).	Average
	impulse This indicator reflects the initial momentum of an article directly after its publication, based on the underlying citation network.	Average

Found an issue? Give us feedback

0

Average