
We identify an internal signal that detects impending grokking ∼3,000 training stepsbefore the accuracy transition without any knowledge of the learned algorithm. Thesignal is a non-monotonic re-rise of inter-layer CKA (Ob01, between layers 0 and 1 ofa small transformer): after an initial drop during memorization, Ob01 begins rising again atstep ≈750 exclusively in models that will grokk. This re-rise discriminates grokking fromnon-grokking in 16/16 vs 0/5 seeds (Fisher exact p = 0.000049; one-sample t = 4.51,p < 0.001), with onset consistently ∼3,297 ± 461 steps before the accuracy jump. The signal replicates across two primes (p = 97, p = 113) and across a structurally distinct modulartask: modular multiplication (6/6 seeds, onset step 750, lead 3,083±186 steps) identical onset to addition despite dierent underlying algorithms. A task-contrast experimentwith k-sparse parity reveals the boundary of the signal's scope: parity generalizes quicklybut Ob01 drops rather than rises the opposite dynamic. This demarcates a meaningfuldistinction: the re-rise is a signature of tasks requiring distributed inter-layer coordination (Fourier-type algorithms), not of grokking or generalization in general. Practically,the re-rise onset provides an early-termination criterion within the modular arithmetic family: runs showing no re-rise by step 1,000 do not grokk in any tested conguration, enablingdiagnosis thousands of steps before any test accuracy signal appears. We motivate Ob01 viaO(X, Y ) = I(X; Y )/H(Y ) and hierarchical predictive coding theory, while being explicitthat CKA is a linear proxy for O, not an estimator of it. Replacing CKA with MINE orkNN MI estimators is the priority follow-up.
Fourier algorithm, grokking,, mechanistic interpretability,, centered kernel alignment,, inter-layer coordination,, CKA,, emergent generalization,, utual information,, transformers,, MINE,, modular arithmetic,, utual information,
Fourier algorithm, grokking,, mechanistic interpretability,, centered kernel alignment,, inter-layer coordination,, CKA,, emergent generalization,, utual information,, transformers,, MINE,, modular arithmetic,, utual information,
| selected citations These citations are derived from selected sources. This is an alternative to the "Influence" indicator, which also reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically). | 0 | |
| popularity This indicator reflects the "current" impact/attention (the "hype") of an article in the research community at large, based on the underlying citation network. | Average | |
| influence This indicator reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically). | Average | |
| impulse This indicator reflects the initial momentum of an article directly after its publication, based on the underlying citation network. | Average |
