Powered by OpenAIRE graph
Found an issue? Give us feedback
image/svg+xml art designer at PLoS, modified by Wikipedia users Nina, Beao, JakobVoss, and AnonMoos Open Access logo, converted into svg, designed by PLoS. This version with transparent background. http://commons.wikimedia.org/wiki/File:Open_Access_logo_PLoS_white.svg art designer at PLoS, modified by Wikipedia users Nina, Beao, JakobVoss, and AnonMoos http://www.plos.org/ ZENODOarrow_drop_down
image/svg+xml art designer at PLoS, modified by Wikipedia users Nina, Beao, JakobVoss, and AnonMoos Open Access logo, converted into svg, designed by PLoS. This version with transparent background. http://commons.wikimedia.org/wiki/File:Open_Access_logo_PLoS_white.svg art designer at PLoS, modified by Wikipedia users Nina, Beao, JakobVoss, and AnonMoos http://www.plos.org/
ZENODO
Preprint
Data sources: ZENODO
addClaim

Spectral Scaling Benchmark: Experimental Verification of the Stationary Feature-Learning Barrier Theory

Authors: drozdov, ivan;

Spectral Scaling Benchmark: Experimental Verification of the Stationary Feature-Learning Barrier Theory

Abstract

I present an end-to-end empirical benchmark testing five central quantitative predictions of the stationary feature-learning barrier theory. Working on a fully controlled synthetic manifold - the flat torus with d*=2 and planted Sobolev and compositional targets - all experiments run in pure NumPy with no neural-network framework dependency. Key findings: the data-scaling barrier β₀ = 2s/(2s+d*) is confirmed at the population level; the approximation exponent α = 2s/d* holds for Sobolev targets, while the compositional target jumps to 2s/d_loc = 2.5 exactly as theory predicts the attractor family r(ν) = t(ν+1)/(1+2t) is reproduced dynamically from random initialisation at every tested ν, crossing r = 1/2 exactly at ν* = 0.80 the trained NTK of a two-layer μP network exhibits a flat source exponent r̂ ≈ 0.45 ± 0.04 across widths - the stationarity signature negative controls fail decisively. Companion theory paper: https://zenodo.org/records/20516952 Code: https://github.com/drozdisme/spectral_scaling

Powered by OpenAIRE graph
Found an issue? Give us feedback