
PALA (Plateau-Aware Learning with Adaptive Memory) matches ReduceLROnPlateau in final accuracywhile using 25% fewer training epochs, reaching competitive accuracy 13 epochs sooner and early-stopping ata mean of 75 epochs versus 100 for all fixed-schedule baselines. This convergence efficiency is PALA’s primarypractical contribution: the same validation accuracy for a quarter less GPU compute, with no schedule-specifichyperparameter tuning.PALA achieves this by diagnosing why a model has stalled, not merely that it has stalled, and respondingwith a geometry-adapted escape strategy. It reads the first and second Adam moments already maintained bythe optimizer at zero additional cost: the gradient signal-to-noise ratio SNR = ∥E[g]∥/∥E[g2]1/2∥ and thegradient variance proxy GVar = E[∥v1/2t ∥] are jointly mapped through a threshold classifier to one of threelandscape categories (SADDLE, FLAT, or SHARP), each triggering a type-specific cosine learning-rate bounce. Aregion-of-interest (ROI) memory accumulates (floor_lr, loss_gain, plateau_type) triples acrossbounces; the next floor is selected by a recency–ROI weighted mean over type-matched entries. Adaptive β2scaling accelerates second-moment adaptation during bounces, and a β1 reduction on saddle exits increasesgradient direction reactivity. A gradient-agreement detector escalates bounce amplitude when the optimizerre-enters the same trap. On CIFAR-100 (PalaCNN) and STL-10, PALA outperforms plain Adam by +3.8 pp onaverage and matches ReduceLROnPlateau to within 0.2 pp in final accuracy. Our ablation further reveals thata more aggressive floor decay (γ=0.1) yields 63.3%, surpassing ReduceLROnPlateau (62.0%) by 1.3 pp.On ResNet-18, PALA underperforms in the 100-epoch regime due to BatchNorm’s suppression of gradientvariance signals.
