Powered by OpenAIRE graph
Found an issue? Give us feedback
image/svg+xml art designer at PLoS, modified by Wikipedia users Nina, Beao, JakobVoss, and AnonMoos Open Access logo, converted into svg, designed by PLoS. This version with transparent background. http://commons.wikimedia.org/wiki/File:Open_Access_logo_PLoS_white.svg art designer at PLoS, modified by Wikipedia users Nina, Beao, JakobVoss, and AnonMoos http://www.plos.org/ ZENODOarrow_drop_down
image/svg+xml art designer at PLoS, modified by Wikipedia users Nina, Beao, JakobVoss, and AnonMoos Open Access logo, converted into svg, designed by PLoS. This version with transparent background. http://commons.wikimedia.org/wiki/File:Open_Access_logo_PLoS_white.svg art designer at PLoS, modified by Wikipedia users Nina, Beao, JakobVoss, and AnonMoos http://www.plos.org/
ZENODO
Dataset
Data sources: ZENODO
addClaim

Full Raw Evidence and Pinned Inputs for Early Gradient Spectra Predict Useful Low-Rank Adaptation

Authors: Lin, Ruge;

Full Raw Evidence and Pinned Inputs for Early Gradient Spectra Predict Useful Low-Rank Adaptation

Abstract

Full raw experiment releases, pinned GPT-2/WikiText-2 validation inputs, andStage4 source sidecar for the project "Early Gradient Spectra Predict UsefulLow-Rank Adaptation." This data record contains:- Stage4 raw release archive- synthetic-transformer raw release archive- real-LoRA raw release archive- pinned real-model input archive for GPT-2/WikiText-2 validation- Stage4 source sidecar at the recorded source commit- manifests, file-size records, and SHA-256 checksums The companion software/code release is archived separately. The central claim is that activation-whitened early-gradient spectra predictuseful LoRA rank in a reduced-rank population model and controlled spikedmatrix simulations. Transformer allocation evidence is bounded: thesynthetic-transformer allocation test is null under its primary condition, andthe GPT-2/WikiText-2 result is restricted to the plan-locked c_attn/c_fc modulesuite.Keywords:LoRA; low-rank adaptation; PEFT; transformer fine-tuning; early gradient spectra;activation whitening; effective rank; useful rank; rank allocation; random matrixtheory; GPT-2; WikiText-2; reproducible machine learning

Powered by OpenAIRE graph
Found an issue? Give us feedback