
Data Size Effects on Pre-Training Small-BERT (Part 1) This record pertains to Data Size Effects on Pre-Training Experiments. It includes the following files: small-binidx-0-ms-1234-ds-1234.tar.gz small-binidx-0-ms-1234-ds-2345.tar.gz small-binidx-0-ms-2345-ds-1234.tar.gz small-binidx-1-ms-1234-ds-1234.tar.gz small-binidx-1-ms-1234-ds-2345.tar.gz small-binidx-1-ms-2345-ds-1234.tar.gz small-binidx-2-ms-1234-ds-1234.tar.gz small-binidx-2-ms-1234-ds-2345.tar.gz small-binidx-2-ms-2345-ds-1234.tar.gz small-binidx-3-ms-1234-ds-1234.tar.gz small-binidx-3-ms-1234-ds-2345.tar.gz small-binidx-3-ms-2345-ds-1234.tar.gz Each tar file contains all model artifacts (checkpoints, random-number generator states, optimizer states etc.), training logs (Tensorboard, MLFlow and Weights & Biases), and evaluation results, configuration files, run scripts, SLURM sbatch driver scripts, and any additional artifacts generated during the experiments. Preprint: https://doi.org/10.48550/arXiv.2603.13627
