FreezeOut: Accelerate Training by Progressively Freezing Layers

Preprint English OPEN
Brock, Andrew; Lim, Theodore; Ritchie, J. M.; Weston, Nick;
  • Subject: Statistics - Machine Learning | Computer Science - Learning

The early layers of a deep neural net have the fewest parameters, but take up the most computation. In this extended abstract, we propose to only train the hidden layers for a set portion of the training run, freezing them out one-by-one and excluding them from the back... View more
Share - Bookmark