publication . Preprint . 2018

Fast Weight Long Short-Term Memory

Keller, T. Anderson; Sridhar, Sharath Nittur; Wang, Xin;
Open Access English
  • Published: 17 Apr 2018
Abstract
Associative memory using fast weights is a short-term memory mechanism that substantially improves the memory capacity and time scale of recurrent neural networks (RNNs). As recent studies introduced fast weights only to regular RNNs, it is unknown whether fast weight memory is beneficial to gated RNNs. In this work, we report a significant synergy between long short-term memory (LSTM) networks and fast weight associative memories. We show that this combination, in learning associative retrieval tasks, results in much faster training and lower test error, a performance boost most prominent at high memory task difficulties.
Subjects
free text keywords: Computer Science - Neural and Evolutionary Computing, Computer Science - Learning
Download from
Powered by OpenAIRE Open Research Graph
Any information missing or wrong?Report an Issue