Actions
  • shareshare
  • link
  • cite
  • add
add
Research data . Audiovisual . 2021 . Embargo end date: 01 Aug 2021

Rational LAMOL: A Rationale-based Lifelong Learning Framework

The Joint Conference of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing 2021; Horsuwan, Thanapapas; Kanwatchara, Kasidis;
Published: 01 Jan 2021
Publisher: Underline Science Inc.
Abstract

Read paper: https://www.aclanthology.org/2021.acl-long.229 Abstract: Lifelong learning (LL) aims to train a neural network on a stream of tasks while retaining knowledge from previous tasks. However, many prior attempts in NLP still suffer from the catastrophic forgetting issue, where the model completely forgets what it just learned in the previous tasks. In this paper, we introduce Rational LAMOL, a novel end-to-end LL framework for language models. In order to alleviate catastrophic forgetting, Rational LAMOL enhances LAMOL, a recent LL model, by applying critical freezing guided by human rationales. When the human rationales are not available, we propose exploiting unsupervised generated rationales as substitutions. In the experiment, we tested Rational LAMOL on permutations of three datasets from the ERASER benchmark. The results show that our proposed framework outperformed vanilla LAMOL on most permutations. Furthermore, unsupervised rationale generation was able to consistently improve the overall LL performance from the baseline without relying on human-annotated rationales.

Subjects

Condensed Matter Physics, FOS: Physical sciences, Deep Learning, Semantics, Information and Knowledge Engineering, Neural Network, Electromagnetism, Computational Linguistics

Download from
question_mark
moresidebar