Downloads provided by UsageCountsPlease grant OpenAIRE to access and update your ORCID works.
This Research product is the result of merged Research products in OpenAIRE.
You have already added 0 works in your ORCID record related to the merged Research product.
You have already added 0 works in your ORCID record related to the merged Research product.
This Research product is the result of merged Research products in OpenAIRE.
You have already added 0 works in your ORCID record related to the merged Research product.
You have already added 0 works in your ORCID record related to the merged Research product.
Adapting Multilingual Neural Machine Translation to Unseen Languages
Lakew, Surafel M.; Karakanta, Alina; Federico, Marcello; Negri, Matteo; Turchi, Marco;
Lakew, Surafel M.; Karakanta, Alina; Federico, Marcello; Negri, Matteo; Turchi, Marco;
Adapting Multilingual Neural Machine Translation to Unseen Languages
Multilingual Neural Machine Translation (MNMT) for low-resource languages (LRL) can be enhanced by the presence of related high-resource languages (HRL), but the relatedness of HRL usually relies on predefined linguistic assumptions about language similarity. Recently, adapting MNMT to a LRL has shown to greatly improve performance. In this work, we explore the problem of adapting an MNMT model to an unseen LRL using data selection and model adaptation. In order to improve NMT for LRL, we employ perplexity to select HRL data that are most similar to the LRL on the basis of language distance. We extensively explore data selection in popular multilingual NMT settings, namely in (zero-shot) translation, and in adaptation from a multilingual pre-trained model, for both directions (LRL-en). We further show that dynamic adaptation of the model's vocabulary results in a more favourable segmentation for the LRL in comparison with direct adaptation. Experiments show reductions in training time and significant performance gains over LRL baselines, even with zero LRL data (+13.0 BLEU), up to +17.0 BLEU for pre-trained multilingual model dynamic adaptation with related data selection. Our method outperforms current approaches, such as massively multilingual models and data augmentation, on four LRL.
Comment: Accepted at the 16th International Workshop on Spoken Language Translation (IWSLT), November, 2019
- Fondazione Bruno Kessler Italy
Computation and Language (cs.CL), FOS: Computer and information sciences, Computer Science - Computation and Language
Computation and Language (cs.CL), FOS: Computer and information sciences, Computer Science - Computation and Language
3314
[3] M. Johnson, M. Schuster, Q. V. Le, M. Krikun, Y. Wu, Z. Chen, N. Thorat, F. Vie´gas, M. Wattenberg, G. Corrado, M. Hughes, and J. Dean, “Google's Multilingual Neural Machine Translation System: Enabling Zero-Shot Translation,” Transactions of the Association for Computational Linguistic, vol. 5, pp. 339-351, 2017. [Online]. Available: https: //aclweb.org/anthology/Q/Q17/Q17-1024.pdf
[4] A. Axelrod, X. He, and J. Gao, “Domain adaptation via pseudo in-domain data selection,” in Proceedings of the 2011 Conference on Empirical Methods in Natural Language Processing, 2011, pp. 355-362. [Online]. Available: http://aclweb.org/anthology/D11-1033
[5] M. v. d. Wee, A. Bisazza, and C. Monz, “Dynamic Data Selection for Neural Machine Translation,” in Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, 2017, pp. 1400-1410. [Online]. Available: https://aclweb.org/anthology/D17-1147
[6] R. Aharoni, M. Johnson, and O. Firat, “Massively Multilingual Neural Machine Translation,” in Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers). Minneapolis, Minnesota: Association for Computational Linguistics, June 2019, pp. 3874- 3884.
[7] G. Neubig and J. Hu, “Rapid Adaptation of Neural Machine Translation to New Languages,” in Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, 2018, pp. 875-880. [Online]. Available: https://aclweb.org/anthology/D18-1103
[8] X. Wang, H. Pham, P. Arthur, and G. Neubig, “Multilingual Neural Machine Translation With Soft Decoupled Encoding,” in Proceedings of the 7th International Conference on Learning Representations, 2019.
[9] M. Xia, X. Kong, A. Anastasopoulos, and G. Neubig, “Generalized Data Augmentation for Low-Resource Translation,” in Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. Florence, Italy: Association for Computational Linguistics, July 2019, pp. 5786-5796.
[10] P. Gamallo, J. R. Pichel, and I. Alegria, “From language identification to language distance,” Physica A: Statistical Mechanics and its Applications, vol. 484, pp. 152- 162, 2017.
[11] J. Gu, H. Hassan, J. Devlin, and V. O. Li, “Universal neural machine translation for extremely low resource languages,” in Proceedings of NAACL-HLT 2018. New Orleans, Louisiana: Association for Computational Linguistics, 2018, pp. 344-354. [Online]. Available: https://aclweb.org/anthology/N18-1032
[12] S. M. Lakew, A. Erofeeva, M. Negri, M. Federico, and M. Turchi, “Transfer Learning in Multilingual Neural Machine Translation with Dynamic Vocabulary,” in Proceedings of the International Workshop on Spoken Language Translation (IWSLT), 2018, 2018, pp. 54- 61. [Online]. Available: https://workshop2018.iwslt. org/downloads/Proceedings IWSLT 2018.pdf
citations This is an alternative to the "Influence" indicator, which also reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically).4 popularity This indicator reflects the "current" impact/attention (the "hype") of an article in the research community at large, based on the underlying citation network.Average influence This indicator reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically).Average impulse This indicator reflects the initial momentum of an article directly after its publication, based on the underlying citation network.Average views 158 downloads 107 citations This is an alternative to the "Influence" indicator, which also reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically).4 popularity This indicator reflects the "current" impact/attention (the "hype") of an article in the research community at large, based on the underlying citation network.Average influence This indicator reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically).Average impulse This indicator reflects the initial momentum of an article directly after its publication, based on the underlying citation network.Average - 158views107downloads
citations
Citations provided by BIP!
This is an alternative to the "Influence" indicator, which also reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically).
Citations provided by BIP!popularityPopularity provided by BIP!
This indicator reflects the "current" impact/attention (the "hype") of an article in the research community at large, based on the underlying citation network.
Popularity provided by BIP!influenceInfluence provided by BIP!
This indicator reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically).
Influence provided by BIP!impulseImpulse provided by BIP!
This indicator reflects the initial momentum of an article directly after its publication, based on the underlying citation network.
Impulse provided by BIP!viewsViews provided by UsageCounts
Views provided by UsageCountsdownloadsDownloads provided by UsageCounts
Downloads provided by UsageCounts 4
Average
Average
Average
158
107
Do the share buttons not appear? Please make sure, any blocking addon is disabled, and then reload the page.
- Fondazione Bruno Kessler Italy
Multilingual Neural Machine Translation (MNMT) for low-resource languages (LRL) can be enhanced by the presence of related high-resource languages (HRL), but the relatedness of HRL usually relies on predefined linguistic assumptions about language similarity. Recently, adapting MNMT to a LRL has shown to greatly improve performance. In this work, we explore the problem of adapting an MNMT model to an unseen LRL using data selection and model adaptation. In order to improve NMT for LRL, we employ perplexity to select HRL data that are most similar to the LRL on the basis of language distance. We extensively explore data selection in popular multilingual NMT settings, namely in (zero-shot) translation, and in adaptation from a multilingual pre-trained model, for both directions (LRL-en). We further show that dynamic adaptation of the model's vocabulary results in a more favourable segmentation for the LRL in comparison with direct adaptation. Experiments show reductions in training time and significant performance gains over LRL baselines, even with zero LRL data (+13.0 BLEU), up to +17.0 BLEU for pre-trained multilingual model dynamic adaptation with related data selection. Our method outperforms current approaches, such as massively multilingual models and data augmentation, on four LRL.
Comment: Accepted at the 16th International Workshop on Spoken Language Translation (IWSLT), November, 2019
Do the share buttons not appear? Please make sure, any blocking addon is disabled, and then reload the page.