publication . Conference object . Other literature type . 2019

Lexical Micro-adaptation for Neural Machine Translation

Xu, Jitao; Crego, Josep; Senellart, Jean;
Open Access English
  • Published: 02 Nov 2019
  • Publisher: HAL CCSD
  • Country: France
Abstract
International audience; This work is inspired by a typical machine translation industry scenario in which translators make use of in-domain data for facilitating translation of similar or repeating sentences. We introduce a generic framework applied at inference in which a subset of segment pairs are first extracted from training data according to their similarity to the input sentences. These segments are then used to dynamically update the parameters of a generic NMT network, thus performing a lexical micro-adaptation. Our approach demonstrates strong adaptation performance to new and existing datasets including pseudo in-domain data. We evaluate our approach ...
Subjects
free text keywords: [INFO.INFO-CL]Computer Science [cs]/Computation and Language [cs.CL]
Download fromView all 6 versions
Zenodo
Other literature type . 2019
Provider: Datacite
Zenodo
Other literature type . 2019
Provider: Datacite
Hyper Article en Ligne
Conference object . 2019
40 references, page 1 of 3

[2] I. Sutskever, O. Vinyals, and Q. V. Le, “Sequence to sequence learning with neural networks,” in Advances in Neural Information Processing Systems 27, Z. Ghahramani, M. Welling, C. Cortes, N. D. Lawrence, and K. Q. Weinberger, Eds. Curran Associates, Inc., 2014, pp. 3104- 3112. [Online]. Available: http://papers.nips.cc/paper/ 5346-sequence-to-sequence-learning-with-neural-networks. pdf

[3] K. Cho, B. van Merrienboer, C. Gulcehre, D. Bahdanau, F. Bougares, H. Schwenk, and Y. Bengio, “Learning phrase representations using RNN encoder-decoder for statistical machine translation,” in Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP). Doha, Qatar: Association for Computational Linguistics, Oct. 2014, pp. 1724-1734. [Online]. Available: https://www.aclweb.org/anthology/D14-1179

[4] D. Bahdanau, K. Cho, and Y. Bengio, “Neural machine translation by jointly learning to align and translate,” in 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings, 2015. [Online]. Available: http://arxiv.org/abs/1409.0473

[5] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need,” in Advances in Neural Information Processing Systems 30, I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, Eds. Curran Associates, Inc., 2017, pp. 5998- 6008. [Online]. Available: http://papers.nips.cc/paper/ 7181-attention-is-all-you-need.pdf

[6] C. Kobus, J. Crego, and J. Senellart, “Domain control for neural machine translation,” in Proceedings of the International Conference Recent Advances in Natural Language Processing, RANLP 2017. Varna, Bulgaria: INCOMA Ltd., Sept. 2017, pp. 372- 378. [Online]. Available: https://doi.org/10.26615/ 978-954-452-049-6 049

[7] M.-T. Luong and C. D. Manning, “Stanford neural machine translation systems for spoken language domain,” in International Workshop on Spoken Language Translation, Da Nang, Vietnam, 2015. [Online]. Available: https://nlp.stanford.edu/pubs/ luong-manning-iwslt15.pdf

[8] P. Michel and G. Neubig, “Extreme adaptation for personalized neural machine translation,” in Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers). Melbourne, Australia: Association for Computational Linguistics, July 2018, pp. 312- 318. [Online]. Available: https://www.aclweb.org/ anthology/P18-2050

[9] J. Wuebker, P. Simianer, and J. DeNero, “Compact personalized models for neural machine translation,” in Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. Brussels, Belgium: Association for Computational Linguistics, Oct.-Nov. 2018, pp. 881-886. [Online]. Available: https://www.aclweb.org/anthology/D18-1104 [OpenAIRE]

[10] D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” in 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings, Y. Bengio and Y. LeCun, Eds., 2015. [Online]. Available: http://arxiv.org/abs/1412.6980

[11] U. Manber and G. Myers, “Suffix arrays: A new method for on-line string searches,” in Proceedings of the First Annual ACM-SIAM Symposium on Discrete Algorithms, ser. SODA '90. Philadelphia, PA, USA: Society for Industrial and Applied Mathematics, 1990, pp. 319-327. [Online]. Available: http://dl.acm.org/ citation.cfm?id=320176.320218

[12] A. Conneau, D. Kiela, H. Schwenk, L. Barrault, and A. Bordes, “Supervised learning of universal sentence representations from natural language inference data,” in Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing. Copenhagen, Denmark: Association for Computational Linguistics, Sept. 2017, pp. 670-680. [Online]. Available: https://www.aclweb.org/anthology/D17-1070 [OpenAIRE]

[13] J. Johnson, M. Douze, and H. Je´gou, “Billionscale similarity search with gpus,” arXiv preprint arXiv:1702.08734, 2017. [OpenAIRE]

[14] G. Klein, Y. Kim, Y. Deng, J. Senellart, and A. Rush, “OpenNMT: Open-source toolkit for neural machine translation,” in Proceedings of ACL 2017, System Demonstrations. Vancouver, Canada: Association for Computational Linguistics, 2017, pp. 67-72. [Online]. Available: http://aclweb.org/anthology/P17-4012 [OpenAIRE]

[15] R. Sennrich, B. Haddow, and A. Birch, “Neural machine translation of rare words with subword units,” in Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Berlin, Germany: Association for Computational Linguistics, Aug. 2016, pp. 1715- 1725. [Online]. Available: https://www.aclweb.org/ anthology/P16-1162

[16] P. Koehn, H. Hoang, A. Birch, C. Callison-Burch, M. Federico, N. Bertoldi, B. Cowan, W. Shen, C. Moran, R. Zens, C. Dyer, O. Bojar, A. Constantin, and E. Herbst, “Moses: Open source toolkit for statistical machine translation,” in Proceedings of the 45th Annual Meeting of the Association for Computational Linguistics Companion Volume Proceedings of the Demo and Poster Sessions. Prague, Czech Republic: Association for Computational Linguistics, June 2007, pp. 177-180. [Online]. Available: https://www.aclweb.org/anthology/P07-2045

40 references, page 1 of 3
Abstract
International audience; This work is inspired by a typical machine translation industry scenario in which translators make use of in-domain data for facilitating translation of similar or repeating sentences. We introduce a generic framework applied at inference in which a subset of segment pairs are first extracted from training data according to their similarity to the input sentences. These segments are then used to dynamically update the parameters of a generic NMT network, thus performing a lexical micro-adaptation. Our approach demonstrates strong adaptation performance to new and existing datasets including pseudo in-domain data. We evaluate our approach ...
Subjects
free text keywords: [INFO.INFO-CL]Computer Science [cs]/Computation and Language [cs.CL]
Download fromView all 6 versions
Zenodo
Other literature type . 2019
Provider: Datacite
Zenodo
Other literature type . 2019
Provider: Datacite
Hyper Article en Ligne
Conference object . 2019
40 references, page 1 of 3

[2] I. Sutskever, O. Vinyals, and Q. V. Le, “Sequence to sequence learning with neural networks,” in Advances in Neural Information Processing Systems 27, Z. Ghahramani, M. Welling, C. Cortes, N. D. Lawrence, and K. Q. Weinberger, Eds. Curran Associates, Inc., 2014, pp. 3104- 3112. [Online]. Available: http://papers.nips.cc/paper/ 5346-sequence-to-sequence-learning-with-neural-networks. pdf

[3] K. Cho, B. van Merrienboer, C. Gulcehre, D. Bahdanau, F. Bougares, H. Schwenk, and Y. Bengio, “Learning phrase representations using RNN encoder-decoder for statistical machine translation,” in Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP). Doha, Qatar: Association for Computational Linguistics, Oct. 2014, pp. 1724-1734. [Online]. Available: https://www.aclweb.org/anthology/D14-1179

[4] D. Bahdanau, K. Cho, and Y. Bengio, “Neural machine translation by jointly learning to align and translate,” in 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings, 2015. [Online]. Available: http://arxiv.org/abs/1409.0473

[5] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need,” in Advances in Neural Information Processing Systems 30, I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, Eds. Curran Associates, Inc., 2017, pp. 5998- 6008. [Online]. Available: http://papers.nips.cc/paper/ 7181-attention-is-all-you-need.pdf

[6] C. Kobus, J. Crego, and J. Senellart, “Domain control for neural machine translation,” in Proceedings of the International Conference Recent Advances in Natural Language Processing, RANLP 2017. Varna, Bulgaria: INCOMA Ltd., Sept. 2017, pp. 372- 378. [Online]. Available: https://doi.org/10.26615/ 978-954-452-049-6 049

[7] M.-T. Luong and C. D. Manning, “Stanford neural machine translation systems for spoken language domain,” in International Workshop on Spoken Language Translation, Da Nang, Vietnam, 2015. [Online]. Available: https://nlp.stanford.edu/pubs/ luong-manning-iwslt15.pdf

[8] P. Michel and G. Neubig, “Extreme adaptation for personalized neural machine translation,” in Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers). Melbourne, Australia: Association for Computational Linguistics, July 2018, pp. 312- 318. [Online]. Available: https://www.aclweb.org/ anthology/P18-2050

[9] J. Wuebker, P. Simianer, and J. DeNero, “Compact personalized models for neural machine translation,” in Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. Brussels, Belgium: Association for Computational Linguistics, Oct.-Nov. 2018, pp. 881-886. [Online]. Available: https://www.aclweb.org/anthology/D18-1104 [OpenAIRE]

[10] D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” in 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings, Y. Bengio and Y. LeCun, Eds., 2015. [Online]. Available: http://arxiv.org/abs/1412.6980

[11] U. Manber and G. Myers, “Suffix arrays: A new method for on-line string searches,” in Proceedings of the First Annual ACM-SIAM Symposium on Discrete Algorithms, ser. SODA '90. Philadelphia, PA, USA: Society for Industrial and Applied Mathematics, 1990, pp. 319-327. [Online]. Available: http://dl.acm.org/ citation.cfm?id=320176.320218

[12] A. Conneau, D. Kiela, H. Schwenk, L. Barrault, and A. Bordes, “Supervised learning of universal sentence representations from natural language inference data,” in Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing. Copenhagen, Denmark: Association for Computational Linguistics, Sept. 2017, pp. 670-680. [Online]. Available: https://www.aclweb.org/anthology/D17-1070 [OpenAIRE]

[13] J. Johnson, M. Douze, and H. Je´gou, “Billionscale similarity search with gpus,” arXiv preprint arXiv:1702.08734, 2017. [OpenAIRE]

[14] G. Klein, Y. Kim, Y. Deng, J. Senellart, and A. Rush, “OpenNMT: Open-source toolkit for neural machine translation,” in Proceedings of ACL 2017, System Demonstrations. Vancouver, Canada: Association for Computational Linguistics, 2017, pp. 67-72. [Online]. Available: http://aclweb.org/anthology/P17-4012 [OpenAIRE]

[15] R. Sennrich, B. Haddow, and A. Birch, “Neural machine translation of rare words with subword units,” in Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Berlin, Germany: Association for Computational Linguistics, Aug. 2016, pp. 1715- 1725. [Online]. Available: https://www.aclweb.org/ anthology/P16-1162

[16] P. Koehn, H. Hoang, A. Birch, C. Callison-Burch, M. Federico, N. Bertoldi, B. Cowan, W. Shen, C. Moran, R. Zens, C. Dyer, O. Bojar, A. Constantin, and E. Herbst, “Moses: Open source toolkit for statistical machine translation,” in Proceedings of the 45th Annual Meeting of the Association for Computational Linguistics Companion Volume Proceedings of the Demo and Poster Sessions. Prague, Czech Republic: Association for Computational Linguistics, June 2007, pp. 177-180. [Online]. Available: https://www.aclweb.org/anthology/P07-2045

40 references, page 1 of 3
Powered by OpenAIRE Research Graph
Any information missing or wrong?Report an Issue