Deep learning evaluation using deep linguistic processing
Kuhnle, Alexander; Copestake, Ann;
Subject: Computer Science - Computation and Language | Computer Science - Computer Vision and Pattern Recognition | Computer Science - Artificial Intelligence | Computer Science - Learning
We discuss problems with the standard approaches to evaluation for tasks like visual question answering, and argue that artificial data can be used to address these as a complement to current practice. We demonstrate that with the help of existing 'deep' linguistic proc... View more
Adi, Y., E. Kermany, Y. Belinkov, O. Lavi, and Y. Goldberg (2017). Fine-grained analysis of sentence embeddings using auxiliary prediction tasks. Proceedings of the International Conference on Learning Representations 2017 (ICLR 2017).
Agrawal, A., D. Batra, and D. Parikh (2016). Analyzing the behavior of visual question answering models. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, EMNLP 2016, Austin, Texas, pp. 1955-1960.
Andreas, J., M. Rohrbach, T. Darrell, and D. Klein (2016). Neural module networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2016.
Antol, S., A. Agrawal, J. Lu, M. Mitchell, D. Batra, C. L. Zitnick, and D. Parikh (2015). VQA: Visual question answering. In Proceedings of the IEEE International Conference on Computer Vision, ICCV 2015.
Arthur, P., G. Neubig, and S. Nakamura (2016). Incorporating discrete translation lexicons into neural machine translation. In Conference on Empirical Methods in Natural Language Processing (EMNLP), Austin, Texas, USA.
Bellemare, M. G., Y. Naddaf, J. Veness, and M. Bowling (2013). The arcade learning environment: An evaluation platform for general agents. Journal of Artificial Intelligence Research 47(1), 253-279.
Brockman, G., V. Cheung, L. Pettersson, J. Schneider, J. Schulman, J. Tang, and W. Zaremba (2016). OpenAI Gym.
Copestake, A. (2009). Slacker semantics: Why superficiality, dependency and avoidance of commitment can be the right way to go. In Proceedings of the 12th Meeting of the European Chapter of the Association for Computational Linguistics, Athens, Greece, pp. 1-9.
Copestake, A., G. Emerson, M. W. Goodman, M. Horvat, A. Kuhnle, and E. Muszyn´ska (2016). Resources for building applications with Dependency Minimal Recursion Semantics. In Proceedings of the 10th International Conference on Language Resources and Evaluation (LREC-16), Portorozˇ, Slovenia, pp. 1240-1247. European Language Resources Association (ELRA).
Copestake, A., D. Flickinger, C. Pollard, and I. A. Sag (2005). Minimal Recursion Semantics: An introduction. Research on Language and Computation 3(4), 281-332.