Semi-Supervised Learning of Statistical Models for Natural Language Understanding

Article English OPEN
Zhou, Deyu ; He, Yulan (2014)
  • Publisher: Hindawi Publishing Corporation
  • Journal: The Scientific World Journal, volume 2,014 (issn: 2356-6140, eissn: 1537-744X)
  • Related identifiers: pmc: PMC4127215, doi: 10.1155/2014/121650
  • Subject: Research Article | Science (General) | Q1-390 | Article Subject

Natural language understanding is to specify a computational model that maps sentences to their semantic mean representation. In this paper, we propose a novel framework to train the statistical models without using expensive fully annotated data. In particular, the input of our framework is a set of sentences labeled with abstract semantic annotations. These annotations encode the underlying embedded semantic structural relations without explicit word/semantic tag alignment. The proposed framework can automatically induce derivation rules that map sentences to their semantic meaning representations. The learning framework is applied on two statistical models, the conditional random fields (CRFs) and the hidden Markov support vector machines (HM-SVMs). Our experimental results on the DARPA communicator data show that both CRFs and HM-SVMs outperform the baseline approach, previously proposed hidden vector state (HVS) model which is also trained on abstract semantic annotations. In addition, the proposed framework shows superior performance than two other baseline approaches, a hybrid framework combining HVS and HM-SVMs and discriminative training of HVS, with a relative error reduction rate of about 25% and 15% being achieved in F-measure.
  • References (26)
    26 references, page 1 of 3

    [1] J. Dowding, R. Moore, F. Andry, and D. Moran, “Interleaving syntax and semantics in an eficient bottom-up parser,” in Proceedings of the 32th Annual Meeting of the Association for Computational Linguistics, pp. 110-116, Las Cruces, NM, USA, 1994.

    [2] W. Ward and S. Issar, “Recent improvements in the cmu spoken language understanding system,” in Proceedings of the Workshop on Human Language Technology, pp. 213-216, Plainsboro, NJ, USA, 1994.

    [3] J. D. Laefrty, A. McCallum, and F. C. N. Pereira, “Conditional random efilds: probabilistic models for segmenting and labeling sequence data,” in Proceedings of the 18th International Conference on Machine Learning (ICML '11), pp. 282-289, 2001.

    [4] Y. Altun, I. Tsochantaridis, and T. Hofmann, “Hidden markov support vector machines,” in Proceedings of the International Conference in Machine Learning, pp. 3-10, 2003.

    [5] Y. He and S. Young, “Semantic processing using the hidden vector state model,” Computer Speech and Language, vol. 19, no. 1, pp. 85-106, 2005.

    [6] R. J. Kate and R. J. Mooney, “Using string-kernels for learning semantic parsers,” in Proceedings of the 21st International Conference on Computational Linguistics and the 44th Annual Meeting of the Association for Computational Linguistics (ACL '06), pp. 913-920, 2006.

    [7] Y. W. Wong and R. J. Mooney, “Learning synchronous grammars for semantic parsing with lambda calculus,” in Proceedings of the 45th Annual Meeting of the Association for Computational Linguistics (ACL '07), pp. 960-967, June 2007.

    [8] W. Lu, H. Ng, W. Lee, and L. Zettlemoyer, “A generative model for parsing natural language to meaning representations,” in Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP '08), pp. 783-792, Stroudsburg, PA, USA, October 2008.

    [9] R. Ge and R. Mooney, “Learning a compositional semantic parser using an existing syntactic parser,” in Proceedings of the 47th Annual Meeting of the ACL, pp. 611-619, 2009.

    [10] M. Dinarelli, A. Moschitti, and G. Riccardi, “Discriminative reranking for spoken language understanding,” IEEE Transactions on Audio, Speech and Language Processing, vol. 20, no. 2, pp. 526-539, 2012.

  • Metrics
    No metrics available
Share - Bookmark