publication . Preprint . Conference object . Other literature type . 2017

Neural Models for Documents with Metadata

Chenhao Tan;
Open Access English
  • Published: 25 May 2017
Abstract
Most real-world document collections involve various types of metadata, such as author, source, and date, and yet the most commonly-used approaches to modeling text corpora ignore this information. While specialized models have been developed for particular applications, few are widely used in practice, as customization typically requires derivation of a custom inference algorithm. In this paper, we build on recent advances in variational inference methods and propose a general neural framework, based on topic models, to enable flexible incorporation of metadata and allow for rapid exploration of alternative models. Our approach achieves strong performance, with...
Subjects
free text keywords: Statistics - Machine Learning, Computer Science - Computation and Language, Personalization, Coherence (physics), Topic model, Text corpus, Machine learning, computer.software_genre, computer, Perplexity, Artificial intelligence, business.industry, business, Computer science, Inference, Metadata
41 references, page 1 of 3

Amr Ahmed and Eric P. Xing. Staying informed: Supervised and semi-supervised multi-view topical analysis of ideological perspective. In Proceedings of EMNLP, 2010.

David M. Blei. Build, compute, critique, repeat: Data analysis with latent variable models. Annual Review of Statistics and its Application, 1:203-232, January 2014. [OpenAIRE]

David M. Blei and John D. Lafferty. Dynamic topic models. In Proceedings of ICML, 2006.

David M. Blei, Andrew Y. Ng, and Michael I. Jordan. Latent Dirichlet allocation. J. Mach. Learn. Res., 3:993-1022, 2003.

David M. Blei, Thomas L. Griffiths, and Michael I. Jordan. The nested chinese restaurant process and bayesian nonparametric inference of topic hierarchies. J. ACM, 57(2), February 2010.

Samuel R. Bowman, Luke Vilnis, Oriol Vinyals, Andrew M. Dai, Rafal Józefowicz, and Samy Bengio. Generating sentences from a continuous space. In Proceedings of CONLL, 2016.

Tamara Broderick, Nicholas Boyd, Andre Wibisono, Ashia C Wilson, and Michael I Jordan. Streaming variational bayes. In Proceedings of NIPS. 2013.

Ali Taylan Cemgil. Bayesian inference for nonnegative matrix factorisation models. Computational Intelligence and Neuroscience, pages 4:1-4:17, January 2009.

Jonathan Chang, Sean Gerrish, Chong Wang, Jordan L Boyd-graber, and David M Blei. Reading Tea Leaves: How Humans Interpret Topic Models. In Proceedings of NIPS, 2009.

Michael Collins, Sanjoy Dasgupta, and Robert E. Schapire. A generalization of principal component analysis to the exponential family. In Proceedings of NIPS, 2001.

John Duchi, Elad Hazan, and Yoram Singer. Adaptive subgradient methods for online learning and stochastic optimization. Technical report, EECS Department, University of California, Berkeley, 2010.

Jacob Eisenstein and Eric Xing. The CMU 2008 political blog corpus. Technical report, Carnegie Mellon University, 2010.

Jacob Eisenstein, Amr Ahmed, and Eric P. Xing. Sparse additive generative models of text. In Proceedings of ICML, 2011.

Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016.

Geoffrey E Hinton and Ruslan R Salakhutdinov. Replicated Softmax: An Undirected Topic Model. In Proceedings of NIPS, 2009.

41 references, page 1 of 3
Abstract
Most real-world document collections involve various types of metadata, such as author, source, and date, and yet the most commonly-used approaches to modeling text corpora ignore this information. While specialized models have been developed for particular applications, few are widely used in practice, as customization typically requires derivation of a custom inference algorithm. In this paper, we build on recent advances in variational inference methods and propose a general neural framework, based on topic models, to enable flexible incorporation of metadata and allow for rapid exploration of alternative models. Our approach achieves strong performance, with...
Subjects
free text keywords: Statistics - Machine Learning, Computer Science - Computation and Language, Personalization, Coherence (physics), Topic model, Text corpus, Machine learning, computer.software_genre, computer, Perplexity, Artificial intelligence, business.industry, business, Computer science, Inference, Metadata
41 references, page 1 of 3

Amr Ahmed and Eric P. Xing. Staying informed: Supervised and semi-supervised multi-view topical analysis of ideological perspective. In Proceedings of EMNLP, 2010.

David M. Blei. Build, compute, critique, repeat: Data analysis with latent variable models. Annual Review of Statistics and its Application, 1:203-232, January 2014. [OpenAIRE]

David M. Blei and John D. Lafferty. Dynamic topic models. In Proceedings of ICML, 2006.

David M. Blei, Andrew Y. Ng, and Michael I. Jordan. Latent Dirichlet allocation. J. Mach. Learn. Res., 3:993-1022, 2003.

David M. Blei, Thomas L. Griffiths, and Michael I. Jordan. The nested chinese restaurant process and bayesian nonparametric inference of topic hierarchies. J. ACM, 57(2), February 2010.

Samuel R. Bowman, Luke Vilnis, Oriol Vinyals, Andrew M. Dai, Rafal Józefowicz, and Samy Bengio. Generating sentences from a continuous space. In Proceedings of CONLL, 2016.

Tamara Broderick, Nicholas Boyd, Andre Wibisono, Ashia C Wilson, and Michael I Jordan. Streaming variational bayes. In Proceedings of NIPS. 2013.

Ali Taylan Cemgil. Bayesian inference for nonnegative matrix factorisation models. Computational Intelligence and Neuroscience, pages 4:1-4:17, January 2009.

Jonathan Chang, Sean Gerrish, Chong Wang, Jordan L Boyd-graber, and David M Blei. Reading Tea Leaves: How Humans Interpret Topic Models. In Proceedings of NIPS, 2009.

Michael Collins, Sanjoy Dasgupta, and Robert E. Schapire. A generalization of principal component analysis to the exponential family. In Proceedings of NIPS, 2001.

John Duchi, Elad Hazan, and Yoram Singer. Adaptive subgradient methods for online learning and stochastic optimization. Technical report, EECS Department, University of California, Berkeley, 2010.

Jacob Eisenstein and Eric Xing. The CMU 2008 political blog corpus. Technical report, Carnegie Mellon University, 2010.

Jacob Eisenstein, Amr Ahmed, and Eric P. Xing. Sparse additive generative models of text. In Proceedings of ICML, 2011.

Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016.

Geoffrey E Hinton and Ruslan R Salakhutdinov. Replicated Softmax: An Undirected Topic Model. In Proceedings of NIPS, 2009.

41 references, page 1 of 3
Powered by OpenAIRE Open Research Graph
Any information missing or wrong?Report an Issue
publication . Preprint . Conference object . Other literature type . 2017

Neural Models for Documents with Metadata

Chenhao Tan;