publication . Preprint . 2014

Gaussian Process Pseudo-Likelihood Models for Sequence Labeling

Srijith, P. K.; Balamurugan, P.; Shevade, Shirish;
Open Access English
  • Published: 25 Dec 2014
Abstract
Several machine learning problems arising in natural language processing can be modeled as a sequence labeling problem. We provide Gaussian process models based on pseudo-likelihood approximation to perform sequence labeling. Gaussian processes (GPs) provide a Bayesian approach to learning in a kernel based framework. The pseudo-likelihood model enables one to capture long range dependencies among the output components of the sequence without becoming computationally intractable. We use an efficient variational Gaussian approximation method to perform inference in the proposed model. We also provide an iterative algorithm which can effectively make use of the in...
Subjects
free text keywords: Statistics - Machine Learning, Computer Science - Learning
Download from
25 references, page 1 of 2

Y. Altun, T. Hofmann, and A. J. Smola. Gaussian Process Classification for Segmenting and Annotating Sequences. In ICML, 2004. [OpenAIRE]

P. Balamurugan, S. Shevade, S. Sundararajan, and S.S. Keerthi. A Sequential Dual Method for Structural SVMs. In SDM, pages 223-234, 2011.

D. P. Bertsekas. Nonlinear Programming. Athena Scientific, 1999.

J. Besag. Statistical analysis of non-lattice data. The Statistician, 24:179-195, 1975.

L. Bottou. Large-Scale Machine Learning with Stochastic Gradient Descent. In COMPSTAT, 2010.

S. Boyd and L. Vandenberghe. Convex Optimization. Cambridge University Press, 2004.

S. Bratieres, N. Quadrianto, and Z. Ghahramani. Bayesian Structured Prediction Using Gaussian Processes. IEEE transactions on Pattern Analysis and Machine Intelligence, 2014a. [OpenAIRE]

S. Bratieres, N. Quadrianto, S. Nowozin, and Z. Ghahramani. Scalable Gaussian Process Structured Prediction for Grid Factor Graph Applications. ICML, 2014b. [OpenAIRE]

K. M. A. Chai. Variational Multinomial Logit Gaussian Process. J. Mach. Learn. Res., 13, 2012.

M. Girolami and S. Rogers. Variational Bayesian Multinomial Probit Regression with Gaussian Process Priors. Neural Computation, 18(8):1790-1817, 2006. [OpenAIRE]

D. Heckerman, D. M. Chickering, C. Meek, R. Rounthwaite, and C. Kadie. Dependency Networks for Inference, Collaborative Filtering, and Data Visualization. J. Mach. Learn. Res., 1:49-75, 2001.

M. E. Khan, S. Mohamed, and K. P. Murphy. Fast Bayesian Inference for NonConjugate Gaussian Process Regression. In NIPS, pages 3149-3157, 2012.

J. D. Lafferty, A. McCallum, and F. C. N. Pereira. Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data. In ICML, pages 282- 289, 2001.

J. D. Lafferty, X. Zhu, and Y Liu. Kernel Conditional Random Fields: Representation and Clique Selection. In ICML, volume 69, 2004.

Q Li, J Wang, D. P. Wipf, and Z. Tu. Fixed-Point Model For Structured Labeling. In ICML, pages 214-221, 2013.

25 references, page 1 of 2
Related research
Abstract
Several machine learning problems arising in natural language processing can be modeled as a sequence labeling problem. We provide Gaussian process models based on pseudo-likelihood approximation to perform sequence labeling. Gaussian processes (GPs) provide a Bayesian approach to learning in a kernel based framework. The pseudo-likelihood model enables one to capture long range dependencies among the output components of the sequence without becoming computationally intractable. We use an efficient variational Gaussian approximation method to perform inference in the proposed model. We also provide an iterative algorithm which can effectively make use of the in...
Subjects
free text keywords: Statistics - Machine Learning, Computer Science - Learning
Download from
25 references, page 1 of 2

Y. Altun, T. Hofmann, and A. J. Smola. Gaussian Process Classification for Segmenting and Annotating Sequences. In ICML, 2004. [OpenAIRE]

P. Balamurugan, S. Shevade, S. Sundararajan, and S.S. Keerthi. A Sequential Dual Method for Structural SVMs. In SDM, pages 223-234, 2011.

D. P. Bertsekas. Nonlinear Programming. Athena Scientific, 1999.

J. Besag. Statistical analysis of non-lattice data. The Statistician, 24:179-195, 1975.

L. Bottou. Large-Scale Machine Learning with Stochastic Gradient Descent. In COMPSTAT, 2010.

S. Boyd and L. Vandenberghe. Convex Optimization. Cambridge University Press, 2004.

S. Bratieres, N. Quadrianto, and Z. Ghahramani. Bayesian Structured Prediction Using Gaussian Processes. IEEE transactions on Pattern Analysis and Machine Intelligence, 2014a. [OpenAIRE]

S. Bratieres, N. Quadrianto, S. Nowozin, and Z. Ghahramani. Scalable Gaussian Process Structured Prediction for Grid Factor Graph Applications. ICML, 2014b. [OpenAIRE]

K. M. A. Chai. Variational Multinomial Logit Gaussian Process. J. Mach. Learn. Res., 13, 2012.

M. Girolami and S. Rogers. Variational Bayesian Multinomial Probit Regression with Gaussian Process Priors. Neural Computation, 18(8):1790-1817, 2006. [OpenAIRE]

D. Heckerman, D. M. Chickering, C. Meek, R. Rounthwaite, and C. Kadie. Dependency Networks for Inference, Collaborative Filtering, and Data Visualization. J. Mach. Learn. Res., 1:49-75, 2001.

M. E. Khan, S. Mohamed, and K. P. Murphy. Fast Bayesian Inference for NonConjugate Gaussian Process Regression. In NIPS, pages 3149-3157, 2012.

J. D. Lafferty, A. McCallum, and F. C. N. Pereira. Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data. In ICML, pages 282- 289, 2001.

J. D. Lafferty, X. Zhu, and Y Liu. Kernel Conditional Random Fields: Representation and Clique Selection. In ICML, volume 69, 2004.

Q Li, J Wang, D. P. Wipf, and Z. Tu. Fixed-Point Model For Structured Labeling. In ICML, pages 214-221, 2013.

25 references, page 1 of 2
Related research
Powered by OpenAIRE Open Research Graph
Any information missing or wrong?Report an Issue