Variational inference for Student-t MLP models

Article English OPEN
Thi-hang, Nguyen ; Nabney, Ian T. (2010)

This paper presents a novel methodology to infer parameters of probabilistic models whose output noise is a Student-t distribution. The method is an extension of earlier work for models that are linear in parameters to nonlinear multi-layer perceptrons (MLPs). We used an EM algorithm combined with variational approximation, the evidence procedure, and an optimisation algorithm. The technique was tested on two regression applications. The first one is a synthetic dataset and the second is gas forward contract prices data from the UK energy market. The results showed that forecasting accuracy is significantly improved by using Student-t noise models.
  • References (13)
    13 references, page 1 of 2

    [1] I.T. Nabney, NETLAB: Algorithms for Pattern Recognition, (Springer, Great Britain, 2002).

    [2] H.T. Nguyen and I.T. Nabney, Combining the wavelet transform and forecasting models to predict gas forward prices, in: Proc. ICMLA’08, the 2008 Seventh International Conference on Machine Learning and Applications, (IEEE Computer Society, 2008) 311-317.

    [3] H.T. Nguyen and I.T. Nabney, Energy forward price prediction with a hybrid adaptive model, in: CIFEr’09, IEEE Symposium on Computational Intelligence for Financial Engineering, (2009) 66-71.

    [4] M.E. Tipping and N.D. Lawrence. Variational inference for Student-t models: Robust Bayesian interpolation and generalised component analysis, Neurocomputing, 69 (2005) 123-141.

    [5] C.M. Bishop and M. Svensén, Robust Bayesian mixture modelling, Neurocomputing, 64(2005) 235-252.

    [6] C. Archambeau and M. Verleysen, Robust Bayesian clustering, Neural Networks, 20(1) (2007) 129-138.

    [7] J. Berger, Statistical Decision Theory and Bayesian Analysis (Springer, 1985).

    [8] D.J.C. MacKay, Bayesian method for backprop network, in E. Domany, J. van Hemmen, and K. Schulten ed., Models of Neural Networks, III, (Springer, 1994) 211-254.

    [9] M. I. Jordan, Z. Ghahramani, T. S. Jaakkola and L. K. Saul. An introduction to variational methods for graphical models, Machine Learning, 37 (1999) 183-233.

    [10] M. Abramowitz and I. A. Stegun, Handbook of Mathematical Functions with Formulas, Graphs, and Mathematical Tables, (Dover, New York, 1964).

  • Metrics
    views in OpenAIRE
    views in local repository
    downloads in local repository

    The information is available from the following content providers:

    From Number Of Views Number Of Downloads
    Aston Publications Explorer - IRUS-UK 0 2
Share - Bookmark