publication . Preprint . 2017

The Kernel Mixture Network: A Nonparametric Method for Conditional Density Estimation of Continuous Random Variables

Ambrogioni, Luca; Güçlü, Umut; van Gerven, Marcel A. J.; Maris, Eric;
Open Access English
  • Published: 19 May 2017
Abstract
This paper introduces the kernel mixture network, a new method for nonparametric estimation of conditional probability densities using neural networks. We model arbitrarily complex conditional densities as linear combinations of a family of kernel functions centered at a subset of training points. The weights are determined by the outer layer of a deep neural network, trained by minimizing the negative log likelihood. This generalizes the popular quantized softmax approach, which can be seen as a kernel mixture network with square and non-overlapping kernels. We test the performance of our method on two important applications, namely Bayesian filtering and gener...
Subjects
ACM Computing Classification System: ComputingMethodologies_PATTERNRECOGNITION
free text keywords: Statistics - Machine Learning
Related Organizations
Download from
29 references, page 1 of 2

1. S. Särkkä, Bayesian Filtering and Smoothing. Cambridge University Press, 2013.

2. A. van den Oord, S. Dieleman, H. Zen, K. Simonyan, O. Vinyals, A. Graves, N. Kalchbrenner, A. Senior, and K. Kavukcuoglu, “Wavenet: A generative model for raw audio,” arXiv preprint arXiv:1609.03499, 2016.

3. A. van den Oord, N. Kalchbrenner, L. Espeholt, O. Vinyals, A. Graves, and K. Koray, “Conditional image generation with PixelCNN decoders,” Advances in Neural Information Processing Systems, 2016.

4. C. M. Bishop, “Mixture density networks,” Technical Report NCRG/94/004, 1994.

5. L. Theis and M. Bethge, “Generative image modeling using spatial LSTMs,” Advances in Neural Information Processing Systems, 2015.

6. T. Salimans, A. Karpathy, X. Chen, and D. P. Kingma, “PixelCNN++: Improving the PixelCNN with discretized logistic mixture likelihood and other modifications,” arXiv preprint arXiv:1701.05517, 2017. [OpenAIRE]

7. A. van den Oord, N. Kalchbrenner, and K. Kavukcuoglu, “Pixel recurrent neural networks,” arXiv preprint arXiv:1601.06759, 2016.

8. M. Rosenblatt, “Remarks on some nonparametric estimates of a density function,” The Annals of Mathematical Statistics, vol. 27, no. 3, pp. 832-837, 1956.

9. E. Parzen, “On estimation of a probability density function and mode,” The Annals of Mathematical Statistics, vol. 33, no. 3, pp. 1065-1076, 1962.

10. L. Ambrogioni, U. Güçlü, E. Maris, and M. van Gerven, “Estimating nonlinear dynamics with the ConvNet smoother,” arXiv preprint arXiv:1702.05243, 2017. [OpenAIRE]

11. A. G. Wilson, Z. Hu, R. n. Salakhutdinov, and E. P. Xing, “Deep kernel learning,” Proceedings of the 19th International Conference on Artificial Intelligence and Statistics, pp. 370-378, 2016.

12. A. G. Wilson, Z. Hu, R. R. Salakhutdinov, and E. P. Xing, “Stochastic variational deep kernel learning,” Advances in Neural Information Processing Systems, pp. 2586-2594, 2016.

13. M. D. Buhmann, Radial Basis Functions: Theory and Implementations, vol. 12. Cambridge University Press, 2003.

14. F. Monti, D. Boscaini, J. Masci, E. Rodolà, J. Svoboda, and M. M. Bronstein, “Geometric deep learning on graphs and manifolds using mixture model CNNs,” arXiv preprint arXiv:1611.08402, 2016. [OpenAIRE]

15. M. M. Bronstein, J. Bruna, Y. LeCun, A. Szlam, and P. Vandergheynst, “Geometric deep learning: going beyond euclidean data,” arXiv preprint arXiv:1611.08097, 2016.

29 references, page 1 of 2
Powered by OpenAIRE Open Research Graph
Any information missing or wrong?Report an Issue