Robust Kernel (Cross-) Covariance Operators in Reproducing Kernel Hilbert Space toward Kernel Methods

Preprint English OPEN
Alam, Md. Ashad ; Fukumizu, Kenji ; Wang, Yu-Ping (2016)
  • Subject: Statistics - Machine Learning
    acm: ComputingMethodologies_PATTERNRECOGNITION
To the best of our knowledge, there are no general well-founded robust methods for statistical unsupervised learning. Most of the unsupervised methods explicitly or implicitly depend on the kernel covariance operator (kernel CO) or kernel cross-covariance operator (kernel CCO). They are sensitive to contaminated data, even when using bounded positive definite kernels. First, we propose robust kernel covariance operator (robust kernel CO) and robust kernel crosscovariance operator (robust kernel CCO) based on a generalized loss function instead of the quadratic loss function. Second, we propose influence function of classical kernel canonical correlation analysis (classical kernel CCA). Third, using this influence function, we propose a visualization method to detect influential observations from two sets of data. Finally, we propose a method based on robust kernel CO and robust kernel CCO, called robust kernel CCA, which is designed for contaminated data and less sensitive to noise than classical kernel CCA. The principles we describe also apply to many kernel methods which must deal with the issue of kernel CO or kernel CCO. Experiments on synthesized and imaging genetics analysis demonstrate that the proposed visualization and robust kernel CCA can be applied effectively to both ideal data and contaminated data. The robust methods show the superior performance over the state-of-the-art methods.
  • References (42)
    42 references, page 1 of 5

    J.G. Adrover and S. M. donato. A robust predictive approach for canonical correlation analysis. Journal of Multivariate Analysis., 133:356-376, 2015.

    S. Akaho. A kernel method for canonical correlation analysis. International meeting of psychometric Society., 35:321-377, 2001.

    M. A. Alam and K. Fukumizu. Hyperparameter selection in kernel principal component analysis. Journal of Computer Science, 10(7):1139-1150, 2014.

    M. A. Alam and K. Fukumizu. Higher-order regularized kernel canonical correlation analysis. International Journal of Pattern Recognition and Artificial Intelligence, 29(4):1551005(1-24), 2015.

    M. A. Alam, M. Nasser, and K. Fukumizu. A comparative study of kernel and robust canonical correlation analysis. Journal of Multimedia., 5:3-11, 2010.

    C. Alzate and J. A. K. Suykens. A regularized kernel CCA contrast function for ICA. Neural Networks, 21:170-181, 2008.

    T. W. Anderson. An Introduction to Multivariate Statistical Analysis. John Wiley& Sons, third edition, 2003.

    N. Aronszajn. Theory of reproducing kernels. Transactions of the American Mathematical Society, 68:337-404, 1950.

    F. R. Bach. Consistency of the group lasso and multiple kernel learning. Journal of Machine Learning Research, 9:1179-1225, 2008.

    F. R. Bach and M. I. Jordan. Kernel independent component analysis. Journal of Machine Learning Research, 3:1-48, 2002.

  • Metrics
    No metrics available
Share - Bookmark