Abstract Recently, multi-view feature extraction has attracted great interest and Canonical Correlation Analysis (CCA) is a powerful technique for finding the linear correlation between two view variable sets. However, CCA does not consider the structure and cross view information in feature extraction, which is very important for subsequence tasks. In this paper, a new approach called Canonical Sparse Cross-view Correlation Analysis (CSCCA) is proposed to address this problem. We first construct similarity matrices by performing sparse representation between within-class samples. Then local manifold information and cross-view correlations are incorporated into CCA. Furthermore, a kernel version of CSCCA (KCSCCA) is proposed to reveal the nonlinear correlation relationship between two sets of features. We compare CSCCA and KCSCCA with existing multi-view feature extraction methods and perform experiments on both artificial data set and real world databases including multiple features and face data sets. The experimental results demonstrate the merits of our proposed method.