The classification of high-resolution (HR) synthetic aperture radar (SAR) image is of great significance to SAR scene interpretation and understanding. However, the HR SAR image contains rich ground information and complex spatial structural features. The extraction of effective distinguishing features under the influence of coherent speckle is still a challenging task. A multi-scale anisotropic convolution sparse coding (MACSC) algorithm is proposed for HR SAR image classification. First, the low-order statistical features are extracted to improve the richness of SAR image. Then an unsupervised MACSC model is utilized to learn a set of sparse feature maps and convolution filters for the above statistical features by solving the designed objective function. In MACSC, the multi-scale anisotropic Gaussian kernels are utilized to initialize the dictionary of MACSC. Compared with the isotropic Gaussian kernel of the original CSC, these kernels can more accurately describe the details of different directions and scales in the SAR scene. Meanwhile, the adaptive sparse control factor is introduced into the MACSC model, which can make the learned sparse feature maps suppress the speckle and capture the abundant edge and texture information of the SAR image. After that, the aggregation operation is conducted on multi-scale and multi-direction sparse feature maps to integrate the neighbor pixels and reduce the computing burden. Finally, the obtained feature vector is input into the support vector machine classifier to realize classification. Experiments on three challenging SAR data sets demonstrate that MACSC achieves better classification performance than related sparse representation methods.