Abstract In the financial business, there are cumbersome and error-prone manual procedures and long workflows. In this paper, MSCNN model is proposed to solve automatically identify and classify images such as documents. First, the image data is preprocessed to enhance the image features. Then, the image is extracted by the cross-connect structure and multi-channel convolution. Finally, the image is classified by softmax layer. The experimental results on the dataset show that the proposed model has a high classification accuracy in image classification.