In the fine-grained categories, images have lager diversity in their intra categories. Meanwhile, they have more similarity in their inter categories. Therefore, images are difficultly distinguish during fine-grained visual classification(FGVC). This paper proposes a deep sparse coding framework to implement the fine-grained visual categorization. In our framework, deep layer structures with sparse coding are used to learn different spatial features. Especially, for categories with asymmetric structure, a quick and efficient pose estimation method is introduced to calibrate their poses. This framework is evaluated using two fine-grained datasets, i.e. Oxford 102 flowers dataset and the CUB-200-2011 bird dataset. Final experimental results show that the performance of our proposed system is highly competitive with state-of-the-art algorithms.