Abstract Classification, recognition, and quality assessment of aerial images strongly depends on detecting and identifying their discriminative visual features. In practice, aerial images provide clues for various applications, including disaster prediction, automatic navigation, and military target detection. However, the detection of discriminative cues in aerial images is quite problematic since the aerial image quality is susceptible to luminance and noise, while aerial images have significantly different topological structures. We propose a novel method to explore quality-related and topological cues from aerial images for visual classification to mitigate these problems. We first decompose aerial images into several components, each being processed via the morphological filtering. Subsequently, we leverage the quality model to generate discriminative regions and topologies. Each aerial image is represented using a feature vector extracted from these regions. Afterward, we train a CNN-based visual classification model to predict aerial image categories. Experimental results have shown that our method can effectively predict aerial image categories, and the proposed algorithm is more robust than other state-of-the-art ones.