Remote sensing (RS) image classification plays an important role in the earth observation technology using RS data, having been widely exploited in both military and civil fields. However, due to the characteristics of RS data such as high dimensionality and relatively small amounts of labeled samples available, performing RS image classification faces great scientific and practical challenges. In recent years, as new deep learning (DL) techniques emerge, approaches to RS image classification with DL have achieved significant breakthroughs, offering novel opportunities for the research and development of RS image classification. In this paper, a brief overview of typical DL models is presented first. This is followed by a systematic review of pixel‐wise and scene‐wise RS image classification approaches that are based on the use of DL. A comparative analysis regarding the performances of typical DL‐based RS methods is also provided. Finally, the challenges and potential directions for further research are discussed.This article is categorized under: Application Areas > Science and Technology Technologies > Classification