Abstract Aims Taxon identification is an important step in many plant ecological studies. Its efficiency and reproducibility might greatly benefit from partly automating this task. Image-based identification systems exist, but mostly rely on hand-crafted algorithms to extract sets of features chosen a priori to identify species of selected taxa. In consequence, such systems are restricted to these taxa and additionally require involving experts that provide taxonomical knowledge for developing such customized systems. The aim of this study was to develop a deep learning system to learn discriminative features from leaf images along with a classifier for species identification of plants. By comparing our results with customized systems like LeafSnap we can show that learning the features by a convolutional neural network (CNN) can provide better feature representation for leaf images compared to hand-crafted features. Methods We developed LeafNet, a CNN-based plant identification system. For evaluation, we utilized the publicly available LeafSnap, Flavia and Foliage datasets. Results Evaluating the recognition accuracies of LeafNet on the LeafSnap, Flavia and Foliage datasets reveals a better performance of LeafNet compared to hand-crafted customized systems. Conclusions Given the overall species diversity of plants, the goal of a complete automatisation of visual plant species identification is unlikely to be met solely by continually gathering assemblies of customized, specialized and hand-crafted (and therefore expensive) identification systems. Deep Learning CNN approaches offer a self-learning state-of-the-art alternative that allows adaption to different taxa just by presenting new training data instead of developing new software systems.