Pollen records are the most important proxy for reconstructing past terrestrial vegetation. While new approaches for improved quantitative interpretation of pollen data have been developed over the last decades, the availability of pollen records remains mostly limited because pollen samples are still analysed manually, which is a time-consuming task and requires extensive training. Here, we present an approach for automated recognition of pollen and spores from lake sediments using deep convolutional neural networks and machine learning. The approach includes two stages. The detector first locates pollen and spores in the sample matrix, and the classifier then classifies these objects. We have trained the approach on two pollen datasets from two lakes in north-eastern Germany. So far, our approach is able to automatically recognise 10 pollen types and Lycopodium spores with high accuracy. As soon as more training data are available, more pollen and spore types can be added. The preparation of training data, the training of the neural networks and their application are accessible via a freely available, browser-based user interface called TOFSI.