Identifying landmarks in images and video is relevant for AR applications, as well as for mining images or key frames from sparsely annotated collections in order to create 3D assets of these landmarks to be used in XR experiences. For a specific XR production, landmarks beyond those found in large public datasets may be needed. We present a comprehensive investigation of the application of incremental learning for landmark recognition using the capabilities of a fine-grained image classification network. We analyse the effect of different incremental learning strategies on the performance for base and novel classes. The work shows that incrementally training landmark classifiers is feasible in a few-shot setting, achieving similar performance as for batch training all classes.