AbstractThis paper aims to evaluate sources of uncertainty and variability associated with rock glacier inventories compiled from remotely sensed imagery. To this end, we ran three mapping exercises in Kaiserbergtal, Austria. To evaluate interactions between mapping style and imagery resolution, we asked six operators to map and assess the degree of activity of all rock glaciers they could identify – first on Google Earth™ (GE) imagery and then on a set of higher‐resolution orthophotos and LiDAR‐derived images (LO) (Exercise 1). To compare the mapping attributes on a common set of rock glaciers, we asked 14 operators to delineate the outline of four designated rock glaciers (Exercise 2) and to classify the activity of 15 designated rock glaciers (Exercise 3) on LO. Results show strikingly high inter‐operator variability. Specifically, we show that the number of mapped rock glaciers on GE can vary up to a factor of 3, and that using LO lowers this figure to a factor of 2, while producing an increase in the number of mapped landforms, which become systematically smaller (Exercise 1). Examination of polygon outlines identifies highest inter‐operator variability at the transition with the rooting zone and, in polymorphic rock glaciers, on adjacent creeping lobes. Variability is higher for relict landforms (Exercise 2). Operators’ activity assessments, evaluated against an independent quantitative activity index (RGI), display reasonable agreement for active and relict landforms, whereas inactive counterparts meet least consensus and therefore are problematic (Exercise 3). We further show that this variability in mapping outcomes propagates across compound variables, and must be considered when assessing uncertainties and confidence levels of environmental evaluations that rely on rock glacier inventories, such as the lower altitudinal limit of discontinuous permafrost, or water storage potential. We call for an international effort to establish guidelines for rock glacier classification and mapping, towards more homogeneous inventories. © 2019 John Wiley & Sons, Ltd.