Landslide inventory maps are effective and easily understandable products for both experts, such as geomorphologists, and for non experts, including decision-makers, planners, and civil defense managers. Landslide inventories are essential to understand the evolution of landscapes, and to ascertain landslide susceptibility and hazard. Despite landslide maps being compiled every year in the word at different scales, limited efforts are made to critically compare landslide maps prepared using different techniques or by different investigators. Based on the experience gained in 20 years of landslide mapping in Italy, and on the limited literature on landslide inventory assessment, we propose a general framework for the quantitative comparison of landslide inventory maps. To test the proposed framework we exploit three inventory maps. The first map is a reconnaissance landslide inventory prepared for the Umbria region, in central Italy. The second map is a detailed geomorphological landslide map, also prepared for the Umbria region. The third map is a multi-temporal landslide inventory compiled for the Collazzone area, in central Umbria. Results of the experiment allow for establishing how well the individual inventories describe the location, type and abundance of landslides, to what extent the landslide maps can be used to determine the frequency-area statistics of the slope failures, and the significance of the inventory maps as predictors of landslide susceptibility. We further use the results obtained in the Collazzone area to estimate the quality and completeness of the two regional landslide inventory maps, and to outline general advantages and limitations of the techniques used to complete the inventories.