Abstract This paper reviews the existing heritage building modelling approaches, both parametric and reality-based, with the focus being upon geometric accuracy and semantic richness of the generated models. Our review identifies that there is currently no approach that offers both and that there is always a trade-off between visual fidelity and parametric flexibility of the final model. We address this major challenge by proposing a novel concept for heritage building information modelling (BIM) based on a platform that uses a layered structure, where integrated and interconnected layers can include data and models of different types such as a polygonal textured mesh, point cloud, and parametric building information model. The proposed approach combines parametric and reality-based modelling approaches and takes advantage of the strengths of each, thus meeting the requirements of a variety of applications, and it simplifies the scan-to-BIM process which has been recognised as a bottleneck in heritage BIM.