Cultural heritage buildings represent invaluable historical and architectural assets, but their preservation often conflicts with the need for modern energy efficiency. This paper presents a novel framework for digital twin development that adopts a multimodal approach to provide an open-source environment of a CH building with the goal to simulate and optimize energy performance in heritage structures. By integrating diverse data modalities, such as structural analysis, environmental monitoring, and multimedia documentation, the proposed digital twin provides a comprehensive, dynamic representation of the building’s energy behavior. This approach enables more accurate simulations of energy performance, facilitating informed decision-making for sustainable retrofitting while respecting the unique constraints of cultural heritage conservation. The application of this framework in a real case study demonstrates the potential of multimodal digital twins to bridge the gap between preservation and energy efficiency, to increase immersiveness and to highlight their role in advancing sustainable management practices for heritage sites. The findings underscore the transformative potential of digital twin technology in fostering sustainable, data-driven solutions for cultural heritage conservation.