Relatively few housing energy efficiency action plans have been able to meet their goals, primarily as a result of lack of a human centred approach to this concept, and lack of attention to the interrelationships between the different aspects of housing energy efficiency, such as the building, human, built and natural environment, and ecology. To meet the energy efficiency target baseline, especially in smaller sized housing, it is accordingly necessary to provide to users a balanced interplay between all of its aspects. Doing so requires a multi-criteria decision-making platform, which would integrate the considerable data and information related to housing sustainability and energy efficiency and connect it to occupants' characteristics and perceptions. This study accordingly aims at identifying the interrelationships among the different aspects of housing essential in the provision of a balanced interplay between different aspects of housing energy efficiency. Initial research has developed a prototype online photo-based database through a combination of secondary data related to different aspects of housing energy efficiency, and primary data related to occupants’ perceptions of housing energy efficiency. Development of this type of database can aid in the evaluation of the energy efficiency of a house in relation to occupants’ needs, expectations, and preferences; and in providing a balanced interplay between the different aspects of housing energy efficiency. To further develop this approach to a larger scale requires the application of artificial intelligence to predict housing energy efficiency through a multi-criteria decision making platform, which integrates the different aspects of housing energy efficiency and its perception by housing occupants. The prediction would be made through the assessment of the outside view of the building using deep learning methodology. Such a system has the potential to make a significant contribution to the development of user-oriented housing energy efficiency actions plans.