This paper presents the use of buildings' thermal inertia as virtual thermal energy storage (TES) in smart home energy management systems (SHEMS). TES provides residential users with an additional degree of flexibility and helps them scheduling the energy usage in the off-peak periods to reduce the energy expenditure. Phase change materials (PCM) are added in the insulation of a lightweight building to increase its thermal inertia. We show through simulations that the addition of the PCM makes it possible to significantly shift the operation of the Heating, Ventilation and Air Conditioning (HVAC) system while still meeting the residents' comfort requirements. In the SHEMS, TES is treated an additional distributed energy resource. We develop the mathematical model of the underlying thermodynamical system and analyse the impact of the PCM layer. Due to the inherent properties of the PCM, the resulting optimisation problem becomes non-convex, which requires the use of heuristic optimisation tools. In this paper, we used genetic algorithm to minimise the energy expenditure. The results show that the use of PCMS effectively sheds and shifts the demand to off-peak periods thus reducing the cost for the user.