Internet of things (IoT) along with big data technologies can accrue significant added value in several domains and improve people’s everyday life. One of the domains that can be benefitted the most by the aforementioned technologies is Smart Buildings. This is because, several aspects of people’s everyday lives can be improved through IoT services, such as energy consumption, health, heating, building security and more. IoT services can be divided to near real-time, and static based on the time that they require in order to return results. Significant amount of research papers has been dedicated to the second for services such as energy forecasting, while for near real-time services there are not so many publications, while, most of the existing ones focusing mostly on obtaining meaningful results. In this publication we propose a conceptual architecture for building a near real-time Anomaly Detection service for smart buildings using the Fog Computing paradigm, to achieve scalability and low latency. Moreover, we provide a technical glance of the proposed solution, suggesting specific technologies for each functionality as well as restrictions for each technology. It is worth mentioning that the proposed approach can be easily adapted for other near real-time services with little modifications.