Abstract With the world population growing exponentially reaching 7.8 billion people in 2020, the issue of crowd management has become more difficult especially when the situation requires social distancing (e.g. due to COVID-19). The Internet of Things (IoT) technology can help in tackling such issues. In this article, we propose a behavior analysis-based IoT services architecture for crowd management. We propose to use a behavior analysis approach based on using generative model as Hidden Markov Model to help crowd managers to make good decisions in invoking IoT services. The proposed approach is based on sectioning video segments captured from surveillance cameras of locations that require crowd management into spatio-temporal flow-blocks for marginalization of arbitrarily dense flow field. Then, each flow-block is classified as normal and abnormal. To demonstrate our approach, we used a real case study where crowd management is required namely, Muslim’s pilgrimage (i.e. Hajj and Umrah), where real dataset is used for experimenting. The results of the experiments we have conducted are promising in real-time performance. Such results are expected to compare favorably to those found in the literature by other researchers.