The widespread use of pervasive sensing technolo- gies such as wireless sensors and street cameras allows the de- ployment of crowd estimation solutions in smart cities. However, existing Wi-Fi-based systems do not provide highly accurate crowd size estimation. Furthermore, these systems do not adapt to the dynamic changes in-the-wild, such as unexpected crowd gatherings. This paper presents a new adaptive machine learning system, called CountMeIn, to address the crowd estimation problem using polynomial regression and neural networks. The approach transfers the calibration task from cameras to machine learning after a short training with people counting from stereo- scopic cameras, Wi-Fi probe packets, and temporal features. After the training, CountMeIn calibrates Wi-Fi using the trained model and maintains high accuracy for a longer duration without cameras. We test the approach in our pilot study in Gold Coast, Australia, for about five months. CountMeIn achieves 44% and 72% error reductions in minutely and hourly crowd estimations compared to the state-of-the-art methods.