Predicting trip purpose from comprehensive and continuous smart card data is beneficial for transport and city planners investigating travel behaviours, and mobility research in urban areas. Here we propose a framework, ActivityNET, using machine learning (ML) algorithms to predict passengers' trip purpose from smart card data and Points-Of-Interest (POIs). The feasibility of the framework is demonstrated in two phases. Phase I focuses on extracting activities from individuals' daily travel patterns from smart card data and combining them with POIs using the proposed "activity-POIs consolidation algorithm". Meaningful match helps to generate an understanding of human mobility and urban flows in cities. Phase II feeds the extracted features into an artificial neural network (ANN) with multiple scenarios and predict trip purpose under primary activities (home and work) and secondary activities (entertainment, eating, shopping, child drop-offs/pick-ups and part-time work) with high accuracy. As a case study, the proposed ActivityNET framework is applied in Greater London and illustrates a robust competence to predict trip purpose. The promising outcomes demonstrate that the cost-effective framework offers valuable insight into transport planning.