Network Intrusion Detection Systems (NIDS) play a crucial role in IoT security. In recent years, deep learning-based intrusion detection systems have demonstrated excellent performance. However, the high computational and storage requirements make these impractical for most IoT devices. To address this pressing issue, we propose PNet-IDS, a novel lightweight convolutional neural network (CNN)-based method to reduce computational complexity and optimize on-device resource usage for real-time intrusion detection. The key contribution of the proposed method is the reduced number of floating point operations (FLOPs) and effective utilization of on-device computational resources at high accuracies and precision, making PNet-IDS lightweight and efficient for real-time next generation IoT intrusion detection. Moreover, PNet-IDS’ robustness against distribution shifts in network traffic is enhanced by through a knowledge distillation framework. Comprehensive experimental evaluations using the popular BoT-IoT and CIC-IDS2017 benchmark datasets prove the superiority of the proposed PNet-IDS over competitive related methods in terms of reduced parameters count, reduced FLOPs, reduced model size while maintaining high accuracy and precision. By combining PNet-IDS’ efficiency with knowledge distillation’s adaptability, the proposed method offers a scalable and resilient solution for IoT intrusion detection.