The rapid proliferation of the Internet of Things (IoT) has increased the risk of sophisticated cyber-attacks, particularly botnets, which can lead to privacy breaches, service disruptions, and infrastructure damage. Traditional security solutions, such as firewalls and signature-based (IDS), are often ineffective due to their static nature and inability to adapt to evolving threats. To address these limitations, this paper proposes Bot-EnsIDS, a dynamic and intelligent intrusion detection system that integrates ensemble bio-inspired optimizer algorithms with a hybrid deep learning classifier. The proposed approach addresses key challenges in existing IDS frameworks, including the inability to efficiently handle high-dimensional IoT traffic data and adapt to detecting newly emerging or obfuscated attacks. The Bot-EnsIDS framework comprises multiple stages, including data preprocessing, a novel multi-objective function for feature selection, and an ensemble-based optimization using Particle Swarm Optimization (PSO) and Gorilla Troops Optimizer (GTO) to extract mutual features. It also incorporates enhanced automatic data augmentation using a modified Generative Adversarial Network (GAN), followed by detection using a hybrid CNN-LSTM deep learning model capable of efficiently detecting botnet attacks by learning from spatial and temporal data features through integrating Convolutional Neural Networks (CNN) and Long Short-Term Memory (LSTM) algorithms. The empirical evaluation of the proposed approach is performed using the BoT-IoT benchmark dataset and demonstrates significant improvements in detection performance over 300 training epochs. The findings revealed that the proposed approach achieves an enhanced accuracy of 97%, recall of 97.5%, precision of 97.5%, and F-measure of 97.5%, indicating significant enhancements in correctly identifying normal and anomalous traffic. The false-positive rate dropped to 0.025, highlighting the system’s precision and reduced false alarms. Overall, this paper introduces a sophisticated IDS architecture that addresses existing gaps and sets a new standard in IoT security by integrating hybrid deep learning techniques and bio-inspired algorithms, promising a more secure IoT ecosystem.