In recent times, malware has progressed by utilizing distinct advanced machine learning techniques for detection. However, the model becomes complicated and the singular value decomposition and depth-based malware detectors failed to detect the malware significantly with minimum time and overhead. This paper proposes a multinomial linked latent dirichlet and modular double q learning (MLLD-MDQL) to efficiently detect malware based on the network behavior patterns. First, multinomial linked latent dirichlet network behavior extraction (ML-LDNBE) is applied to the input network for anomaly detection that extracts the behavior based on the network pattern. The behavior is extracted which are grouped to perform on the path protocol for analyzing repeated behaviors. Finally, the modular double q learning malware classification model is the grouped behaviors for significant malware detection. The effectiveness of proposed MLLD-M DQL method is compared with existing models. The results obtained by the proposed method show that the model combined with the machine learning (ML) significantly determined malware detection time and also reduced the false positive rate (FPR). The results showed that the false positive rate is significantly reduced by 42% for the proposed method better when compared to the existing behavior based malware detection model that obtained 62% of FPR.