CSO-DQN: Circle Search Optimization-based Deep Q-Learning Network for Intrusion Detection System in Cloud Environment

Abstract

In general, the attackers probably create security-related threats from diverse insecure applications. These attackers execute various nasty activities like establishing hidden rootkit processes, hindering host-based security systems, changing the characteristics of applications, etc. The attackers utilize compromised tenant virtual machines for producing attacks with minor changes to avoid accurate prediction. To deal with these shortcomings, we proposed an efficient novel Deep Q-learning network-based circle search (DQL-CS) algorithm to detect intrusion in a cloud environment. The dataset is pre-processed initially by employing the Term Frequency-Inverse Document Frequency (TF-IDF) approach, which sequentially organizes normal and intrusion traces, and then the collected feature sequences are extracted using n-gram sampling. This process makes the classifier enhance the detection and classification performance. Followed by the extraction process, the classification process is executed using the proposed Deep Q-learning network-based circle search (DQL-CS) algorithm. The proposed DQL-CS algorithm accurately identifies and categorizes the attacks as two different classes normal and malevolent. Then the applications generating the malicious behavior are examined by the cloud admin using an alert generation system and are isolated automatically. The developed intrusion detection system is evaluated using three different malware databases such as the UNM dataset, windows malware dataset, and KDD99 dataset as the input. The accuracy rate of the UNM dataset of the proposed DQL-CS method is higher than other conventional approaches.