Overcoming the Challenges of Data Lack, Leakage, and Dimensionality in Intrusion Detection Systems: A Comprehensive Review

Abstract

The Internet of Things (IoT) and cloud computing are rapidly gaining momentum as decentralized internet-based technologies and have led to an increase in information in nearly every technical and commercial industry. However, ensuring the security of IoT systems is a pressing issue due to the complexities involved in connected and shared environments. Networks are guarded by Intrusion Detection Systems (IDS) against various cyber threats such as malware, viruses, and unauthorized access. IDS have recently adopted Machine Learning (ML) and Deep Learning (DL) techniques to identify and classify security risks. However, the effective utilization of these technologies depends on the availability, quality, and characteristics of the data used to train models. Moreover, data lack, data leak, and dimensionality (DLLD) are common problems in data science and ML. This paper surveys existing research and suggests solutions for overcoming DLLD-related issues to improve IDS model.