Dimensionality Reduction for Intrusion Detection Systems in Multi-data Streams—A Review and Proposal of Unsupervised Feature Selection Scheme

Abstract

An Intrusion Detection System (IDS) is a security mechanism that is intended to dynamically inspect traffic in order to detect any suspicious behaviour or launched attacks. However, it is a challenging task to apply IDS for large and high dimensional data streams. Data streams have characteristics that are quite distinct from those of statistical databases, which greatly impact on the performance of the anomaly-based ID algorithms used in the detection process. These characteristics include, but are not limited to, the processing of large data as they arrive (real-time), the dynamic nature of data streams, the curse of dimensionality, limited memory capacity and high complexity. Therefore, the main challenge in this area of research is to design efficient data-driven ID systems that are capable of efficiently dealing with data streams by considering these specific traffic characteristics. This chapter provides an overview of some of the relevant work carried out in three major fields related to the topic, namely feature selections (FS), intrusion detection systems (IDS) and anomaly detection in multi data streams. This overview is intended to provide the reader with a better understanding of the major recent works in the area. By critically investigating and combining those three fields, researchers and practitioners will be better able to develop efficient and robust IDS for data streams. At the end of this chapter, we provide two basic models: an Unsupervised Feature Selection to Improve Detection Accuracy for Anomaly Detection (UFSAD) and its extension (UFSAD-MS) for multi streams, that could reduce the volume and the dimensionality of the big data resulting from the streams. The reduction is based on the selection of only the relevant features and removing irrelevant and redundant ones. The last section of the chapter provides an example of the developed UFSAD model, followed by some experimental results. UFSAD-MS is provided as a conceptual model as it is in the implementation phase.