A Model for Generating Synthetic Network Flows and Accuracy Index for Evaluation of Anomaly Network Intrusion Detection Systems

Abstract

Objectives: This study proposes a model for generating synthetic network flows inserting malicious fragments randomly and a new metric for measuring the performance of an Anomaly Network Intrusion Detection System (ANIDS). Method: A simulation model is developed for generating synthetic network flows inserting malicious fragments that reflect Denial of Service (DoS) and Probe attacks. An ANIDS shall maximize true positives and true negatives which is equivalent to minimizing Type-I and Type-II errors. The geometric mean of True Positive Rate (TPR) and True Negative Rate (TNR) is proposed as a metric, namely, Geometric Mean Accuracy Index (GMAI) for measuring the performance of any proposed ANIDS. Findings: The task of detecting anomalous network flows by inspecting at fragment level boils down to discrete binary classification problem. The Receiver Operating Characteristic (ROC) curve considers False Positive Rates (FPR) and True Positive Rate (TPR) only. It does not reflect the minimization of Type-I and Type-II errors. Maximizing GMAI is the reflection of minimizing 1-GMAI which is equivalent to minimizing Type-I and Type-II errors. Further, the GMAI can be employed as service level for evaluating acceptance sampling based ANIDS. The domain of DoS and Probe attacks, mostly employed by the intruders at fragment level is studied. A conceptual simulation model is developed for generating synthetic network flows incorporating malicious fragments randomly from the domain of DoS and Probe attacks. The conceptual model is translated into operational model (a set computer programs) and synthetic network flows are generated. Using the operational model, the 1000 synthetic network flows are generated for each percentage of anomalous flows varying from 0.1 to 0.9 and employing discrete uniform probability distribution for selecting a fragment for transforming it into malicious. The generated network flows for each percentage of anomalous flows are represented graphically as histogram. It is found that they follow discrete uniform distribution. Hence, the model is validated. Applications: The simulation model can be used for generating synthetic networks flows for evaluating ANIDS. The GMAI can be used as service level for evaluating a discrete binary classifier irrespective of domain.