Data-Driven Network Anomaly Detection with Cyber Attack and Defense Visualization

Abstract

The exponential growth in data volumes, combined with the inherent complexity of network algorithms, has drastically affected network security. Data activities are producing voluminous network logs that often mask critical vulnerabilities. Although there are efforts to address these hidden vulnerabilities, the solutions often come at high costs or increased complexities. In contrast, the potential of open-source tools, recognized for their security analysis capabilities, remains under-researched. These tools have the potential for detailed extraction of essential network components, and they strengthen network security. Addressing this gap, our paper proposes a data analytics-driven network anomaly detection model, which is uniquely complemented with a visualization layer, making the dynamics of cyberattacks and their subsequent defenses distinctive in near real-time. Our novel approach, based on network scanning tools and network discovery services, allows us to visualize the network based on how many IP-based networking devices are live, then we implement a data analytics-based intrusion detection system that scrutinizes all network connections. We then initiate mitigation measures, visually distinguishing malicious from benign connections using red and blue hues, respectively. Our experimental evaluation shows an F1 score of 97.9% and a minimal false positive rate of 0.3% in our model, demonstrating a marked improvement over existing research in this domain.