Multi-scale Self-Organizing Map assisted Deep Autoencoding Gaussian Mixture Model for unsupervised intrusion detection

Abstract

In an age when the Internet has become the backbone of communications, a robust and safe network environment is critical. Intrusion detection techniques are thus valuable for IT infrastructure. The state of the art (SOTA) solution, Deep Autoencoding Gaussian Mixture Model (DAGMM), outperforms those approaches relying on decoupled two-stage training and the standard Expectation–Maximization optimization algorithm. However, DAGMM suffers from the failure in preserving the input topology, caused by the bottleneck layer of the adopted deep autoencoder as well as the method of constructing the input for the follow-up density estimation. This research first presents a Self-Organizing Map assisted Deep Autoencoding Gaussian Mixture Model (SOM-DAGMM) for overcoming the above-mentioned shortcoming of DAGMM, through well balancing the low-dimensional demand of Gaussian Mixture Model (GMM) and the topology-preserving requirement. The proposed SOM-DAGMM employs a self-organizing map to extract features as a supplement with well-preserved input space topology for better network intrusion detection. The paper also studies the superiority of multi-scale topology over the single-scale one in improving the performance of DAGMM. The better performance of the SOM-DAGMM is empirically proven by extensive experiments involving six datasets. Experimental results show that single/multi-scale SOM-DAGMMs outperform the SOTA DAGMM on all tests and achieve up to 110.38% improvement in F1 score and with better stability.