Knowing the unknowns: Network traffic detection with open-set semi-supervised learning

Abstract

Network traffic classification plays a crucial role in network security and network quality of service. The new emerging traffic data in the real world is unlabeled and may contain unknown classes, which introduces the problem of identifying unknown-class traffic in unlabeled data. However, existing deep learning-based methods either rely on large amounts of labeled data or assume the labeled and unlabeled data share the common label space. Therefore, they are unable to identify unknown traffic, resulting in decreased detection accuracy. This paper introduces DivinEye, a novel method for unknown network traffic detection based on open-set semi-supervised learning. DivinEye utilizes a small quantity of labeled data and a large amount of unlabeled data to train the model, achieving the ability to identify known classes accurately while also detecting unknown ones. To leverage incoming unlabeled traffic, DivinEye employs open-set semi-supervised techniques to select known-class data from unlabeled data to optimize the known traffic detection model. To detect unknown traffic, DivinEye combines the known traffic classifier and multi-binary classifier to construct an unknown traffic detection model, generating auxiliary targets for all unlabeled data as supervision to train the model. Trace-driven experiments demonstrate that DivinEye outperforms state-of-the-art methods by a large margin.