Semi-supervised learning approach for malicious URL detection via adversarial learning1

Abstract

Uniform Resource Location (URL) is the network unified resource location system that specifies the location and access method of resources on the Internet. At present, malicious URL has become one of the main means of network attack. How to detect malicious URL timely and accurately has become an engaging research topic. The recent proposed deep learning-based detection models can achieve high accuracy in simulations, but several problems are exposed when they are used in real applications. These models need a balanced labeled dataset for training, while collecting large numbers of the latest labeled URL samples is difficult due to the rapid generation of URL in the real application environment. In addition, in most randomly collected datasets, the number of benign URL samples and malicious URL samples is extremely unbalanced, as malicious URL samples are often rare. This paper proposes a semi-supervised learning malicious URL detection method based on generative adversarial network (GAN) to solve the above two problems. By utilizing the unlabeled URLs for model training in a semi-supervised way, the requirement of large numbers of labeled samples is weakened. And the imbalance problem can be relieved with the synthetic malicious URL generated by adversarial learning. Experimental results show that the proposed method outperforms the classic SVM and LSTM based methods. Specially, the proposed method can obtain high accuracy with insufficient labeled samples and unbalanced dataset. e.g., the proposed method can achieve 87.8% /91.9% detection accuracy when the number of labeled samples is reduced to 20% /40% of that of conventional methods.