Incremental Update based Closed-Loop Malicious URL Detection System

Abstract

Listen

Translate article

In recent years, deep learning has been widely applied to the detection of malicious URLs and has made a great success. However, the latent patterns of malicious URLs change constantly, which requires frequent model update for the detection system. Existing URL detection system update basically relies on model retaining, which is time-consuming and resource-consuming. In this paper, we propose an incremental update based closed-loop URL detection system, where the URL detection model continually updates to fuse the knowledge of the samples with patterns different from the previously known samples in the training set. We develop a convolutional neural network (CNN) based automatic semantic feature extraction and a fully connected network based URL detection method. In order to detect the samples with new patterns, we conceive a least confidence algorithm based suspicious URL detection. A knowledge distillation based lightweight incremental update method is proposed to rapidly aggregate the knowledge of newly added URLs with only a small part of history data. In this way, a closed-loop URL detection system is formed by automatic feature extraction, initial model training, suspicious URL detection, and detection model update, so that the detection model can keep pace with the evolution of URLs. Finally, the experiments implemented on the public URL dataset ISCX-URL2016 demonstrate that our proposed closed-loop URL detection system and method can obtain a high detection accuracy with a low update time.