RPAD: An Unsupervised HTTP Request Parameter Anomaly Detection Method

Abstract

Web servers in the Internet are vulnerable to Web attacks. A general way to launch Web attacks is to carry attack payloads in HTTP request parameters, e.g. SQL Injection and XSS attacks. To detect Web attacks, a commonly used method is to detect anomalies in the request parameters by making regular-expression-based matching rules for the parameters based on known security threats. However, such methods cannot detect unknown anomalies well and they can also be easily bypassed by using techniques like transcoding. Moreover, existing anomaly detection methods are usually based on supervised learning methods that require a large number of high-quality labelled samples as training sets, which are difficult to obtain in real situations. In this paper, we propose an unsupervised HTTP Request Parameter Anomaly Detection method called RPAD. RPAD uses five features of HTTP request parameters to perform anomaly detection including type, length, number of tokens, encoding type and character feature. After extracting the five features, RPAD uses the DBSCAN algorithm to cluster the parameters of each target access request and outputs the outliers found in the clustering process as anomalies. We evaluate the performance of RPAD on several datasets from multiple real websites of a Cyber Security Company. The results indicate that RPAD is highly efficient in detecting deviating abnormal parameter values with an accuracy of 99%.