A Comprehensive Evaluation of HTTP Header Features for Detecting Malicious Websites

Abstract

Security researchers have used website features including the URL, webpage content, HTTP headers, and others to detect malicious websites. In prior research, features derived from HTTP headers have shown promise for malicious website detection. This paper includes a comprehensive evaluation of HTTP header features to assess whether additional HTTP header features improve malicious website detection. We analyze HTTP headers from 6,021 malicious and 39,853 benign websites. We define malicious websites as those identified by Cisco Talos Threat Intelligence Group for association with phishing, drive-by downloads, and command and control infrastructure. Benign websites consist of popular websites from the Alexa Traffic Rank. We collect 672 HTTP header features from these websites and identify 22 for further analysis. Among these, 11 have been studied in prior research while the other 11 are new and identified in our research. From these 22 features, eight features, three identified by our study, consistently rank as the most important features and represent 80% of the total feature importance. We build eight models with supervised learning techniques and observe that the detection performance metrics for the 22 features are consistently better than for the 11 previously studied features. We also apply two feature transformation techniques and find that performing Principal Component Analysis on the features identified increases detection ability. From our results, we postulate that use of additional HTTP header features will lead to more accurate detection of malicious websites.