Discovering abnormal behaviors via HTTP header fields measurement

Abstract

SummaryIn recent years, more and more intrusion detection systems and firewalls have been used to detect and block malicious applications or unknown protocols in order to enhance the security of systems. Therefore, some malicious applications begin to shape themselves as common ones to escape malicious protocol detection. Being an important protocol for many Internet services, hypertext transfer protocol (HTTP) is responsible for nearly 10% of the traffic volume on the Internet. Therefore, many malicious applications pretend their traffic to be HTTP protocol to go into hiding their malicious behaviors. In the paper, we study the problem of discovering these abnormal behaviors in HTTP protocol traffic. We find that the characteristics of many abnormal behaviors are performed in the header fields of their shaping HTTP such as Tor and malicious web crawlers, and the information of HTTP header fields of HTTP traffic generated by normal application is also discussed. And then, a method based on the measurement of HTTP header fields proposed three patterns that make them specific to detect abnormal behaviors of shaping HTTP protocol. The experimental results indicate that the proposed method is effective for abnormal behaviors by shaping to be HTTP on large‐scale traffic of one Internet service provider. The experimental results also show that the proposed method could be extended to large‐scale and high‐speed network environment for detecting abnormal behaviors of shaping HTTP protocol. Copyright © 2016 John Wiley & Sons, Ltd.