Improving ModSecurity WAF Using a Structured-Language Classifier

Abstract

Cyber security have always been a major concern for internet applications and the demand for website protection is on the rise. Web Application Firewalls (WAFs) services are commonly used, as they are convenient as plug-and-play services and provide protection against multiple type of attacks. However, they have high false positive rates. Their rules are often concrete to provide one-size-fit-all services, so rule-based WAFs are either too strict that they block all the incoming requests, or too general that they do not block any malicious requests at all. A feasible solution to concrete rule-based WAFs is applying machine learning approaches, which can help WAFs to monitor and adapt with each specific situation of website and application. Many WAF providers have already migrated to this approaches: Cloudfare, Amazon, Fortinet, etc. Most of these effort concentrated on observing users’ behavior. It is hard to draw a line between normal behavior and malicious behavior, and they require enormous machine learning models. Therefore, we have developed a simple machine learning system to categorize the requests and support traditional WAFs, with the goal of eliminating the high false positive rates and providing better a surveillance system for web applications. Our model is based on our observation that legitimate requests to a website are usually similar. The module uses CNN to categorize the network requests and determine whether each incoming request is abnormal. The output of our model is combined with the result of a rule-based WAF (ModSecurity in our implementation) to conclude that should the incoming request be blocked or not. In our experiments, the model greatly improve the ModSecurity WAF with false positive rate reduced from 24% to only 3%, keeping pace with other notable studies on using machine learning models to improve WAFs and only requiring processing time 0.05 s per request.