Grey-Box Fuzzing Based on Reinforcement Learning for XSS Vulnerabilities

Abstract

Cross-site scripting (XSS) vulnerabilities are significant threats to web applications. The number of XSS vulnerabilities reported has increased annually for the past three years, posing a considerable challenge to web application maintainers. Black-box scanners are mainstream tools for security engineers to perform penetration testing and detect XSS vulnerabilities. Unfortunately, black-box scanners rely on crawlers to find input points of web applications and cannot guarantee all input points are tested. To this end, we propose a grey-box fuzzing method based on reinforcement learning, which can detect reflected and stored XSS vulnerabilities for Java web applications. We first use static analysis to identify potential input points from components (i.e., Java code, configuration files, and HTML files) of the Java web application. Then, an XSS vulnerability payload generation method is proposed, which is used together with the reinforcement learning model. We define the state, action, and reward functions of three reinforcement learning models for XSS vulnerability detection scenarios so that the fuzz loop can be performed automatically. To demonstrate the effectiveness of the proposed method, we compare it against four state-of-the-art web scanners. Experimental results show that our method finds all XSS vulnerabilities and has no false positives.