A Binary Protocol Fuzzing Method Based on SeqGAN

Abstract

Fuzzing test is an important method to mine the security vulnerabilities of network protocol programs. Most of the existing fuzzing methods are based on protocol specifications. However, these methods will fail if the protocol specifications are unavailable. To solve the problem, some researchers proposed fuzzing methods based on deep learning, which try to use neural networks to learn protocol specifications from network data and generate test cases. However, the methods suffer from problems such as gradient disappearance during model learning. In order to solve the problems of fuzzing methods’ protocol specification dependency and gradient disappearance of fuzzing test method based on deep learning, this paper proposed a binary protocol fuzzing method based on SeqGAN, and the method consists of four stages: data preprocessing, SeqGAN training, test cases generation and fuzzing test. Firstly, the application layer messages are extracted from network datagrams. Secondly, the application layer messages are input into SeqGAN and used to train the SeqGAN model. Thirdly, the generator model in SeqGAN construct test cases. Finally, test cases are fed into the target system, and the exceptions of target system are captured. Experimental results show that the method can effectively discover the vulnerabilities of binary network protocol programs, and it has higher acceptance rate of test input and vulnerability detection ability.