A mutation-based fuzz testing approach for network protocol vulnerability detection

Abstract

Mutation-based fuzz testing is a very effective approach to improve the security and reliability of protocol implementations, however, present mutation-based fuzzing technique is only effective on one field at one time, and not effective on multiple fields. What's more, there are not effective tools on fuzzing protocols based on MAC Layer. This paper presents an efficient mutation-based approach based on RATM model for multi-fields fuzzing test, which analysis the relationship among the protocol fields and the influence relationship of the fields collected. Based on these relationships, it can directly mutate corresponding testing case that might trigger the suspect vulnerabilities. And also by analyzing the results of testing, it changes the parameters of RATM to improve the quality of testing cases; besides, our methods could also fuzz protocol implementations based on MAC layer very effectively. Moreover, we design a testing framework to validate the effectiveness of RATM. We applied our approach to compare with peach on several protocol implementations and the results showed that our approach can find flaws of protocol implementation in one field and also multiple fields very effectively.