Model-Based Grey-Box Fuzzing of Network Protocols

Abstract

The widely used network protocols play a crucial role in various systems. However, the protocol vulnerabilities caused by the design of the network protocol or its implementation by programmers lead to multiple security incidents and substantial losses. Hence, it is important to study the protocol fuzzing in order to ensure its correctness. However, the challenges of protocol fuzzing are the mutation of protocol messages and the deep interactivity of the protocol implementation. This paper proposes a model-based grey-box fuzzing approach for protocol implementations, including the server-side and client-side. The proposed method is divided into two phases: automata learning based on the minimally adequate teacher (MAT) framework and grey-box fuzzing guided by the learned model and code coverage. The StateFuzzer tool used for evaluation is presented to demonstrate the validity and feasibility of the proposed approach. The server-side fuzzing can achieve similar or higher code coverage and vulnerability discovery capability than those of AFLNET and StateAFL. Considering the client, the results show that it achieves 1.5X branch coverage (on average) compared with the default AFL, and 1.3X branch coverage compared with AFLNET and StateAFL, using the typical implementations such as OpenSSL, LibreSSL, and Live555. The StateFuzzer identifies a new memory corruption bug in Live555 (2021-08-25) and 14 distinct discrepancies based on differential testing.