Testing Error Handling Code With Software Fault Injection and Error-Coverage-Guided Fuzzing

Abstract

Real-world programs require error handling code to handle various kinds of possible errors. However, these errors just infrequently occur due to special conditions, so error handling code is difficult to test. Coverage-guided fuzzing and software fault injection (SFI) are two common techniques that can test error handling code, but they still have major limitations. Specifically, existing fuzzing approaches generate program inputs guided by code coverage, but many occasional errors (such as insufficient memory) are unrelated to inputs, and code coverage cannot effectively reflect the execution contexts of these errors; existing SFI approaches often inject single or random faults, without exploring fault space or using program feedback. In this paper, we propose a new fuzzing framework named EH-Fuzz, to effectively test error handling code. EH-Fuzz uses a context-sensitive SFI-based fuzzing approach to explore fault space and perform fault injection, guided by a new metric named <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">error coverage</i> . We evaluate EH-Fuzz on 9 user-level programs and 6 kernel-level modules, and find 45 new real bugs, 31 of which have been confirmed and fixed. We compare EH-Fuzz to existing fuzzing approaches (including AFL, AFL++, Syzkaller, FIZZER and FIFUZZ), and EH-Fuzz finds many real bugs missed by these approaches with higher testing coverage.