Fuzz Testing Research Articles

TaPaFuzz: Hardware-accelerated RISC-V bare-metal firmware fuzzing using rapid job launches

Fuzz testing serves as a key technique in software security aimed at identifying unexpected program behaviors by repeatedly executing the target program with auto-generated random inputs. Testing is integral to IoT device security but is hampered by the minimal observability features of typical in-market IoT devices. Moreover, the slow nature of a RISC-V software emulation on x86 host CPUs and the inaccuracies introduced by compiling IoT applications to a different ISA for execution on host systems pose significant challenges. Our software-hardware co-design surmounts these hurdles. Fuzzing jobs are prepared and evaluated on a host computer, while the actual execution with high-throughput tracing is performed on an FPGA. Advances in the host-to-FPGA interface together with an accelerated reset procedure between Fuzzer jobs effectively hide the costly host-FPGA communication, increasing the single-thread fuzzing performance by up to factor 11.7x that of the leading QEMU-based fuzzer AFL++ running on a very fast x86 CPU. We demonstrate practical usability by evaluating our framework on a collection of bare-metal applications.

Developing a framework for enhancing security testing of android applications

Mobile applications have advanced a lot and now offer several features that help make our lives easier. Android is currently the most popular mobile operating system, and it is susceptible to exploitation attempts by malicious entities. This has led to an increased focus on the security of Android applications. This dissertation proposed the development of a framework which provides a systematic approach to testing the security of Android applications. This framework was developed based on a comprehensive review of existing security testing methodologies and tools. In achieving the study objectives, a test application was run on an emulator, Burp Suite was used as a proxy tool to capture HTTP and HTTPS traffic for analysis, reverse engineering was carried out, static and dynamic analysis were executed, network traffic was captured and analysed with tcpdump and Wireshark, intent sniffing was carried out, fuzz testing was discussed, and a proof-of-concept tool (automation script) was developed. This work covers various aspects of Android applications’ security testing, and the proposed framework provides developers with a practical and effective approach to testing the security of their Android applications, thereby improving the overall security of the Android application ecosystem.

Coverage-guided fuzz testing method based on reinforcement learning seed scheduling

Abstract The existing fuzz testing methods for industrial control protocols suffer from insufficient coverage, false positives, and an inability to handle protocol semantics. This paper proposes a reinforcement learning-based seed scheduling coverage-guided fuzz testing method. Building upon coverage-guided fuzz testing techniques, we integrate reinforcement learning with seed scheduling to optimize the seed selection strategy, thereby enhancing the efficiency of protocol vulnerability detection. Experimental results demonstrate the feasibility and effectiveness of this approach. Through reinforcement learning guidance, seed scheduling is optimized, thereby strengthening the performance of fuzz testing in exploring vulnerabilities in industrial control protocols.

IEmu: Interrupt modeling from the logic hidden in the firmware

The security of embedded firmware has become a critical issue in light of the rapid development of the Internet of Things. Current security analysis approaches, such as dynamic analysis, still face bottlenecks and difficulties due to the wide variety of devices and systems. Recent dynamic analysis approaches for embedded firmware have attempted to provide a general solution but heavily rely on detailed device manuals. Meanwhile, approaches that do not rely on manuals have randomness in interrupt triggering, which weakens emulation fidelity and dynamic analysis efficiency. In this paper, we propose a redundant-check-based embedded firmware interrupt modeling and security analysis method that does not rely on commercial manuals. This method involves reverse engineering the control flow of firmware binary and accurately extracting the correct interrupt triggering rules to emulate embedded firmware. We have implemented functional prototypes on QEMU, called IEmu, and evaluated it with 26 firmware in different MCUs. Our results demonstrate significant advantages compared to the recent state-of-the-art approach. On average, IEmu has improved interrupt path exploration efficiency by 2.4 times and fuzz testing coverage by 19%. IEmu restored the interrupt triggering logic in the manual, and emulated three firmware where the state-of-the-art emulator have limitations and found vulnerabilities.

Timing Side-Channel Mitigation via Automated Program Repair

Side-channel vulnerability detection has gained prominence recently due to Spectre and Meltdown attacks. Techniques for side-channel detection range from fuzz testing to program analysis and program composition. Existing side-channel mitigation techniques repair the vulnerability at the IR/binary level or use runtime monitoring solutions. In both cases, the source code itself is not modified, can evolve while keeping the vulnerability, and the developer would get no feedback on how to develop secure applications in the first place. Thus, these solutions do not help the developer understand the side-channel risks in her code and do not provide guidance to avoid code patterns with side-channel risks. In this paper, we present Pendulum , the first approach for automatically locating and repairing side-channel vulnerabilities in the source code, specifically for timing side channels. Our approach uses a quantitative estimation of found vulnerabilities to guide the fix localization, which goes hand-in-hand with a pattern-guided repair. Our evaluation shows that Pendulum can repair a large number of side-channel vulnerabilities in real-world applications. Overall, our approach integrates vulnerability detection, quantization, localization, and repair into one unified process. This also enhances the possibility of our side-channel mitigation approach being adopted into programming environments.

A novel generative adversarial network-based fuzzing cases generation method for industrial control system protocols

With the interconnection and networking of industrial control systems (ICS), security vulnerabilities in ICS protocols have become a major source of threats to these systems. In order to discover these vulnerabilities and maintain ICS security, generative adversarial network-based fuzz testing (Fuzzing) has been introduced as a popular method to discover vulnerabilities in industrial control system protocols (ICSPs). However, the current approach based on generative adversarial network fuzz testing suffers from problems in terms of compatibility and repetitive generation of test examples, and low acceptance rate. To address these issues, this paper proposes a fuzzing approach based on the synergy of Wasserstein Generative Intelligent Networks (WGAN) and Variation Autoencoder (VAE). In addition, we design an intelligent fuzz framework, WGGFuzz, to efficiently generate a large number of test cases while ensuring the accuracy of the testing results for efficiently generating a large number of test cases in a short period of time. The framework, which does not incorporate ICSPs applicable to a wide range of, shows through experiments that WGGFuzz can be able to achieve 91% in terms of TCRR, 1.41 in terms of ATE, and better performance in terms of DGD.

A fresh look on semantics of Concurrent State Based Language (ConStaBL)

Statechart is a visual modelling language for systems. Our variant of the statechart has local variables, which interact significantly with the remainder of the language semantics. It allows arbitrary interleaving of concurrently executing action code, which allows more precise modelling of systems and upstream analysis of the same. We also establish the criteria based on our semantics for defining conflicting transitions and valid simulations. Our semantics do not allow transition conflicts in simulations and are stricter than most other available semantics of statecharts in that sense. In this paper, we extend the work on modular statecharts and present updated operational semantics with concurrency. We present the operational semantics in the form of the simulation algorithm. Our executable semantics can be used to simulate statechart models and verify their correctness. We present a preliminary setup to carry out fuzz testing of statechart models, an idea that does not have precedent in the literature to the best of our knowledge. We have used our simulator in conjunction with a well-known fuzzer to do fuzz testing of statechart models of non-trivial sizes and have found issues in them that would have been hard to find through inspection.

An Analysis of Network Protocol Vulnerability Mining Using Fuzz Testing Combined with Deep Learning Models

An Analysis of Network Protocol Vulnerability Mining Using Fuzz Testing Combined with Deep Learning Models

CriticalFuzz: A critical neuron coverage-guided fuzz testing framework for deep neural networks

Context:Deep neural networks (DNN) have been widely deployed in safety-critical domains, such as autonomous cars and healthcare, where error behaviors can lead to serious accidents, testing DNN is extremely important. Neuron coverage-guided fuzz testing (NCFT) has become an effective whitebox testing approach for testing DNN, which iteratively generates new test cases with the guidance of neuron coverage to explore different logics of DNN, and has found numerous defects. However, existing NCFT approaches ignore that the role of neurons is distinct for the final output of DNN. Given an input, only a fraction of neurons determines the final output of the DNN. These neurons hold the essential logic of the DNN. Objective:To ensure the quality of DNN and improve testing efficiency, NCFT should first cover neurons containing major logic of DNN. Method:In this paper, we propose the critical neurons that hold essential logic of DNN. In order to prioritize the detection of potential defects of critical neurons, we propose a fuzz testing framework, named CriticalFuzz, which mainly contains the energy-based test case generation and the critical neuron coverage criteria. The energy-based test case generation has the capability to produce test cases that are more likely to cover critical neurons and involves energy-based seed selection, power schedule, and seed mutation. The critical neuron coverage as a mechanism for providing feedback to guide the CriticalFuzz in prioritizing the coverage of critical neurons. To evaluate the significance of critical neurons and the performance of CriticalFuzz, we conducted experiments on popular DNNs and datasets. Results:The experiment results show that (1) the critical neurons have a 100% impact on the output of models, while the non-critical neurons have a lesser effect; (2) CriticalFuzz is effective in achieving 100% coverage of critical neurons and covering 10 classes of critical neurons, outperforming both DeepHunter and TensorFuzz. (3) CriticalFuzz exhibits exceptional error detection capabilities, successfully identifying thousands of errors across 10 diverse error classes within DNN. Conclusion:The critical neurons defined in this paper hold more significant logic of DNN than non-critical neurons. CriticalFuzz can preferentially cover critical neurons, thereby improving the efficiency of the NCFT process. Additionally, CriticalFuzz is capable of identifying a greater number of errors, thus enhancing the reliability and effectiveness of the NCFT.

Sniping at web applications to discover input-handling vulnerabilities

Web applications play a crucial role in modern businesses, offering various services and often exposing sensitive data that can be enticing to attackers. As a result, there is a growing interest in finding innovative approaches for discovering vulnerabilities in web applications. In the evolving landscape of web security, the realm of fuzz testing has garnered substantial attention for its effectiveness in identifying vulnerabilities. However, existing literature has often underemphasized the nuances of web-centric fuzzing methodologies. This article presents a comprehensive exploration of fuzzing techniques specifically tailored to web applications, addressing the gap in the current research. Our work presents a holistic perspective on web-centric fuzzing, introduces a modular architecture that improves fuzzing effectiveness, demonstrates the reusability of certain fuzzing steps, and offers an open-source software package for the broader security community. By addressing these key contributions, we aim to facilitate advancements in web application security, empower researchers to explore new fuzzing techniques, and ultimately enhance the overall cybersecurity landscape.

Dynamic fuzz testing of UAV configuration parameters based on dual guidance of fitness and coverage

ArduCopter's configuration parameter verification defects may cause the Unmanned Aerial Vehicle (UAV) in abnormal status. However, traditional UAV configuration parameter defect detection methods based on fuzz testing lack guidance design and inadequately detect configuration parameter defects. This paper proposes an improved configuration security defect analysis method based on fuzz testing. Using the fitness feedback mechanism based on the CAG neural network to guide the generation of fuzz testing cases, and using multiple coverage feedback mechanisms to guide the exploration direction of fuzz testing. Experimental results show that this method almost covers ArduCopter's position and attitude controller, guiding the UAV into abnormal states such as spin and crash, and detecting specific instances of configuration parameter defects.

Bud hunting with directed fuzz testing and source code vulnerability detection with advanced graph neural networks

Bud hunting with directed fuzz testing and source code vulnerability detection with advanced graph neural networks

Development of the method for fuzz testing mail services based on an intelligent input parameter mutation

This article proposes a method of fuzz testing mail services based on an intelligent input parameter mutation, which is applicable not only to standard email protocols but also to their extended versions. The research examines the main extensions of POP, IMAP, and SMTP protocols. The described method was applied in laboratory conditions to evaluate its effectiveness compared to traditional fuzz testing methods. The results confirmed the possibility of using the method for fuzz testing not only email clients but also servers. Further research is planned to create intelligent methods for evaluating testing coverage in the absence of prior instrumentation of email client and server source code.

Development of a method for fuzz testing of programs for file parsing using the PolyTracker tool

The article proposes a method of phasing-testing programs for parsing using the PolyTracker tool. During the research, types of parsers are considered and their disadvantages are revealed. The proposed method eliminates the identified drawbacks, increasing code coverage and leveling the problem of state explosion.

CrossFuzz: Cross-contract fuzzing for smart contract vulnerability detection

Context:Smart contracts are computer programs that run on a blockchain. As the functions implemented by smart contracts become increasingly complex, the number of cross-contract interactions within them also rises. Consequently, the combinatorial explosion of transaction sequences poses a significant challenge for smart contract security vulnerability detection. Existing static analysis-based methods for detecting cross-contract vulnerabilities suffer from high false-positive rates and cannot generate test cases, while fuzz testing-based methods exhibit low code coverage and may not accurately detect security vulnerabilities. Objective:The goal of this paper is to address the above limitations and efficiently detect cross-contract vulnerabilities. To achieve this goal, we present CrossFuzz, a fuzz testing-based method for detecting cross-contract vulnerabilities. Method:First, CrossFuzz generates parameters of constructors by tracing data propagation paths. Then, it collects inter-contract data flow information. Finally, CrossFuzz optimizes mutation strategies for transaction sequences based on inter-contract data flow information to improve the performance of fuzz testing. Results:We implemented CrossFuzz, which is an extension of ConFuzzius, and conducted experiments on a real-world dataset containing 396 smart contracts. The results show that CrossFuzz outperforms xFuzz, a fuzz testing-based tool optimized for cross-contract vulnerability detection, with a 10.58% increase in bytecode coverage. Furthermore, CrossFuzz detects 1.82 times more security vulnerabilities than ConFuzzius. Conclusion:Our method utilizes data flow information to optimize mutation strategies. It significantly improves the efficiency of fuzz testing for detecting cross-contract vulnerabilities.

Towards Unveiling Exploitation Potential With Multiple Error Behaviors for Kernel Bugs

Nowadays, fuzz testing has significantly expedited the vulnerability discovery of Linux kernel. Security analysts use the manifested error behaviors to infer the exploitability of one bug and thus prioritize the patch development. However, only using an error behavior in the report, security analysts might underestimate the exploitability of the kernel bug because it could manifest various error behaviors indicating different exploitation potentials. In this work, we conduct an empirical study on multiple error behaviors of kernel bugs to understand 1) the prevalence of multiple error behaviors and the possible impact of multiple error behaviors towards the exploitation potential; 2) the factors that manifest multiple error behaviors with different exploitation potential. We collected <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">all the fixed kernel bugs</i> reported on Syzbot from September 2017 to January 2022, including 3,352 bug reports. We observed that multiple error behaviors manifested by kernel bugs are prevalent in the real world, and more error behaviors help unveil the exploitability of kernel bugs. Then we organized Linux kernel experts to analyze a sample of kernel bug dataset (484 bug reports, unique 162 bugs) and identified 6 key contributing factors to the mutiple error behaviors. Finally, based on the empirical findings, we propose an object-driven fuzzing technique to explore all possible error behaviors that a kernel bug might bring about. To evaluate the utility of our proposed technique, we implement our fuzzing tool <monospace xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">GREBE</monospace> and apply it to 60 real-world Linux kernel bugs. On average, <monospace xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">GREBE</monospace> could manifest 2+ additional error behaviors for each of the kernel bugs. For 26 kernel bugs, <monospace xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">GREBE</monospace> discovers higher exploitation potential. We report to kernel vendors some of the bugs – the exploitability of which was wrongly assessed and the corresponding patch has not yet been carefully applied – resulting in their rapid patch adoption.

A Fuzz Testing Approach for Embedded Avionic Software

The objective of the research was to find if it was possible to apply fuzz testing, a technique that can be used to test software to discover vulnerabilities, on an embedded avionic software using an open-source fuzz tool. The open-source fuzz tool AFL++ was applied to an NH90 Airbus Helicopter embedded software component to find the vulnerabilities. The proposed setup was able to find a few crashes related to data parsing associated with Ada strong typing declaration requirement. Moreover, the experiments outline practical guidelines and considerations for implementing fuzz testing on embedded avionic software.

A Control Flow Graph Optimization Method for Enhancing Fuzz testing

A Control Flow Graph Optimization Method for Enhancing Fuzz testing

Machine Learning-Based Fuzz Testing Techniques: A Survey

Machine Learning-Based Fuzz Testing Techniques: A Survey

Formal-Guided Fuzz Testing: Targeting Security Assurance From Specification to Implementation for 5G and Beyond

Formal-Guided Fuzz Testing: Targeting Security Assurance From Specification to Implementation for 5G and Beyond