Branch Coverage Research Articles

Patterns of Dynamic Adaptability of the Circle of Willis in Response to Major Branch Artery Coverage With a Flow Diverter

The plasticity of the Circle of Willis represents an underexplored yet intriguing dimension of vascular anatomy in cerebrovascular disorders. We outline distinct patterns of change in response to aneurysm treatment using flow diversion (FD) after covering major branches. Retrospective analysis of digital subtraction angiographies (DSA) from intracranial aneurysms treated with FD from 2013 to 2023. Vessel diameters, including those covered by the stent and adjacent arteries, were measured. Angiographic changes were evaluated at last imaging follow-up. Of the 622 patients, 49 had angiographic follow-up for pattern assessment. The median age was 62 years; females represented 71.4%. The median size of the treated aneurysms was 4.7mm. Four patterns of angiographic change were identified: (1)Patients with supraclinoid aneurysms, A1-ACA caliber increased (hypoplastic: 1.05 to 2.00 mm; non-hypoplastic: 2.45 to 2.75 mm) after FD coverage of the contralateral ACA. (2)Patients with paraclinoid aneurysms and hypoplastic-fetal P1-PCA, the diameter increased from 0.80 to 1.7 mm (p<0.01) after covering the ipsilateral PComA origin. (3)Patients with basilar-tip and proximal PCA aneurysms showed increased ipsilateral PComA size from 1.2 to 2 mm (p<0.01) after PCA origin coverage. (4)Patients with anterior communicating aneurysms, the diameter of the contralateral hypoplastic A1 segment increased from 1.0 to 1.35 mm (p=0.39) or non-hypoplastic A1-ACA from 2.75 to 3.05 mm (p=0.10) after FD coverage. The circle of Willis displays both hemodynamic and anatomic plasticity after major branch coverage with a flow diverter. This phenomenon is aimed at preserving blood flow in the distal territory of the covered vessel.

LPBFuzz: Binary program fuzzing with enhanced coverage of low-frequency program branches

Abstract Fuzzing detects hidden defects and vulnerabilities in software by generating a large amount of malformed input data and monitoring program anomalies. The existing fuzzing methods suffer from data imbalance when modeling program branching behaviors, making it difficult to accurately locate data fields that are strongly correlated with low-frequency program branching decisions. In addition, the existing methods do not determine variation direction when mutating the test data and do not distinguish between valid/invalid data fields, resulting in duplicate testing of certain program branches. In this paper, we propose a fuzzing method to enhance the coverage of low-frequency program branches. A low-frequency perception network and a forward gradient-guided mutating strategy are proposed to clarify the magnitude and sign when mutating key data fields. Experiments show that the proposed method effectively improves code coverage by increasing low-frequency branch coverage.

A3Test: Assertion-Augmented Automated Test case generation

Context:Test case generation is a critical yet challenging task in software development. Recently, AthenaTest – a Deep Learning (DL) approach for generating unit test cases has been proposed. However, our revisiting study reveals that AthenaTest can generate less than one-fifth of the test cases correctly, due to a lack of assertion knowledge and test signature verification. Objective:This paper introduces A3Test, a novel DL-based approach to the generation of test cases, enhanced with assertion knowledge and a mechanism to verify consistency of the name and signatures of the tests. A3Test aims to adapt domain knowledge from assertion generation to test case generation. Method:A3Test employs domain adaptation principles and introduces a verification approach to name consistency and test signatures. We evaluate its effectiveness using 5,278 focal methods from the Defects4j dataset. Results:Our findings indicate that A3Test outperforms AthenaTest and ChatUniTest. A3Test generates 2.16% to 395.43% more correct test cases, achieves 2.17% to 34.29% higher method coverage, and 25.64% higher line coverage. A3Test achieves 2.13% to 12.20% higher branch coverage, 2.22% to 12.20% higher mutation scores, and 2.44% to 55.56% more correct assertions compared to both ChatUniTest and AthenaTest respectively for one iteration. When generating multiple test cases per method A3Test still shows improvements and comparable efficacy to ChatUnitTest. A survey of developers reveals that the majority of the participants 70.51% agree that test cases generated by A3Test are more readable than those generated by EvoSuite. Conclusions:A3Test significantly enhances test case generation through its incorporation of assertion knowledge and test signature verification, contributing to the generation of correct test cases.

MSFuzz: Augmenting Protocol Fuzzing with Message Syntax Comprehension via Large Language Models

Network protocol implementations, as integral components of information communication, are critically important for security. Due to its efficiency and automation, fuzzing has become a popular method for protocol security detection. However, the existing protocol-fuzzing techniques face the critical problem of generating high-quality inputs. To address the problem, in this paper, we propose MSFuzz, which is a protocol-fuzzing method with message syntax comprehension. The core observation of MSFuzz is that the source code of protocol implementations contains detailed and comprehensive knowledge of the message syntax. Specifically, we leveraged the code-understanding capabilities of large language models to extract the message syntax from the source code and construct message syntax trees. Then, using these syntax trees, we expanded the initial seed corpus and designed a novel syntax-aware mutation strategy to guide the fuzzing. To evaluate the performance of MSFuzz, we compared it with the state-of-the-art (SOTA) protocol fuzzers, namely, AFLNET and CHATAFL. Experimental results showed that compared with AFLNET and CHATAFL, MSFuzz achieved average improvements of 22.53% and 10.04% in the number of states, 60.62% and 19.52% improvements in the number of state transitions, and 29.30% and 23.13% improvements in branch coverage. Additionally, MSFuzz discovered more vulnerabilities than the SOTA fuzzers.

Fine-grained Coverage-based Fuzzing

Fuzzing is a popular software testing method that discovers bugs by massively feeding target applications with automatically generated inputs. Many state-of-the-art fuzzers use branch coverage as a feedback metric to guide the fuzzing process. The fuzzer retains inputs for further mutation only if branch coverage is increased. However, branch coverage only provides a shallow sampling of program behaviors and hence may discard interesting inputs to mutate. This work aims to take advantage of the large body of research in defining finer-grained code coverage metrics (such as control-flow, data-flow, or mutation coverage) and to evaluate how fuzzing performance is impacted when using these metrics to select interesting inputs for mutation. We propose to make branch coverage-based fuzzers support most fine-grained coverage metrics out of the box (i.e., without changing fuzzer internals). We achieve this by making the test objectives defined by these metrics (such as conditions to activate or mutants to kill) explicit as new branches in the target program. Fuzzing such a modified target is then equivalent to fuzzing the original target, but the fuzzer will also retain inputs covering the additional metric objectives for mutation. In addition, all the fuzzer mechanisms to penetrate hard-to-cover branches will help in covering the additional metric objectives. We use this approach to evaluate the impact of supporting two fine-grained coverage metrics (multiple condition coverage and weak mutation) over the performance of two state-of-the-art fuzzers (AFL++ and QSYM) with the standard LAVA-M and MAGMA benchmarks. This evaluation suggests that our mechanism for runtime fuzzer guidance, where the fuzzed code is instrumented with additional branches, is effective and could be leveraged to encode guidance from human users or static analyzers. Our results also show that the impact of fine-grained metrics over fuzzing performance is hard to predict before fuzzing and most of the time either neutral or negative. As a consequence, we do not recommend using them to guide fuzzers, except maybe in some possibly favorable circumstances yet to be investigated, like for limited parts of the code or to complement classical fuzzing campaigns.

Basis path coverage testing of MPI programs based on multi-task evolutionary optimization

A Message-Passing Interface (MPI) program usually consists of several processes, and a target path of this program is composed of a target sub-path selected in each process. However, there may be coverage conflict between different target sub-paths in the target path, and there is currently no effective approach to avoid this problem. Therefore, this paper proposes an approach of basis path coverage testing of MPI programs based on multi-task evolutionary optimization, which is used to generate a test case set that meets the criteria of branch and sentence coverage. Firstly, the proposed approach samples a certain number of representative test inputs, and calculates the coverage level of each test input for each target sub-path in the target path, which is used to group the target sub-paths with coverage conflict. Then, we construct an optimization model for each group of the target sub-paths, which can guide test case generation with high effectiveness. Finally, each optimization model is solved by an evolutionary algorithm, and the efficiency of solving these optimization models is improved based on a multi-task framework. The proposed approach is applied to basis path coverage testing of seven benchmark MPI programs under test, and compared with several state-of-the-art approaches. Experimental results indicate that the proposed approach has significant advantages in test case generation, as well as performs good scalability.

HotCFuzz: Enhancing Vulnerability Detection through Fuzzing and Hotspot Code Coverage Analysis

Software vulnerabilities present a significant cybersecurity threat, particularly as software code grows in size and complexity. Traditional vulnerability-mining techniques face challenges in keeping pace with this complexity. Fuzzing, a key automated vulnerability-mining approach, typically focuses on code branch coverage, overlooking syntactic and semantic elements of the code. In this paper, we introduce HotCFuzz, a novel vulnerability-mining model centered on the coverage of hot code blocks. Leveraging vulnerability syntactic features to identify these hot code blocks, we devise a seed selection algorithm based on their coverage and integrate it into the established fuzzing test framework AFL. Experimental results demonstrate that HotCFuzz surpasses AFL, AFLGo, Beacon, and FairFuzz in terms of efficiency and time savings.

Incremental Concolic Testing of Register-Transfer Level Designs

Concolic testing is a scalable solution for automated generation of directed tests for validation of hardware designs. Unfortunately, concolic testing fails to cover complex corner cases such as hard-to-activate branches. In this article, we propose an incremental concolic testing technique to cover hard-to-activate branches in register-transfer level (RTL) models. We show that a complex branch condition can be viewed as a sequence of easy-to-activate events. We map the branch coverage problem to the coverage of a sequence of events. We propose an efficient algorithm to cover the sequence of events using concolic testing. Specifically, the test generated to activate the current event is used as the starting point to activate the next event in the sequence. Experimental results demonstrate that our approach can be used to generate directed tests to cover complex corner cases in RTL models while state-of-the-art methods fail to activate them.

Financial inclusion, economic development, and inequality: Evidence from Brazil

We study a financial inclusion policy targeting Brazilian cities with low bank branch coverage using data on the universe of employees from 2000–2014. The policy leads to bank entry and to similar increases in both deposits and lending. It also fosters entrepreneurship, employment, and wage growth, especially for cities initially in banking deserts. These gains are not shared equally and instead increase with workers’ education, implying a substantial increase in wage inequality. The changes in inequality are concentrated in cities where the initial supply of skilled workers is low, indicating that talent scarcity can drive how financial development affects inequality.

DiPri : Distance-based Seed Prioritization for Greybox Fuzzing

Greybox fuzzing is a powerful testing technique. Given a set of initial seeds, greybox fuzzing continuously generates new test inputs to execute the program under test and drives executions with code coverage as feedback. Seed prioritization is an important step of greybox fuzzing that helps greybox fuzzing choose promising seeds for input generation in priority. However, mainstream greybox fuzzers like AFL++ and Zest tend to neglect the importance of seed prioritization. They may pick seeds plainly according to the sequential order of the seeds being queued or an order produced with a random-based approach, which may consequently degrade their performance in exploring code and exposing bugs. In the meantime, existing state-of-the-art techniques like Alphuzz and K-Scheduler adopt complex strategies to schedule seeds. Although powerful, such strategies also inevitably incur great overhead and will reduce the scalability of the proposed technique. In this paper, we propose a novel distance-based seed prioritization approach named DiPri to facilitate greybox fuzzing. Specifically, DiPri evaluates the queued seeds according to seed distances and chooses the outlier ones, which are the farthest from the others, in priority to improve the probabilities of discovering previously unexplored code regions. To make a profound evaluation of DiPri , we prototype DiPri on AFL++ and conduct large-scale experiments with four baselines and 24 C/C++ fuzz targets, where eight are from widely adopted real-world projects, eight are from the coverage-based benchmark FuzzBench, and eight are from the bug-based benchmark Magma. The results obtained through a fuzzing exceeding 50,000 CPU hours suggest that DiPri can (1) insignificantly influence the host fuzzer’s capability of code coverage by slightly improving the branch coverage on the eight targets from real-world projects and slightly reducing the branch coverage on the eight targets from FuzzBench, and (2) improve the host fuzzer’s capability of finding bugs by triggering five more Magma bugs. Besides the evaluation with the three C/C++ benchmarks, we integrate DiPri into the Java fuzzer Zest and conduct experiments on a Java benchmark composed of five real-world programs for more than 8,000 CPU hours to empirically study the scalability of DiPri . The results with the Java benchmark demonstrate that DiPri is pretty scalable and can help the host fuzzer find bugs more consistently.

Implementation Genetic Algorithm for Optimization of Kotlin Software Unit Test Case Generator

Unit testing has a significant role in software development and its impacts depend on the quality of test cases and test data used. To reduce time and effort, unit test generator systems can help automatically generate test cases and test data. However, there is currently no unit test generator for Kotlin programming language even though this language is popularly used for android application developments. In this study, we propose and develop a test generator system that utilizes genetic algorithm (GA) and ANTLR4 parser. GA is used to obtain the most optimal test cases and data for a given Kotlin code. ANTLR4 parser is used to optimize the mutation process in GA so that the mutation process is not totally random. Our model results showed that the average value of code coverage in generated unit tests against instruction coverage is 95.64%, with branch coverage of 76.19% and line coverage of 96.87%. In addition, only two out of eight generated classes produced duplicate test cases with a maximum of one duplication in each class. Therefore, it can be concluded that our optimization with GA on the unit test generator is able to produce unit tests with high code coverage and low duplication.

A Reinforcement Learning Approach to Guide Web Crawler to Explore Web Applications for Improving Code Coverage

Web crawlers are widely used to automatically explore and test web applications. However, navigating the pages of a web application can be difficult due to dynamic page generation. In particular, the inputs for the web form fields can affect the resulting pages and subsequent navigation. Therefore, choosing the inputs and the order of clicks on a web page is essential for an effective web crawler to achieve high code coverage. This paper proposes a set of actions to quickly fill in web form fields and uses reinforcement learning algorithms to train a convolutional neural network (CNN). The trained agent, named iRobot, can autonomously select actions to guide the web crawler to maximize code coverage. We experimentally compared different reinforcement learning algorithms, neural networks, and actions. The results show that our CNN network with the proposed actions performs better than other neural networks in terms of branch coverage using the Deep Q-learning (DQN) or proximal policy optimization (PPO) algorithm. Furthermore, compared to previous studies, iRobot can increase branch coverage by about 1.7% while reducing training time to 12.54%.

A Model of How Students Engineer Test Cases With Feedback

Background and Context. Students’ programming projects are often assessed on the basis of their tests as well as their implementations, most commonly using test adequacy criteria like branch coverage, or, in some cases, mutation analysis. As a result, students are implicitly encouraged to use these tools during their development process (i.e., so they have awareness of the strength of their own test suites). Objectives. Little is known about how students choose test cases for their software while being guided by these feedback mechanisms. We aim to explore the interaction between students and commonly used testing feedback mechanisms (in this case, branch coverage and mutation-based feedback). Method. We use grounded theory to explore this interaction. We conducted 12 think-aloud interviews with students as they were asked to complete a series of software testing tasks, each of which involved a different feedback mechanism. Interviews were recorded and transcripts were analyzed, and we present the overarching themes that emerged from our analysis. Findings. Our findings are organized into a process model describing how students completed software testing tasks while being guided by a test adequacy criterion. Program comprehension strategies were commonly employed to reason about feedback and devise test cases. Mutation-based feedback tended to be cognitively overwhelming for students, and they resorted to weaker heuristics in order to address this feedback. Implications. In the presence of testing feedback, students did not appear to consider problem coverage as a testing goal so much as program coverage . While test adequacy criteria can be useful for assessment of software tests, we must consider whether they represent good goals for testing, and if our current methods of practice and assessment are encouraging poor testing habits.

Test data generation for covering mutation-based path using MGA for MPI program

Message Passing Interface (MPI) is a communication protocol used for parallel programming in various languages, valued for its reliability and broad applicability. Mutation testing is a software testing method for systematically simulating software faults. However, the significant number of inserted mutation branches in the program escalates the testing cost. To address this issue, we propose the comprehensive PMTDGM framework. The framework first generates mutation-based paths based on the relevancy of mutant branches and the difficulty of mutant branch coverage, followed by the establishment of a multitask model of path coverage. Finally, we employ a multi-population genetic algorithm (MGA) to generate test data. Our experiments, performed on six MPI programs of varying sizes and structures, demonstrate that the mutation-based paths have the small test sets and are easy to cover. Additionally, the multitask model and MGA can significantly improve the efficiency of generating test data and reduce the cost of mutation testing for MPI programs compared to traditional methods.

A Bioinspired Test Generation Method Using Discretized and Modified Bat Optimization Algorithm

The process of software development is incomplete without software testing. Software testing expenses account for almost half of all development expenses. The automation of the testing process is seen to be a technique for reducing the cost of software testing. An NP-complete optimization challenge is to generate the test data with the highest branch coverage in the shortest time. The primary goal of this research is to provide test data that covers all branches of a software unit. Increasing the convergence speed, the success rate, and the stability of the outcomes are other goals of this study. An efficient bioinspired technique is suggested in this study to automatically generate test data utilizing the discretized Bat Optimization Algorithm (BOA). Modifying and discretizing the BOA and adapting it to the test generation problem are the main contributions of this study. In the first stage of the proposed method, the source code of the input program is statistically analyzed to identify the branches and their predicates. Then, the developed discretized BOA iteratively generates effective test data. The fitness function was developed based on the program’s branch coverage. The proposed method was implemented along with the previous one. The experiments’ results indicated that the suggested method could generate test data with about 99.95% branch coverage with a limited amount of time (16 times lower than the time of similar algorithms); its success rate was 99.85% and the average number of required iterations to cover all branches is 4.70. Higher coverage, higher speed, and higher stability make the proposed method suitable as an efficient test generation method for real-world large software.

Effective Test Cases Generation with Harmony Search and RBF Neural Network

Software testing is one of the integral activities during development of software products. Generation and selection of the test cases either in static or dynamic form play pivot role for ensuring the quality of software products. There are numerous approaches in the literature for automatic generation of test cases but coverage criteria and fault detection rate are prominent metrics for checking the effectiveness of the software products during testing phase of software development. In the present, a new Harmony Radial Testing (HRT) is proposed by combining the concepts of Harmony Search Algorithm (HSA) and Radial Basis Function-Neural Network (RBF-NN) approaches. The main objective of the proposed HRT method is to generate automatic test cases by considering the criteria of branch coverage with improvement in the Maximum branch Coverage (MaxC), Average Coverage (AC) and Average Percentage Fault Detection (APFD) rates. The proposed approach combined with the Radial Basis Function (RBF), denoted as a HRT approach. The proposed approach is used to optimize harmony search over the randomly selected sample test cases, training the RBF-NN to simulate the fitness function. Seven Python codes have been tested through proposed approach and computed results are compared with Primal-Dual Genetic Algorithm (PDGA), Simple Genetic Algorithm (SGA) and random methods. It is observed that the proposed HRT algorithm optimizes consistently yielded reliable results, which may be used in future for enriching the software testing process by the software industries.

Deep Learning for Coverage-Guided Fuzzing: How Far are We?

Fuzzing is a widely-used software vulnerability discovery technology, many of which are optimized using coverage-feedback. Recently, some techniques propose to train deep learning (DL) models to predict the branch coverage of an arbitrary input owing to its always-available gradients etc. as a guide. Those techniques have proved their success in improving coverage and discovering bugs under different experimental settings. However, DL models, usually as a magic black-box, are notoriously lack of explanation. Moreover, their performance can be sensitive to the collected runtime coverage information for training, indicating potentially unstable performance. In this work, we conduct a systematic empirical study on 4 types of DL models across 6 projects to (1) revisit the performance of DL models on predicting branch coverage (2) demystify what specific knowledge do the models exactly learn, (3) study the scenarios where the DL models can outperform and underperform the traditional fuzzers, and (4) gain insight into the challenges of applying DL models on fuzzing. Our empirical results reveal that existing DL-based fuzzers do not perform well as expected, which is largely affected by the dependencies between branches, unbalanced sample distribution, and the limited model expressiveness. In addition, the estimated gradient information tends to be less helpful in our experiments. Finally, we further pinpoint the research directions based on our summarized challenges.

TAEF: A Task Allocation-Based Ensemble Fuzzing Framework for Optimizing the Advantages of Heterogeneous Fuzzers

Ensemble fuzzing in parallel with heterogeneous fuzzers has been proposed to leverage the advantages of diverse fuzzers and improve testing efficiency. However, in current ensemble fuzzing methods, the collaboration among different fuzzers is achieved solely by synchronizing the seeds discovered by each fuzzer. This results in a high likelihood of different fuzzers choosing the same seeds and creating a large number of equivalent testcases, thus reducing overall fuzzing efficiency. Meanwhile, the existing task division method proposed by AFLTeam is highly coupled with the fuzzer specially designed for it, making it challenging to apply to ensemble fuzzing directly. So, in this paper, we proposed a callgraph-based task division method suitable for ensemble fuzzing. Firstly, we divided the target program’s callgraph into subgraphs (subtasks) balancing expected workloads. Then, we divided the global seed corpus into subcorpora, each corresponding to a subtask, making fuzzers easily accept the subtasks. Finally, we designed synchronization mechanisms for coverage bitmaps and seeds to realize the collaborative fuzzing among different fuzzers and a cyclic subtask scheduling strategy to fully leverage the benefits of ensemble fuzzing. We implemented a prototype called TAEF. The evaluation results show that in the best-case scenario, our method has up to 24% more branch coverage than previous work.

Valkyrie: Improving fuzzing performance through deterministic techniques

Greybox fuzzing has received much attention from developers and researchers due to its success in discovering bugs within many programs. However, randomized algorithms have limited fuzzers’ effectiveness. First, branch coverage feedback that is based on random edge ID can lead to branch collision. Besides, state-of-the-art fuzzers heavily rely on randomized methods to reach new coverage. Finally, some state-of-the-art fuzzers only employ heuristics-based bug exploitation methods, which are not effective in triggering those that require non-trivial triggering conditions.We believe deterministic techniques deliver consistent and reproducible results. We propose Valkyrie, a greybox fuzzer whose performance is boosted primarily by deterministic techniques. Valkyrie combines collision-free branch coverage with context sensitivity to maintain accuracy while introducing an instrumentation removal algorithm to reduce overhead. It also pioneers a new mutation method, compensated step, allowing fuzzers that use solvers to adapt to real-world fuzzing scenarios without randomness. Additionally, Valkyrie proactively identifies possible exploit points in target programs and utilizes solvers to trigger actual bugs. We implement and evaluate Valkyrie’s effectiveness on the standard benchmark Magma, and a wide variety of real-world programs. Valkyrie triggered 21 unique integer and memory errors, 10.5% and 50% more than AFL++ and Angora, respectively. Valkyrie reached 8.2% and 12.4% more branches in real-world programs, compared with AFL++ and Angora, respectively. We also verify that our branch counting and mutation method is better than the state-of-the-art, which shows that deterministic techniques trump random techniques in consistency, reproducibility, and performance.

BSFuzz: Branch-State Guided Hybrid Fuzzing

Hybrid fuzzing is an automated software testing approach that synchronizes test cases between the fuzzer and the concolic executor to improve performance. The concolic executor solves path constraints to direct the fuzzer to explore the uncovered path. Despite many performance optimizations for hybrid fuzzing, we observe that the concolic executor often repeatedly performs constraint solving on branches with unsolvable constraints and branches covered by multiple test cases. This can cause significant computational redundancies. To be efficient, we propose BSFuzz, which keeps tracking the coverage state and solving state in a lightweight branch state map. BSFuzz synchronizes the current coverage state of all test cases from the fuzzer’s queue with the concolic executor in a timely manner to reduce constraint solving for high-frequency branches. It also records the branch-solving state during the concolic execution to reduce repeated solving of unsolvable branches. Guided by the coverage state and historical solving state, BSFuzz can efficiently discover and solve more branches. The experimental results with real-world programs demonstrate that BSFuzz can effectively increase the speed of the concolic executor and improve branch coverage.