Data Flow Coverage Research Articles

Empirical Comparison of Higher-Order Mutation Testing and Data-Flow Testing of C# with the Aid of Genetic Algorithm

Data-Flow and Higher-Order Mutation are white-box testing techniques. To our knowledge, no work has been proposed to compare data flow and Higher-Order Mutation. This paper compares all def-uses Data-Flow and second-order mutation criteria. The comparison will support the testing decision-making, especially when choosing a suitable criterion. This compassion investigates the subsumption relation between these two criteria and evaluates the effectiveness of test data developed for each. To compare the two criteria, a set of test data satisfying each criterion is generated using genetic algorithms; the set is then used to explore whether one criterion subsumes the other criterion and assess the effectiveness of the test set that was developed for one methodology in terms of the other. The results showed that the mean mutation coverage ratio of the all du-pairs adequate test cover is 80.9%, and the mean data flow coverage ratio of the second-order mutant adequate test cover is 98.7%. Consequently, second-order mutation “ProbSubsumes” the all du-pairs data flow. The failure detection efficiency of the mutation (98%) is significantly better than the failure detection efficiency of data flow (86%). Consequently, second-order mutation testing is “ProbBetter” than all du-pairs data flow testing. In contrast, the size of the test suite of second-order mutation is more significant than the size of the test suite of all du-pairs.

Guided Intelligent Hyper-Heuristic Algorithm for Critical Software Application Testing Satisfying Multiple Coverage Criteria

This paper proposes a novel algorithm that combines symbolic execution and data flow testing to generate test cases satisfying multiple coverage criteria of critical software applications. The coverage criteria considered are data flow coverage as the primary criterion, software safety requirements, and equivalence partitioning as sub-criteria. black The characteristics of the subjects used for the study include high-precision floating-point computation and iterative programs. The work proposes an algorithm that aids the tester in automated test data generation, satisfying multiple coverage criteria for critical software. The algorithm adapts itself and selects different heuristics based on program characteristics. The algorithm has an intelligent agent as its decision support system to accomplish this adaptability. Intelligent agent uses the knowledge base to select different low-level heuristics based on the current state of the problem instance during each generation of genetic algorithm execution. The knowledge base mimics the expert’s decision in choosing the appropriate heuristics. black The algorithm outperforms by accomplishing 100% data flow coverage for all subjects. In contrast, the simple genetic algorithm, random testing and a hyper-heuristic algorithm could accomplish a maximum of 83%, 67% and 76.7%, respectively, for the subject program with high complexity. black The proposed algorithm covers other criteria, namely equivalence partition coverage and software safety requirements, with fewer iterations. black The results reveal that test cases generated by the proposed algorithm are also effective in fault detection, with 87.2% of mutants killed when compared to a maximum of 76.4% of mutants killed for the complex subject with test cases of other methods.

DatAFLow : Toward a Data-Flow-Guided Fuzzer

Coverage-guided greybox fuzzers rely on control-flow coverage feedback to explore a target program and uncover bugs. Compared to control-flow coverage, data-flow coverage offers a more fine-grained approximation of program behavior. Data-flow coverage captures behaviors not visible as control flow and should intuitively discover more (or different) bugs. Despite this advantage, fuzzers guided by data-flow coverage have received relatively little attention, appearing mainly in combination with heavyweight program analyses (e.g., taint analysis, symbolic execution). Unfortunately, these more accurate analyses incur a high run-time penalty, impeding fuzzer throughput. Lightweight data-flow alternatives to control-flow fuzzing remain unexplored. We present datAFLow , a greybox fuzzer guided by lightweight data-flow profiling. We also establish a framework for reasoning about data-flow coverage, allowing the computational cost of exploration to be balanced with precision. Using this framework, we extensively evaluate datAFLow across different precisions, comparing it against state-of-the-art fuzzers guided by control flow, taint analysis, and data flow. Our results suggest that the ubiquity of control-flow-guided fuzzers is well-founded. The high run-time costs of data-flow-guided fuzzing (~10 × higher than control-flow-guided fuzzing) significantly reduces fuzzer iteration rates, adversely affecting bug discovery and coverage expansion. Despite this, datAFLow uncovered bugs that state-of-the-art control-flow-guided fuzzers (notably, AFL++) failed to find. This was because data-flow coverage revealed states in the target not visible under control-flow coverage. Thus, we encourage the community to continue exploring lightweight data-flow profiling; specifically, to lower run-time costs and to combine this profiling with control-flow coverage to maximize bug-finding potential.

Automatic PSO Based Path Generation Technique for Data Flow Coverage

Automatic PSO Based Path Generation Technique for Data Flow Coverage

Integration testing based on indirect interaction for embedded system

<span>Embedded systems comprise several modules that exchange data by interacting among themselves. Exchanging wrong resource data among modules may lead to execution errors or anomalies. Interacting resources produce dependencies between two modules where any change of resources by one module affects the functionality of another module. Several investigations of the embedded system such as aerospace or automobile system show interaction faults between modules are one of the major cause of critical software failures. Therefore, interaction testing is an essential phase to reduce the interaction faults and minimize the risk. The direct and indirect interaction between modules generates interaction faults where indirect interaction is made underneath the interface in which data dependence relationship with resources may cause a different outcome. We investigate errors based on the indirect interaction between modules and introduce a new test criterion for finding errors detectable by existing approaches in unit level but not in integration level. In this paper, we propose a noble approach to generate an interaction model using indirect interaction pattern and design test criteria based on different interaction errors to generate test cases. Finally, we use fault injection and data flow coverage techniques to evaluate the feasibility and effectiveness of our approach</span>

A Hybrid Particle Swarm Optimization and Differential Evolution based Test Data Generation Algorithm for Data-Flow Coverage using Neighbourhood Search Strategy

Meta-heuristic search techniques, mainly Genetic Algorithm (GA), have been widely applied for automated test data generation according to a structural test adequacy criterion . However, it remains a challenging task for more robust adequacy criterion such as data-flow coverage of a program. Now, focus is on the use of other highly-adaptive meta-heuristic search techniques such as Particle Swarm Optimization (PSO) and Differential Evolution (DE). In this paper, a hybrid (adaptive PSO and DE) algorithm is proposed to generate test data for data-flow dependencies of a program with a neighbourhood search strategy to improve the search capability of the hybrid algorithm. The fitness function is based on the concepts of dominance relations and branch distance. The measures considered are mean number of generations and mean percentage coverage. The performance of the hybrid algorithm is compared with that of DE, PSO, GA, and random search. Over several experiments on a set of benchmark programs, it is shown that the hybrid algorithm performed significantly better than DE, PSO, GA and random search in data-flow test data generation with respect to the measures collected.

A Survey on Data-Flow Testing

Data-flow testing (DFT) is a family of testing strategies designed to verify the interactions between each program variable’s definition and its uses. Such a test objective of interest is referred to as a def-use pair . DFT selects test data with respect to various test adequacy criteria (i.e., data-flow coverage criteria ) to exercise each pair. The original conception of DFT was introduced by Herman in 1976. Since then, a number of studies have been conducted, both theoretically and empirically, to analyze DFT’s complexity and effectiveness. In the past four decades, DFT has been continuously concerned, and various approaches from different aspects are proposed to pursue automatic and efficient data-flow testing. This survey presents a detailed overview of data-flow testing, including challenges and approaches in enforcing and automating it: (1) it introduces the data-flow analysis techniques that are used to identify def-use pairs; (2) it classifies and discusses techniques for data-flow-based test data generation, such as search-based testing, random testing, collateral-coverage-based testing, symbolic-execution-based testing, and model-checking-based testing; (3) it discusses techniques for tracking data-flow coverage; (4) it presents several DFT applications, including software fault localization, web security testing, and specification consistency checking; and (5) it summarizes recent advances and discusses future research directions toward more practical data-flow testing.

An Adaptive PSO-based Approach for Data Flow Coverage of a Program

An Adaptive PSO-based Approach for Data Flow Coverage of a Program

Dataflow approach to testing Java programs supported with DFC

Code based (“white box”) approach to testing can be divided into two main types: control flow coverage and data flow coverage methods. Dataflow testing was introduced for structural programming languages and later adopted for object languages. Among many tools supporting code based testing of object programs, only JaBUTi and DFC (Data Flow Coverage) support dataflow testing of Java programs. DFC is a tool implemented at the Institute of Computer Science Warsaw University of Technology as an Eclipse plug-in. The objective of this paper is to present dataflow coverage testing of Java programs supported by DFC. DFC finds all definition-uses pairs in tested unit and provides also the definition-uses graph for methods. After the execution of test information which def-uses pairs were covered is shown. An example of data flow testing of Java program is also presented.

An approach and tool for measurement of state variable based data-flow test coverage for aspect-oriented programs

ContextData-flow testing approaches have been used for procedural and object-oriented programs, and shown to be effective in detecting faults. However, few such approaches have been evaluated for aspect-oriented programs. In such programs, data-flow interactions can occur between base classes and aspects, which can affect the behavior of both. Faults resulting from such interactions are hard to detect unless the interactions are specifically targeted during testing. ObjectiveThis paper presents an approach and tool implementation for measuring data-flow coverage based on state variables defined in base classes or aspects in AspectJ programs. The paper also reports on an empirical study that compares the cost and effectiveness of data-flow test criteria that are based on state variables with two control-flow criteria. MethodEffectiveness of the criteria was evaluated for various fault types. Cost-effectiveness of test suites that cover all state variable definition-use associations (DUAs) was evaluated for three coverage levels: 100%, 90%, and 80%. ResultsThe effort needed to obtain a test case that achieves data-flow coverage is higher than the effort needed to obtain a test case that covers a block or a branch in an advised class. Covering certain data flow associations requires more effort than for other types of data flow associations. The data-flow test criteria based on state variables of a base-class are in general more effective than control-flow criteria. ConclusionsOverall, it is cost-effective to obtain test suites at the 90% coverage level of data-flow criteria.

Discussig of Electricity Embedded Software Testing Technology

Focusing on the feature of embedded software in power industry and to ensure its high reliability, the authors of this article introduced a comprehensive and effective testing technique based on the theory and practice of embedded test, with two phases: static test and dynamic unit test. It focus on the coding rule checking, quality metrics, analysis, flow control and data flow analysis and coverage analysis techniques.

Model-based pairwise testing for feature interaction coverage in software product line engineering

Testing software product lines (SPLs) is very challenging due to a high degree of variability leading to an enormous number of possible products. The vast majority of today's testing approaches for SPLs validate products individually using different kinds of reuse techniques for testing. Because of their reusability and adaptability capabilities, model-based approaches are suitable to describe variability and are therefore frequently used for implementation and testing purposes of SPLs. Due to the enormous number of possible products, individual product testing becomes more and more infeasible. Pairwise testing offers one possibility to test a subset of all possible products. However, according to the best of our knowledge, there is no contribution discussing and rating this approach in the SPL context. In this contribution, we provide a mapping between feature models describing the common and variable parts of an SPL and a reusable test model in the form of statecharts. Thereby, we interrelate feature model-based coverage criteria and test model-based coverage criteria such as control and data flow coverage and are therefore able to discuss the potentials and limitations of pairwise testing. We pay particular attention to test requirements for feature interactions constituting a major challenge in SPL engineering. We give a concise definition of feature dependencies and feature interactions from a testing point of view, and we discuss adequacy criteria for SPL coverage under pairwise feature interaction testing and give a generalization to the T-wise case. The concept and implementation of our approach are evaluated by means of a case study from the automotive domain.

Diversity oriented test data generation using metaheuristic search techniques

We present a new test data generation technique which uses the concept of diversity of test sets as a basis for the diversity oriented test data generation – DOTG. Using DOTG we translate into an automatic test data generation technique the intuitive belief that increasing the variety, or diversity, of the test data used to test a program can lead to an improvement on the completeness, or quality, of the testing performed. We define the input domain perspective for diversity (DOTG-ID), which considers the distances among the test data in the program input domain to compute a diversity value for test sets. We describe metaheuristics which can be used to automate the generation of test sets for the DOTG-ID testing technique: simulated annealing; a genetic algorithm; and a proposed metaheuristic named simulated repulsion. The effectiveness of DOTG-ID was evaluated by using a Monte Carlo simulation, and also by applying the technique to test simple programs and measuring the data-flow coverage and mutation scores achieved. The standard random testing technique was used as a baseline for these evaluations. Results provide an understanding of the potential gains in terms of testing effectiveness of DOTG-ID over random testing and also reveal testing factors which can make DOTG-ID less effective.

Testing coupling relationships in object‐oriented programs

AbstractAs we move toward developing object‐oriented (OO) programs, the complexity traditionally found in functions and procedures is moving to the connections among components. Different faults occur when components are integrated to form higher‐level structures that aggregate the behavior and state. Consequently, we need to place more effort on testing the connections among components. Although OO technologies provide abstraction mechanisms for building components that can then be integrated to form applications, it also adds new compositional relations that can contain faults. This paper describes techniques for analyzing and testing the polymorphic relationships that occur in OO software. The techniques adapt traditional data flow coverage criteria to consider definitions and uses among state variables of classes, particularly in the presence of inheritance, dynamic binding, and polymorphic overriding of state variables and methods. The application of these techniques can result in an increased ability to find faults and to create an overall higher quality software. Copyright © 2010 John Wiley & Sons, Ltd.

A data-flow approach to test multi-agent ASMs

Abstract This paper illustrates the theoretical basis of an approach to apply data flow testing techniques to abstract state machines (ASMs). In particular, we focus on multi-agent ASMs extended with the seq construct for turbo ASMs. We explain why traditional data flow analysis can not simply be applied to ASMs: data flow coverage criteria are strictly based on the mapping between a program and its flow graph whereas in this context we are interested in tracing the flow of data between states in ASM runs as opposed to between nodes in a program’s flow graph. We revise the classical concepts in data flow analysis taking into account the specific, parallel nature of ASMs, and define them on two levels: the syntactic (rule) level, and the computational (run) level. In particular, we analyze the role played by different types of terms in ASMs and deal with the problem of terms that are monitored by a given agent but controlled by another one, terms that are shared between several agents, and derived terms. We also discuss what consequences the use of the turbo ASM construct seq has on our analysis and revise the approach accordingly. Finally, we specify a family of ad hoc data flow coverage criteria for this class of ASMs and introduce a model checking-based approach to generate automatically test cases satisfying a given set of coverage criteria from ASM models.

Data-flow testing of declarative programs

We propose a novel notion of data-flow coverage for testing declarative programs. Moreover, we extend an automatic test-case generator such that it can achieve data-flow coverage. The coverage information is obtained by instrumenting a program such that it collects coverage information during its execution. Finally, we show the benefits of data-flow based testing for a couple of example applications.

White and grey-box verification and validation approaches for safety- and security-critical software systems

In this article, the problem of assessing software trustworthiness is considered from a holistic perspective addressing both safety- and security-critical application domains. In particular, the importance of achieving high structural coverage during component and integration testing phases is stressed. In view of the immense effort required by manual testing activities, the present article suggests novel automatic test case generation techniques, capable of maximizing test coverage and minimizing test amount. The tools developed on the basis of these approaches were successfully applied to achieve high control flow, data flow and interface coverage by means of a low number of test cases.