Effectiveness Of Mutation Testing Research Articles

Applying parallelism to a bisimulation algorithm to improve efficiency in software testing of time-critical systems

Time-Critical Systems (TCS) play a crucial role in environments where strict timing constraints are essential to ensure reliability and correctness. Mutation Testing (MT) is considered a valuable strategy for quality assurance of TCS, but it suffers from the equivalent mutant problem, which is known to increase computational cost and reduce confidence in MT. To address this problem, a strong timed bisimulation equivalence (STBE) algorithm can be used when TCS are modeled as Timed Automata (TA). STBE is computationally expensive and can benefit from parallelism. We survey available STBE implementations, identify opportunities to apply parallelism, build an extension that takes advantage of them, and test its effects. The resulting solution is a Java program that receives multiple TAs expressed in UPPAAL format and determines which TAs are equivalent using an STBE implementation such as TimBrCheck or MUTES and process-based parallelism. Compared to existing solutions, our tests show that our proposal is more efficient, reducing the runtimes of STBE by more than half. This could improve the reach, reliability, and effectiveness of MT for TCS.

Equivalent Mutants Detection Based on Weighted Software Behavior Graph

The equivalent mutants problem is one of the crucial problems in mutation testing. In consequence of its existence, the effectiveness of mutation testing is underestimated. In addition, it will produce a certain amount of useless overhead. Equivalent mutants cannot be detected by any test input. The existing works mostly focus on static analysis to detect, or avoid generating, the equivalent mutants. The essence of these methods is to use prior knowledge to establish some rules of program equivalence. However, (1) it needs a lot of professional labor to sort out the equivalence rules, and (2) only a small part of the rules can be determined in advance, because of the diversity of mutation operators and mutation targets. Consequently, the best result reported so far is 50% of the equivalent mutants can be detected. Since it is generally believed that manual judgment of program equivalence is the most reliable, this paper proposes a novel method to automatically detect equivalent mutants by tracing program behavior like the professionals. The weighted software behavior graph is utilized in the detection of equivalent mutants for the first time. This method can not only figure out different execution paths, but also be sensitive to execution frequency. By comparing the weighted software behavior graphs of an alive mutant and its original program, we are able to examine more precisely whether the alive mutant is the same as the original program, in terms of the state of infection and/or the propagation. Evaluation results on an open dataset of manually evaluated equivalent mutants show that our approach can detect 77.5% of all the equivalent mutants, which is much higher than the existing static methods.

Reduction of Faults in Software Testing by Fault Domination

Although mutation testing is one of the practical ways of enhancing test effectiveness in software testing, it could be sometimes infeasible in practical work for a large scale software so that the mutation testing becomes time-consuming and even in prohibited time. Therefore, the number of faults assumed to exist in the software under test should be reduced so as to be able to confine the time complexity of test within a reasonable period of time. This paper utilizes the concept of fault dominance and equivalence, which has long been employed in hardware testing, for revealing a novel way of reducing the number of faults assumed to hide in software systems. Once the number of faults assumed in software is decreased sharply, the effectiveness of mutation testing will be greatly enhanced and become a feasible way of software testing. Examples and experimental results are presented to illustrate the effectiveness and the helpfulness of the technology proposed in the paper.