Fault Localization Process Research Articles

Test suite optimization under multi-objective constraints for software fault detection and localization: Hybrid optimization based model

Testing and debugging have been the most significant steps of software development since it is tricky for engineers to create error-free software. Software testing takes place after coding with the goal of finding flaws. If errors are found, debugging would be done to identify the source of the errors so that they may be fixed. Detecting as well as locating defects are thus two essential stages in the creation of software. We have created a unique approach with the following two working phases to generate a minimized test suite that is capable of both detecting and localizing faults. In the initial test suite minimization process, the cases were generated and minimized based on the objectives such as D-score and coverage by the utilization of the proposed Blue Monkey Customized Black Widow (BMCBW) algorithm. After this test suite minimization, the fault validation is done which includes the process of fault detection and localization. For this fault validation, we have utilized an improved Long Short-Term Memory (LSTM). At 90% of the learning rate the accuracy of the presented work is 0.97%, 2.20%, 2.52%, 0.97% and 2.81% is better than the other extant models like AOA, COOT, BES, BMO and BWO methods. The results obtained proved that our Blue Monkey Customized Black Widow Optimization-based fault detection and localization approach can provide superior outcomes.

Prediction of Photovoltaic (PV) Thermal Fault Location Using the Catadioptric Device

Photovoltaic (PV) farms are subject to thermal faults that can degrade and reduce module efficiency. The currently used method of detecting thermal fault is time-consuming and hard to locate the fault position, especially in large PV areas. It can result in additional flaws, such as igniting the PV farm. We provide a new approach for monitoring and detecting PV thermal faults through a catadioptric device (CD) that offers fast and continual detection. As an early development stage, in this study, we concentrate on building a mathematical model that can determine the thermal fault coordinate based on two object images from different CD positions. The experiment then verifies the model, and the parameter variation is performed to find the coordinate prediction characteristic. Also, a case study simulation on large PV system monitoring was performed to figure out the thermal fault localization process. The result shows that the mathematical model can be used to determine the coordinate position of the thermal fault with acceptable measurement error. The parameter <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">x_k</i> tends to affect the average measurement error of the coordinate prediction and the error gradient of each axes couple. Greater the <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">x_k</i> , smaller measurement and gradient error can be achieved. In addition, the case study simulation result shows that the thermal fault position can be predicted with the worst PE of less than 10%, with a MAPE, MAE, and RMSE in a reasonable value. Also, the sensitivity pattern can be used for CD condition monitoring.

Theoretical Analysis and Empirical Study on the Impact of Coincidental Correct Test Cases in Multiple Fault Localization

To improve the efficiency of the fault localization process, different automatic fault localization approaches have been proposed. Among these approaches, the spectrum-based fault localization (SBFL) approach has been widely used and studied due to its lightweight and high effectiveness. However, while the existence of coincidental correct (CC) test cases can influence the usefulness of SBFL in single-fault programs, their influence on multiple fault programs has not been thoroughly investigated. Therefore, in this article, we conduct a theoretical analysis and an empirical study to investigate the effect of CC test cases on multiple fault localization. The theoretical analysis is based on a suspiciousness calculation formula of SBFL, which divides CC test cases into three categories (specific, irrelevant, and unspecific) according to their association with a specific faulty statement. Following this analysis, we conduct an empirical study on two well-known open-source repositories (SIR and Defects4J), and the experimental results verify the correctness of our theoretical analysis. Specifically, reducing the number of specific CC test cases for a faulty statement can improve or maintain fault localization accuracy, while eliminating irrelevant CC test cases can have a negative effect. Finally, we design a CC test case identification solution based on the isolation-based multiple fault localization approach and demonstrate its effectiveness via a simulation experiment.

A Belief Network Reasoning Framework for Fault Localization in Communication Networks

A small fault in a large communication network may cause abrupt and large alarms, making the localization of the root cause of failure a difficult task. Traditionally, fault localization is carried out by an operator who uses alarms in alarm lists; however, fault localization process complexity needs to be addressed using more autonomous and intelligent approaches. Here, we present an overall framework that uses a message propagation mechanism of belief networks to address fault localization problems in communication networks. The proposed framework allows for knowledge storage, inference, and message transmission, and can identify a fault’s root cause in an event-driven manner to improve the automation of the fault localization process. Avoiding the computational complexity of traditional Bayesian networks, we perform fault inference in polytrees with a noisy OR-gate model (PTNORgate), which can reduce computational complexity. We also offer a solution to store parameters in a network parameter table, similar to a routing table in communication networks, with the aim of facilitating the development of the algorithm. Case studies and a performance evaluation show that the solution is suitable for fault localization in communication networks in terms of speed and reliability.

Analog and mixed-signal circuits simulation for product level EMMI analysis

The goal of this work is to propose a new flow that integrates the analog and mixed signal simulation of the circuits to replicate the EMMI signals. This supports the fault localization process. We explore the emission typologies of the transistors, focusing the attention on the DMOS structure. First experimental results show the benefits of this approach.

SMBFL: slice-based cost reduction of mutation-based fault localization

Fault localization is one of the most important and difficult tasks in the software debugging process. Therefore, several methods have been proposed to automate and improve this process. Mutation-based fault localization is one of the states of the art techniques that try to locate faults by executing different mutants of the faulty program. In addition to favorable results, it is along with a massive increase in mutation execution cost. In this paper, we propose a new mutation-based fault localization approach called SMBFL, that aim to reduce the execution cost by reducing the number of statements to be mutated. As fewer mutants execute with SMBFL, the whole process will become faster and the cost will decrease. SMBFL only examines the statements in the dynamic slice of the program under test. The statements that present in the dynamic slice have a direct effect on the execution of the program with the specified test case. In the SMBFL method, the suspiciousness score of program statements is measured based on the entropy of their mutants. The proposed formula, MuEn, determines the suspiciousness score based on the result of executing mutants of each statement of the program. SMBFL is evaluated during a series of tests. The results show a relative increase in the accuracy of fault localization, by an average of 14.2%, and a decrease in the execution time of the fault localization process, by an average of 24.3%. Finally, the MuEn formula applies the least execution overhead to the fault localization process.

Probabilistic Techno-Economic Optimization in Medium Voltage Distribution Networks with Fault Passage Indicators and Fault Locators

Fault passage indicators (FPIs) and fault locators (FLs) are employed in modern distribution networks in order to enhance the process of fault localization, thus resulting in reduction of interruption time and improving the reliability of power supply. In this paper, a novel probabilistic techno-economic optimization method is proposed for determining the number and positions of FPIs that lead to maximum reduction of interruptiontime and investment costs in medium voltage (MV) distribution networks with and without FLs. The proposed method is basedon a probabilistic non-sequential Monte Carlo simulation model of the real network, which is a proper compromise between complicated sequential simulation models and too simplified analytical models. The main goal of the method is to obtain maximum improvement of the network reliability indices while using the minimum number of FPIs. The method is tested on a combinedurban/rural MV distribution network in Bosnia and Herzegovina and results are thoroughly discussed.

Fault Propagation Reasoning and Diagnosis for Computer Networks Using Cyclic Temporal Constraint Network Model

Fault diagnosis, including fault detection and isolation, is a critical task for computer networks. Among the various techniques used for online system-level diagnosis, we are interested in the approach based on temporal information processing. The delays of the computer networks are inevitable, and the fault localization process has to take into account bounded delays or the temporal constraints. Temporal information is fundamental in model-based diagnosis. There can be cycles or loops in a computer network, but the fault reasoning methods for such cases are seldom considered in the literature. This paper provides an analytic model based on the cyclic temporal constraint network (CTCN), which aims at the fault diagnosis of cyclic computer networks using temporal information. The goal of the proposed framework is twofold: given the network structures and the predetermined candidate fault causes, the CTCN model corresponding the computer network under test is formulated; based on the CTCN model, given the alarms sequences with timestamps, the fault diagnosis process is executed to determine the most likely fault cause(s) with its/their time interval(s) of occurrence(s). The reasoning method is dependent on time point and time distance information, with which the fault motivators (i.e., actors) and fault responders (i.e., victims) can be identified. The calculation process consists of three steps: 1) establishment of the objective function; 2) determination of the fault propagation paths; and 3) determination of the expected states with a given fault hypothesis. Finally, the proposed method is demonstrated via an application study, and the effectiveness of our proposed method is verified.

A Survey on Software Fault Localization

Software fault localization, the act of identifying the locations of faults in a program, is widely recognized to be one of the most tedious, time consuming, and expensive – yet equally critical – activities in program debugging. Due to the increasing scale and complexity of software today, manually locating faults when failures occur is rapidly becoming infeasible, and consequently, there is a strong demand for techniques that can guide software developers to the locations of faults in a program with minimal human intervention. This demand in turn has fueled the proposal and development of a broad spectrum of fault localization techniques, each of which aims to streamline the fault localization process and make it more effective by attacking the problem in a unique way. In this article, we catalog and provide a comprehensive overview of such techniques and discuss key issues and concerns that are pertinent to software fault localization as a whole.

IPSETFUL: an iterative process of selecting test cases for effective fault localization by exploring concept lattice of program spectra

Fault localization is an important and challenging task during software testing. Among techniques studied in this field, program spectrum based fault localization is a promising approach. To perform spectrum based fault localization, a set of test oracles should be provided, and the effectiveness of fault localization depends highly on the quality of test oracles. Moreover, their effectiveness is usually affected when multiple simultaneous faults are present. Faced with multiple faults it is difficult for developers to determine when to stop the fault localization process. To address these issues, we propose an iterative fault localization process, i.e., an iterative process of selecting test cases for effective fault localization (IPSETFUL), to identify as many faults as possible in the program until the stopping criterion is satisfied. It is performed based on a concept lattice of program spectrum (CLPS) proposed in our previous work. Based on the labeling approach of CLPS, program statements are categorized as dangerous statements, safe statements, and sensitive statements. To identify the faults, developers need to check the dangerous statements. Meantime, developers need to select a set of test cases covering the dangerous or sensitive statements from the original test suite, and a new CLPS is generated for the next iteration. The same process is proceeded in the same way. This iterative process ends until there are no failing tests in the test suite and all statements on the CLPS become safe statements. We conduct an empirical study on several subject programs, and the results show that IPSETFUL can help identifymost of the faults in the program with the given test suite. Moreover, it can save much effort in inspecting unfaulty program statements compared with the existing spectrum based fault localization techniques and the relevant state of the art technique.

A dynamic code coverage approach to maximize fault localization efficiency

Spectrum-based fault localization is amongst the most effective techniques for automatic fault localization. However, abstractions of program execution traces, one of the required inputs for this technique, require instrumentation of the software under test at a statement level of granularity in order to compute a list of potential faulty statements. This introduces a considerable overhead in the fault localization process, which can even become prohibitive in, e.g., resource constrained environments. To counter this problem, we propose a new approach, coined dynamic code coverage (DCC), aimed at reducing this instrumentation overhead. This technique, by means of using coarser instrumentation, starts by analyzing coverage traces for large components of the system under test. It then progressively increases the instrumentation detail for faulty components, until the statement level of detail is reached. To assess the validity of our proposed approach, an empirical evaluation was performed, injecting faults in six real-world software projects. The empirical evaluation demonstrates that the dynamic code coverage approach reduces the execution overhead that exists in spectrum-based fault localization, and even presents a more concise potential fault ranking to the user. We have observed execution time reductions of 27% on average and diagnostic report size reductions of 77% on average.

Metallaxis‐FL: mutation‐based fault localization

SummaryFault localization methods seek to identify faulty program statements based on the information provided by the failing and passing test executions. Spectrum‐based methods are among the most popular ones and assist programmers by assigning suspiciousness values on program statements according to their probability of being faulty. This paper proposes Metallaxis, a fault localization approach based on mutation analysis. The innovative part of Metallaxis is that it uses mutants and links them with the faulty program places. Thus, mutants that are killed mostly by failing tests provide a good indication about the location of a fault. Experimentation using Metallaxis suggests that it is significantly more effective than statement‐based approaches. This is true even in the case where mutation cost‐reduction techniques, such as mutant sampling, are facilitated. Additionally, results from a controlled experiment show that the use of mutation as a testing technique provides benefits to the fault localization process. Therefore, fault localization is significantly improved by using mutation‐based tests instead of block‐based or branch‐based test suites. Finally, evidence in support of the methods’ scalability is also given. Copyright © 2013 John Wiley &amp; Sons, Ltd.

Mutation-oriented test data augmentation for GUI software fault localization

ContextFault localization lies at the heart of program debugging and often proceeds by contrasting the statistics of program constructs executed by passing and failing test cases. A vital issue here is how to obtain these “suitable” test cases. Techniques presented in the literature mostly assume the existence of a large test suite a priori. However, developers often encounter situations where a failure occurs, but where no or no appropriate test suite is available for use to localize the fault. ObjectiveThis paper aims to alleviate this key limitation of traditional fault localization techniques for GUI software particularly, namely, it aims at enabling cost-effective fault localization process for GUI software in the described scenario. MethodTo address this scenario, we propose a mutation-oriented test data augmentation technique, which actually is directed by the “similarity” criterion in GUI software’s test case context towards the generation of test suite with excellent fault localization capabilities. More specifically, the technique mainly uses four proposed novel mutation operators to iteratively mutate some failing GUI test cases’ event sequences to derive new test cases potentially useful to localize the specific encountered fault. We then compare the fault localization performance of the test suite generated using this technique with that of an original provided large event-pair adequate test suite on some GUI applications. ResultsThe results indicate that the proposed technique is capable of generating a test suite that has comparable, if not better, fault localization effectiveness to the event-pair adequate test suite, but it is much smaller and it is generated immediately once a failure is encountered by developers. ConclusionIt is concluded that the proposed technique can truly enable quick-start cost-effective fault localization process under the investigated all-too-common scenario, greatly alleviating one key limitation of traditional fault localization techniques and prompting the test–diagnose–repair cycle.

A Bayesian Network-Based Approach to the Critical Infrastructure Interdependencies Analysis

This paper presents a novel approach to the CI interdependencies analysis, based on the DBN formalism. An original modeling procedure is illustrated, which divides the DBN in three distinct levels: an atomic events level, a propagation level, and a services level. The first level models the adverse events that may impact on the analyzed CIs, the second one properly captures interdependencies among CIs' services and devices, and the last one allows to monitor the state of provided services. The resulting DBN permits to perform three kinds of analysis: a reliability study, which allows to find structural weaknesses of interconnected CIs, an adverse events propagation study, which highlights the role interdependency plays in the propagation of adverse events, and a failure prediction analysis, that can serve as an useful guide to the fault localization process (failures may have many different explanations due to interdependency). A specific case study provided by Israel Electric Corporation is considered, and explicative simulations are presented and discussed in detail.

A family of code coverage-based heuristics for effective fault localization

Locating faults in a program can be very time-consuming and arduous, and therefore, there is an increased demand for automated techniques that can assist in the fault localization process. In this paper a code coverage-based method with a family of heuristics is proposed in order to prioritize suspicious code according to its likelihood of containing program bugs. Highly suspicious code (i.e., code that is more likely to contain a bug) should be examined before code that is relatively less suspicious; and in this manner programmers can identify and repair faulty code more efficiently and effectively. We also address two important issues: first, how can each additional failed test case aid in locating program faults; and second, how can each additional successful test case help in locating program faults. We propose that with respect to a piece of code, the contribution of the first failed test case that executes it in computing its likelihood of containing a bug is larger than or equal to that of the second failed test case that executes it, which in turn is larger than or equal to that of the third failed test case that executes it, and so on. This principle is also applied to the contribution provided by successful test cases that execute the piece of code. A tool, χDebug, was implemented to automate the computation of the suspiciousness of the code and the subsequent prioritization of suspicious code for locating program faults. To validate our method case studies were performed on six sets of programs: Siemens suite, Unix suite, space, grep, gzip, and make. Data collected from the studies are supportive of the above claim and also suggest Heuristics III(a), (b) and (c) of our method can effectively reduce the effort spent on fault localization.

Interactive Fault Localization Using Test Information

Debugging is a time-consuming task in software development. Although various automated approches have been proposed, they are not effective enough. On the other hand, in manual debugging, developers have difficulty in choosing breakpoints. To address these problems and help developers locate faults effectively, we propose an interactive fault-localization framework, combining the benefits of automated approaches and manual debugging. Before, the fault is found, this framework continuously recommends checking points based on statements suspicions, which are calculated according to the execution information of test cases and the feedback information from the developer at earlier checking points. Then we propose a naive approach, which is an initial implementation of this framework. However, with this naive approach or manual debugging, developers' wrong estimation of whether the faulty statement is executed before the checking point (breakpoint) may make the debugging process fail. So we propose another robust approach based on this framework, handling cases where developers make mistakes during the fault-localization process. We performed two experimental studies and the results show that the two interactive approaches are quite effective compared with existing fault-localization approaches. Moreover, the robust approach can help developers find faults when they make wrong estimation at some checking points.

Critical slicing for software fault localization

Developing effective debugging strategies to guarantee the reliability of software is important. By analyzing the debugging process used by experienced programmers, we have found that four distinct tasks are consistently performed: (1) determining statements involved in program failures, (2) selecting suspicious statements that might contain faults, (3) making hypotheses about suspicious faults (variables and locations), and (4) restoring program state to a specific statement for verification. This research focuses support for the second task, reducing the search domain for faults, which we refer to as fault localization. We explored a new approach to enhancing the process of fault localization based on dynamic program slicing and mutation-based testing. In this new approach, we have developed the technique of Critical Slicing to enable debuggers to highlight suspicious statements and thus to confine the search domain to a small region. The Critical Slicing technique is partly based on "statement deletion" mutant operator of the mutation-based testing methodology. We have explored properties of Critical Slicing, such as the relationship among Critical Slicing, Dynamic Program Slicing, and Executable Static Program Slicing; the cost to construct critical slices; and the effectiveness of Critical Slicing. Results of experiments support our conjecture as to the effectiveness and feasibility of using Critical Slicing for fault localization.This paper explains our technique and summarizes some of our findings. From these, we conclude that a debugger equipped with our proposed fault localization method can reduce human interaction time significantly and aid in the debugging of complex software.

Model-based reasoning for electron-beam debugging of VLSI circuits

This article deals with prototype validation of VLSI circuits. Circuits are observed using electron-beam probing used in voltage contrast mode, in such a way that grey level images are obtained and processed in order to determine potential values of connexions. These values are compared against reference values, issued from fault-free simulation of the device under test. The list of discrepancies resulting from comparison constitutes input for a fault localization process done using a knowledge-based system. In this article, the reasons for the choice of such an approach are explained, the approach itself is described, as well as its implementation and the obtained results.

Error retracing and fault localization techniques within Simueva, a system for evaluation of simulation results

This article describes further results of our research for the SIMUEVA system [1]. This system assists the user in evaluation of results from simulations in the design cycle of integrated circuits. Development steps reported so far [2] include extraction and comparison tools. The techniques explained here, accomplish referring of errors found during comparison to parts of the simulated structure. These techniques include the collection of erroneous and good simulation results to value complexes, retracing of single events, non-events, values, and value complexes, and identification of parts most likely being causes of the errors. For determination of appropriate retracing steps, we define classifications for errors and for elements of the simulated structure. In order to enable the system for straightforward retracing, connections between structure elements and the retracing process, and models for structure elements are discussed. The connections between some typical faults and a number of fault hypotheses are described, and some basics of the localization process are outlined. At the end of this paper, an example of a fault localization process is given.

Knowledge-based fault localization in debugging

Fault localization in program debugging is the process of identifying program statements which cause anomalous behavior. We have developed a prototype, knowledge-based model of the fault localization process. Novel features of the model include the integration of prototypic and causal reasoning about faults and a recognition-based mechanism for program abstraction. An explicit division of knowledge from the applications, programming, and language domains facilitate model tuning within as well as across applications domains. We describe model structure and performance for a class of faults associated with master file update programs.