Parallel Debugging Research Articles

Software multi-fault localization via Chameleon clustering in parallel

Fault localization have proven to be useful in various software during the process of software testing and debugging. The presence of multiple faults simultaneously may deteriorate the effectiveness of existing fault-localization techniques. One good solution is to generate fault-focused clusters by the same bug into the same cluster in term of clustering analysis the failed execution traces. We first introduce improved Chameleon cluster analysis to locate multi-fault in parallel under a general framework. In order to make use of execution information, feature select strategy of the program execution trace is proposed. the failed program execution traces are divided into fault-focused clusters by Chameleon cluster analysis according to the likely faults that caused them, and each cluster may contain the same fault. Then, their suspiciousness is calculated after the clusters of failed program execution traces are merged with all successful program execution traces. Finally, multiple faults in the program are located simultaneously in parallel debugging mode by the suspicious ranking generated in each merged cluster. Experiments conducted on four real-world subjects shows that our approach can achieve better performance than the compared ones.

A comprehensive empirical investigation on failure clustering in parallel debugging

The clustering technique has attracted a lot of attention as a promising strategy for parallel debugging in multi-fault scenarios, this heuristic approach (i.e., failure indexing or fault isolation) enables developers to perform multiple debugging tasks simultaneously through dividing failed test cases into several disjoint groups. When using statement ranking representation to model failures for better clustering, several factors influence clustering effectiveness, including the risk evaluation formula (REF), the number of faults (NOF), the fault type (FT), and the number of successful test cases paired with one individual failed test case (NSP1F). In this paper, we present the first comprehensive empirical study of how these four factors influence clustering effectiveness. We conduct extensive controlled experiments on 1060 faulty versions of 228 simulated faults and 141 real faults, and the results reveal that: (1) GP19 is highly competitive across all REFs, (2) clustering effectiveness decreases as NOF increases, (3) higher clustering effectiveness is easier to achieve when a program contains only predicate faults, and (4) clustering effectiveness remains when the scale of NSP1F is reduced to 20%.

Spectrum-based Fault Localization Techniques Application on Multiple-Fault Programs: A Review

Software fault localization is one of the most tedious and costly activities in program debugging in the endeavor to identify faults locations in a software program. In this paper, the studies that used spectrum-based fault localization (SBFL) techniques that makes use of different multiple fault localization debugging methods such as one-bug-at-a-time (OBA) debugging, parallel debugging, and simultaneous debugging in localizing multiple faults are classified and critically analyzed in order to extensively discuss the current research trends, issues, and challenges in this field of study. The outcome strongly shows that there is a high utilization of OBA debugging method, poor fault isolation accuracy, and dominant use of artificial faults that limit the existing techniques applicability in the software industry.

Multiple fault localization of software programs: A systematic literature review

Multiple fault localization (MFL) is the act of identifying the locations of multiple faults (more than one fault) in a faulty software program. This is known to be more complicated, tedious, and costly in comparison to the traditional practice of presuming that a software contains a single fault. Due to the increasing interest in MFL by the research community, a broad spectrum of MFL debugging approaches and solutions have been proposed and developed. The aim of this study is to systematically review existing research on MFL in the software fault localization (SFL) domain. This study also aims to identify, categorize, and synthesize relevant studies in the research domain. Consequently, using an evidence-based systematic methodology , we identified 55 studies relevant to four research questions. The methodology provides a systematic selection and evaluation process with rigorous and repeatable evidence-based studies selection process. The result of the systematic review shows that research on MFL is gaining momentum with stable growth in the last 5 years. Three prominent MFL debugging approaches were identified, i.e. One-bug-at-a-time debugging approach (OBA), parallel debugging approach, and multiple-bug-at-a-time debugging approach (MBA), with OBA debugging approach being utilized the most. The study concludes with some identified research challenges and suggestions for future research. Although MFL is becoming of grave concern, existing solutions in the field are less mature. Studies utilizing real faults in their experiments are scarce. Concrete solutions to reduce MFL debugging time and cost by adopting an approach such as MBA debugging approach are also less, which require more attention from the research community.

Отладчик параллельных программ для ОС Linux

The paper presents a debugger for parallel programs in С/C++, or FORTRAN, which are executed in high-performance computers. The debugger’s program components and mechanism of their interaction are described. The graphic user’s interface capabilities are discussed and the profiling procedure using built-in profiling tools is described. The paper contains of the description of the new parallel debugger capabilities such as a communication treelike scheme of his components connection, and a non-interactive debugging mode, and the support of Nvidia’s graphic accelerators. Currently, the debugger provides launching of debug jobs in the systems of batch processing of jobs such as Open PBS / Torque, SLURM, and CSP JAM but it can be configured for other systems. The PD debugger allows to debug program processes and threads, manage breakpoints and watchpoints, logically divide program processes into subsets, manage them, change and view variables, and profile the debugged program using the free Google Performance Tools and mpiP. The PD debugger is written in the Java programming language, intended for debugging programs on Unix / Linux operating systems, and it uses free software components such as SwingX, JHDF5, Jzy3D, RSyntaxTextArea, and OpenGL.

A Parallel Debugger for Clusters on the Base of OS Linux

A Parallel Debugger for Clusters on the Base of OS Linux

Parallel debugging: An investigative study

AbstractIn the simultaneous localization of multiple software faults, a parallel debugging approach has consistently been utilized. The effectiveness of a parallel debugging approach is critically determined by the type of clustering algorithm and the distance metric used. However, clustering algorithms that group failed tests based on their execution profile similarity with distance metrics such as Euclidean distance, Jaccard distance, and Hamming distance are considered to be problematic and not appropriate. In this paper, we conducted an investigative study on the usefulness of the problematic parallel debugging approach that makes use of k‐means clustering algorithm (that groups failed tests based on their execution profile similarity) with Euclidian distance metric on three similarity coefficient‐based fault localization techniques in terms of localization effectiveness. Secondly, we compare the effectiveness of the problematic parallel debugging approach with one‐bug‐at‐a‐time debugging approach (OBA) and a state‐of‐the‐art parallel debugging approach named MSeer. The empirical evaluation is conducted on 540 multiple‐fault versions of eight medium‐sized to large‐sized subject programs with two, three, four, and five faulty versions. Our results suggest that clustering failed tests based on their execution profile similarity and the utilization of distance metrics such as Euclidean distance is indeed problematic and contributes to the reduction of effectiveness in localizing multiple faults.

An Analysis on the Negative Effect of Multiple-Faults for Spectrum-Based Fault Localization

This paper aims to investigate the negative effects of multiple-faults on spectrum-based fault localization (SBFL). Previously, researchers validated the fact that the occurrence of multiple-faults could have a significant negative impact on fault localization. However, a very little current research addresses the degree of these impacts through a systemic analysis. Furthermore, the fundamental causes underlying that negative impact have not been investigated and are not fully understood. We conducted experiments on fourteen real-life open source programs to explore and possibly solve these problems. Our results indicate that: 1) although multiple-faults generally do have a negative impact on fault localization, different fault localizations displayed various levels of robustness against that negative impact; 2) restoring pass/fail fault interactions only has a modest effect on that negative impact; 3) our investigation of twelve Fault Localization Interactions (FLI) shows that there is a dominant FLI-1 interaction in multiple-fault programs which should be responsible for that negative impact; 4) restoring FLI-1 can significantly improve the performance of both SBFL and parallel debugging techniques; and 5) furthermore, this paper practically validated the revised Kendall Tau distance as an efficient measure to help locate test cases, which have triggered FLI-1. Based on the revised Kendall Tau distance, a fast search algorithm has been suggested to locate FLI-1 test cases. It is expected that this paper can provide some insight into the fundamental causes of multiple-faults’ negative impact on fault localization and drive the development of more efficient fault localization techniques to improve the identification and handling of multiple-faults.

A Community-Based Fault Isolation Approach for Effective Simultaneous Localization of Faults

During program testing, software programs may be discovered to contain multiple faults. Multiple faults in a program may reduce the effectiveness of the existing fault localization techniques due to the complex relationship between faults and failures in the presence of multiple faults. In an ideal case, faults are isolated into fault-focused clusters, each targeting a single fault for developers to localize them simultaneously in parallel. However, the relationship between faults and failures is not easily identified and depends solely on the accuracy of clustering, such as existing clustering algorithms are not able to isolate failed tests to their causative faults effectively which hinder localization effectiveness. This paper proposes a new approach that makes use of a divisive network community clustering algorithm to isolate faults into separate fault-focused communities that target a single fault each. A community weighting and a selection mechanism that aids in prioritizing highly important fault-focused communities to the available developers to debug the faults simultaneously in parallel is also proposed. The approach is evaluated on eight subject programs ranging from medium-sized to large-sized programs (tcas, replace, gzip, sed, flex, grep, make, and ant). Overall, 540 multiple-fault versions of these programs were generated with 2-5 faulty versions. The experimental results have demonstrated that the proposed approach performs significantly better in terms of localization effectiveness in comparison with two other parallel debugging approaches for locating multiple faults in parallel.

Simultaneous Localization of Software Faults Based on Complex Network Theory

Effective debugging is necessary for producing high quality and reliable software. Fault localization plays a vital role in the debugging process. However, fault localization is the most tedious and expensive activity in program debugging, as such, effective fault localization techniques that can identify the exact location of faults is eminent. Despite various fault localization techniques proposed, their application in multiple-fault programs is limited. The presence of multiple faults in a program reduces the efficacy of the existing fault localization techniques to locate faults effectively due to fault interference. Moreover, most of these techniques are unable to localize faults simultaneously. This has led researchers to adopt alternative approaches, such as one-fault-at-a-time debugging and parallel debugging. In this paper, we propose a novel fault localization technique based on complex network theory (FLCN) to improve localization effectiveness in programs with single-fault and multiple-fault and to aid developers to localize multiple faults simultaneously in a single diagnosis rank list. The proposed technique ranks faulty statements based on their behavioral abnormalities and distance between faulty statements in both passed and failed test executions. Two graph-based centrality measures are adopted for fault diagnosis, namely, degree centrality and closeness centrality and a new ranking formula is proposed. FLCN is evaluated across 14 subjects with both single-fault and multiple-fault programs. Our experimental results show that FLCN is more effective at locating faults when compared with existing state-of-the-art fault localization techniques with improvement of fault localization effectiveness in both single-fault and multiple-fault programs.

Performance Prediction for Large-Scale Parallel Applications Using Representative Replay

Automatically predicting performance of parallel applications has been a long-standing goal in the area of high performance computing. However, accurate performance prediction is challenging, since the execution time of parallel applications is determined by several factors, such as sequential computation time, communication time and their complex interactions. Despite previous efforts, accurately estimating the sequential computation time in each process for large-scale parallel applications remains an open problem. In this paper, we propose a novel approach to acquiring accurate sequential computation time using a parallel debugging technique called deterministic replay. The main advantage of our approach is that we only need a single node of a target platform but the whole target platform does not need to be available. Therefore, with this approach we can simply measure the real sequential computation time on a target node for each process on by one. Moreover, we observe that there is great computation similarity in parallel applications, not only within each process but also among different processes. Based on this observation, we further propose representative replay that can significantly reduce replay overhead, because we only need to replay partial iterations for representative processes instead of all of them. Finally, we implement a complete performance prediction system, called Phantom , which combines the above computation-time acquisition approach and a trace-driven simulator. We validate our approach on both traditional HPC platforms and the latest Amazon EC2 cloud platform. On both types of platforms, prediction error of our approach is less than 7 percent on average up to 2,500 processes.

Providing Parallel Debugging for DASH Distributed Data Structures with GDB

Abstract The C + + DASH template library provides distributed data container for Partitioned Global Address Space (PGAS)-like programming. Because DASH is new and under development no debugger is capable to handle the parallel processes or access/modify container elements in a convenient way. This paper describes how the DASH library has to be extended to interrupt the start-up process to connect a debugger with all started processes and to enable the debugger for accessing and modifying DASH container elements. Furthermore, an GDB extension to output well formatted DASH container information is presented.

Accurate application progress analysis for large-scale parallel debugging

Debugging large-scale parallel applications is challenging. In most HPC applications, parallel tasks progress in a coordinated fashion, and thus a fault in one task can quickly propagate to other tasks, making it difficult to debug. Finding the least-progressed tasks can significantly reduce the effort to identify the task where the fault originated. However, existing approaches for detecting them suffer low accuracy and large overheads; either they use imprecise static analysis or are unable to infer progress dependence inside loops. We present a loop-aware progress-dependence analysis tool, Prodometer, which determines relative progress among parallel tasks via dynamic analysis. Our fault-injection experiments suggest that its accuracy and precision are over 90% for most cases and that it scales well up to 16,384 MPI tasks. Further, our case study shows that it significantly helped diagnosing a perplexing error in MPI, which only manifested at large scale.

A data‐centric framework for debugging highly parallel applications

SummaryContemporary parallel debuggers allow users to control more than one processing thread while supporting the same examination and visualisation operations of that of sequential debuggers. This approach restricts the use of parallel debuggers when it comes to large scale scientific applications run across hundreds of thousands compute cores. First, manually observing the runtime data to detect error becomes impractical because the data is too big. Second, performing expensive but useful debugging operations becomes infeasible as the computational codes become more complex, involving larger data structures, and as the machines become larger. This study explores the idea of a data‐centric debugging approach, which could be used to make parallel debuggers more powerful. It discusses the use of ad hoc debug‐time assertions that allow a user to reason about the state of a parallel computation. These assertions support the verification and validation of program state at runtime as a whole rather than focusing on that of only a single process state. Furthermore, the debugger's performance can be improved by exploiting the underlying parallel platform because the available compute cores can execute parallel debugging functions, while a program is idling at a breakpoint. We demonstrate the system with several case studies and evaluate the performance of the tool on a 20 000 cores Cray XE6. Copyright © 2013 John Wiley & Sons, Ltd.

Extending the Eclipse Parallel Tools Platform Debugger with Scalable Parallel Debugging Library

The Eclipse Parallel Tools Platform (PTP) is an open source Integrated Development Environment (IDE) aiding the development of Supercomputer applications. The PTP parallel debugger is used by a growing community of developers in scientific and engineering fields. This paper proposes a method of improving the communication infrastructure of the PTP debugger by taking advantage of a Scalable Parallel Debugging Library (SPDL). Unlike the present communication framework of PTP, the Scalable Debug Manager (SDM), SPDL provides a pluggable architecture that allows developers to select a communication protocol suitable for a targeted supercomputer. It currently supports a number of scalable protocols, including MRNet and SCI. The advanced features provided by these communication trees, like programmable filters and configurable topologies, allow developers to create more flexible solutions of efficient reduction and aggregation operations for parallel debugging. In particular, they allow parallel debuggers to handle the large amounts of back-end messages in peta-scale environments with better efficiency. The architecture of the PTP debugger is extended to support SPDL. The extended architecture combines the advantages of the PTP debugger at the front-end and SPDL at the back-end. It improves the scalability and performance of the PTP debugger. Consequently, it provides a flexible option of utilizing the PTP debugger with pluggable communication protocols to address the debugging challenges in peta-scale environments.

FlexSig

With the advent of chip multiprocessors, new techniques have been developed to make parallel programing easier and more reliable. New parallel programing paradigms and new methods of making the execution of programs more efficient and more reliable have been developed. Usually, these improvements require hardware support to avoid a system slowdown. Signatures based on Bloom filters are widely used as hardware support for parallel programing in chip multiprocessors. Signatures are used in Transactional Memory, thread-level speculation, parallel debugging, deterministic replay and other tools and applications. The main limitation of hardware signatures is the lack of flexibility: if signatures are designed with a given configuration, tailored to the requirements of a specific tool or application, it is likely that they do not fit well for other different requirements. In this paper a new hardware signature organization, called Flexible Signatures ( FlexSig ), is proposed. FlexSig can change dynamically the resources assigned to a given signature and the number of signatures in the system, by redistributing the available hardware resources according to the system requirements. This allows higher flexibility than with traditional fixed-resources signatures based on Bloom filters, while maintaining a low false positive rate. FlexSig has been evaluated by comparing it with signatures based on parallel Bloom filters, and we conclude that FlexSig outperforms (in terms of false positive rate) conventional parallel Bloom filters in most cases, due to its ability to use all the signature resources available.

Pinocchio

To support development tools like debuggers, runtime systems need to provide a meta-programming interface to alter their semantics and access internal data. Reflective capabilities are typically fixed by the Virtual Machine (VM). Unanticipated reflective features must either be simulated by complex program transformations, or they require the development of a specially tailored VM. We propose a novel approach to behavioral reflection that eliminates the barrier between applications and the VM by manipulating an explicit tower of first-class interpreters. Pinocchio is a proof-of-concept implementation of our approach which enables radical changes to the interpretation of programs by explicitly instantiating subclasses of the base interpreter. We illustrate the design of Pinocchio through non-trivial examples that extend runtime semantics to support debugging, parallel debugging, and back-in-time object-flow debugging. Although performance is not yet addressed, we also discuss numerous opportunities for optimization, which we believe will lead to a practical approach to behavioral reflection.

Developer autonomy in the FreeBSD open source project

Delegation of authority is the basic approach to coordination of work in FreeBSD, an open source software (OSS) project that develops and maintains an operating system. This essay combines a software engineering perspective with a knowledge creation perspective to discuss three central mechanisms in FreeBSD: (1) motivation, (2) quality control by frequent building, and (3) bugfixing by parallel debugging. It is argued that frequent building is the project’s key coordinating mechanism, and that build breakdowns play a constructive role in the processes of knowledge creation in FreeBSD.

Extending a traditional debugger to debug massively parallel applications

Beowulf systems, and other proprietary approaches, are placing systems with four or more CPUs in the hands of many researchers and commercial users. In the near future, systems with hundreds of CPUs will become commonly available, with some programmers dealing with tens of thousands of CPUs. The debugging methods used on these systems are a combination of the traditional methods used for debugging single processes and ad-hoc methods to help the user cope with the multitudes of processes. Programmers are usually familiar with a single-process debugger and would like to use it (with minimal user-visible extensions) to debug their distributed program. We present a set of modifications to a traditional debugger that makes it capable of debugging applications running on thousands of processes. Our parallel debugger is composed of individual fully functional debuggers connected with an n-nary aggregating network. This permits us to present to users the results from each debugger at the same time in an aggregated fashion. Users get a global view of the application and can easily see if a given parameter has a different value from either what they expect it to be or from the other processes. Users can then focus on the process sets of interest and investigate the problem. One challenge when debugging thousands of processes is to deal with the amount of output coming from all the debuggers. We present methods to aggregate the overwhelming amount of output from the debuggers into a more manageable subset, which is presented to the user without losing information. Experiments show that the debugger is scalable to thousands of processors. The startup mechanism, as well as users’ command response time scale well. The conclusions presented regarding the architecture and the new parallel debugger's scalability are not specific to the serial debugger we are using in our example implementation.

THE TEMPLATE PROGRAMMING OF PARALLEL ALGORITHMS

The parallel programming tools and packages are evolving rapidly. However the complexity of parallel thinking does not allow to implement many algorithms for the end user. In most cases only expert programmers risk to involve in parallel programming and program debugging. In this paper we extend the ideas from [3] of template programming for a certain class of problems which could be solved by using general master‐slave paradigm. The template is suitable for solution of the coarse grain and middle grain granularity problem set. Actually, it could be applied to solve any problem P, which is decomposable into a set of tasks P = U i N=0ti. The most effective application cases are obtained for the problems where all ti are independent. The template programming sets some requirements for the sequential version of the user program: The main program must comprise of several code blocks: data initialization, computation of one task ti and the processing of the result. The user has to define the data structures: initial data, one task data, the result data. These requirements do not require to rewrite the existing sequential code but to organize it into some logical parts. After these requirements (and naming conventions) are fulfilled, the parallel version of the code is obtained automatically by compiling and linking the code with the Master‐Slave Template library. In this paper we introduce the idea of the template programming and describe the layer structure of the Master‐Slave Template library. We show how the user has to adjust the sequential code to obtain a valid parallel version of the initial program. We also give examples of the prime number search problem and the Mandelbrot set calculation problem.