Software Fault Injection Techniques Research Articles

Injecting software faults in Python applications

Software fault injection techniques have been largely used as means for evaluating the dependability of systems in presence of certain types of faults. Despite the large diversity of tools offering the possibility of emulating the presence of software faults, there is little practical support for emulating the presence of software faults in Python applications, which are increasingly being used to support business critical cloud services. In this paper, we present FIT4Python, a tool for injecting software faults in Python code and then use it, in a mutation testing campaign, to analyse the effectiveness of OpenStack’s test suite against new probable software faults. We begin by analysing the types of faults affecting Nova Compute, the core component of OpenStack. We use our tool to emulate the presence of new faults in Nova Compute API to understand how well OpenStack’s battery of unit, functional, and integration tests cover these new, but probable, situations. Results show clear limitations in the effectiveness of OpenStack developers’ test suites, with many cases of injected faults passing undetected through all three types of tests and that nearly half of the analysed problems could be detected with trivial changes or additions to the unit tests.

Emulating representative software vulnerabilities using field data

Security vulnerabilities are a concern in systems and software exposed via networked interfaces. Previous research has shown that only a minority of vulnerabilities can be emulated through software fault injection techniques. This paper aims to accurately emulate software security vulnerabilities. To this end, the paper provides a field-data study on the operators needed to emulate vulnerabilities in software written in the C programming language. A practical implementation is constructed and the feasibility of emulating software vulnerabilities is evaluated. The emulation operators were obtained by analyzing publicly available vulnerability databases for the Linux kernel, the Xen hypervisor, and the OpenSSH tool. The results show that a typical security vulnerability involves a single function and consists of combinations of up to three fault operator instances. The expected impact of this study is to allow practical emulation of security defects in large software projects, to support software quality and security assessment.

Computing reliability: On the differences between software testing and software fault injection techniques

System reliability has become a main concern during the computer-based system design process. It is one of the most important characteristics of the system quality. The continuous increase of the system complexity makes the reliability evaluation extremely costly. Therefore, there is need to develop new methods with less cost and effort. Furthermore, the system is vulnerable to both software and hardware faults. While the software faults are usually introduced by the programmer either at the design or the implementation stage of the software, the hardware faults are caused by physical phenomena affecting the hardware components, such as environmental perturbations, manufacturing defects, and aging-related phenomena. The software faults can only impact the software components. However, the hardware faults can propagate through the different system layers, and affect both the hardware and the software. This paper discusses the differences between the software testing and the software fault injections techniques used for reliability evaluation. We describe the mutation analysis as a method mainly used in software testing. Then, we detail the fault injection as a technique to evaluate the system reliability. Finally, we discuss how to use software mutation analysis in order to evaluate, at software level, the system reliability against hardware faults. The main advantage of this technique is its usability at early design stage of the system, when the instruction set architecture is not available. Experimental results run to evaluate faults occurring the memory show that the proposed approach significantly reduces the complexity of the system reliability evaluation in terms of time and cost.

Research on Gray-Box Testing Methods for Software Fault Injection

Because of various advantages of software, implemented system function is more and more through software. In order to ensure the system is running, the verification of fault-oriented processing function module need software fault injected techniques to support. The traditional software fault injection technique method mainly studies the injection achievement of various fault modes. In order to achieve better test coverage, it is necessary to analyze software fault related needs and structure. Therefore, this paper combined gray box testing methods and software fault injection technique, it is proposed gray-box testing technique oriented to software fault injection, and give fault injection process framework.

Emulation of Software Faults: A Field Data Study and a Practical Approach

The injection of faults has been widely used to evaluate fault tolerance mechanisms and to assess the impact of faults in computer systems. However, the injection of software faults is not as well understood as other classes of faults (e.g., hardware faults). In this paper, we analyze how software faults can be injected (emulated) in a source-code independent manner. We specifically address important emulation requirements such as fault representativeness and emulation accuracy. We start with the analysis of an extensive collection of real software faults. We observed that a large percentage of faults falls into well-defined classes and can be characterized in a very precise way, allowing accurate emulation of software faults through a small set of emulation operators. A new software fault injection technique (G-SWFIT) based on emulation operators derived from the field study is proposed. This technique consists of finding key programming structures at the machine code-level where high-level software faults can be emulated. The fault-emulation accuracy of this technique is shown. This work also includes a study on the key aspects that may impact the technique accuracy. The portability of the technique is also discussed and it is shown that a high degree of portability can be achieved