Effectiveness Of Software Testing Research Articles

Optimal test case generation for boundary value analysis

Boundary value analysis (BVA) is a common technique in software testing that uses input values that lie at the boundaries where significant changes in behavior are expected. This approach is widely recognized and used as a natural and effective strategy for testing software. Test coverage is one of the criteria to measure how much the software execution paths are covered by the set of test cases. This paper focuses on evaluating test coverage with respect to BVA by defining a metric called boundary coverage distance (BCD). The BCD metric measures the extent to which a test set covers the boundaries. In addition, based on BCD, we consider the optimal test input generation to minimize BCD under the random testing scheme. We propose three algorithms, each representing a different test input generation strategy, and evaluate their fault detection capabilities through experimental validation. The results indicate that the BCD-based approach has the potential to generate boundary values and improve the effectiveness of software testing.

Software Bug Prediction Using Machine Learning Approach

Abstract: Software defect prophecy work focuses on the number of faults remaining in a software system. A prophecy of the number of remaining scars in an audited artefact can be used for decision timber. An accurate prophecy of the number of scars in a software product during system testing contributes not only to the operation of the system testing process but also to the estimation of the product’s demanded conservation. amiss software modules beget software failures, increase development and conservation costs, and drop customer satisfaction. It strives to meliorate software quality and testing effectiveness by constructing predictive models from law attributes to enable a timely identification of fault-prone modules. In this design, we will explore machine knowledge ways for software defect prophecy . This helps the formulators to descry software scars and correct them. Unsupervised ways may be used for defect prophecy in software modules, more so in those cases where defect labels are not available.

Predicting software defects with swarm-intelligence-based machine learning algorithm for improved process quality

The quality and effectiveness of software systems may be significantly impacted by Software Defects (SD). Therefore, enhancing process quality is essential for controlling and minimizing the incidence of faults. Implementing reliable Software Development (SDe) processes and best practices is one way to do this information. SD, commonly referred to as software bugs or software mistakes, are defects or errors that occur in computer programs and cause them to act up or create unintended outcomes. These vulnerabilities may appear for several causes, including programming mistakes, poor design choices, or issues with the SDe cycle. The prediction of software problems based on Machine Learning (ML) using an Enhanced Artificial Neural Network (E-ANN) is implemented in this study to increase software quality and testing effectiveness. Particle swarm optimization and grey wolf optimization, these two algorithms named grey wolf swarm optimization algorithm (GWSA), are combined to recognize the corresponding compensation of the methods following the respective benefits and drawbacks. The hybrid algorithm-based model to the conventional hyperparameter optimization strategy and a single swarm intelligence algorithm's investigation of investigational consequences from six data sets shows that the hybrid algorithm-based model has high and enhanced indicators. Processed by the autoencoder, the model's performance has also improved.

Squeeziness for non-deterministic systems

Failed Error Propagation greatly reduces the effectiveness of Software Testing by masking faults present in the code. This situation happens when the System Under Test executes a faulty statement, the state of the system is affected by this fault, but the expected output is observed. Therefore, it is a must to assess its impact in the testing process. Squeeziness has been shown to be a useful measure to assess the likelihood of fault masking in deterministic systems. The main goal of this paper is to define a new Squeeziness notion that can be used in a scenario where we may have non-deterministic behaviours. The new notion should be a conservative extension of the previous one. In addition, it would be necessary to evaluate whether the new notion appropriately estimates the likelihood that a component of a system introduces Failed Error Propagation. We defined our black-box scenario where non-deterministic behaviours might appear. Next, we presented a new Squeeziness notion that can be used in this scenario. Finally, we carried out different experiments to evaluate the usefulness of our proposal as an appropriate estimation of the likelihood of Failed Error Propagation. We found a high correlation between our new Squeeziness notion and the likelihood of Failed Error Propagation in non-deterministic systems. We also found that the extra computation time with respect to the deterministic version of Squeeziness was negligible. Our new Squeeziness notion is a good measure to estimate the likelihood of Failed Error Propagation being introduced by a component of a system (potentially) showing non-deterministic behaviours. Since it is a conservative extension of the original notion and the extra computation time needed to compute it, with respect to the time needed to compute the former notion, is very small, we conclude that the new notion can be safely used to assess the likelihood of fault masking in deterministic systems.

Testing coverage‐based software reliability growth model considering uncertainty of operating environment

AbstractSoftware reliability is one of the standard critical inherent characteristics of software systems. The testing coverage function (TCF) is a significant parameter for identifying the completeness and effectiveness of software testing. It is defined as the proportion of the code that has been tested up to time t. To capture the dynamic behavior of the number of faults detected over a period of time, several distributions, namely S‐shaped, inflection S‐shaped, logistic, log‐logistic, Weibull, Rayleigh, Erlang, and logarithmic exponentiated, have been used as TCF in literature. However, these distributions are not sufficient to describe TCF's practical behavior due to complexity and vagueness in the collected data. This study proposes two software reliability growth models (SRGMs), which incorporate the generalized inflection S‐shaped (GISS) distribution as TCF. The models have been developed in perfect and imperfect debugging environments while considering fault removal efficiency, error generation, and uncertainty in the operating environment. To analyze the effectiveness, the proposed models are then tested with six failure data sets. The choice of GISS distribution as a TCF improves the software reliability estimation in comparison with the existing models in the literature. Finally, single and multiple parameters sensitivity analysis also has been done and based on it, the critical parameters have been detected. The proposed models may be helpful for the system analyst to predict various parameters about some software systems.

Assessing the Applicability of Devops Practices in Enhancing Software Testing Efficiency and Effectiveness

This study looks into how DevOps approaches are being adopted by Indian software development companies with the goal of evaluating the effects on quality, agility, and responsiveness to business objectives. Utilizing a deductive methodology, information was gathered from Indian IT experts with DevOps expertise. According to the results, DevOps is thought to improve quality, responsiveness, and agility. Sharing, culture, automation, and measurement were found to be important components. To improve the efficacy and efficiency of software testing procedures, recommendations call for the use of DevOps in Indian software enterprises.

On introducing automatic test case generation in practice: A success story and lessons learned

The level and quality of automation dramatically affects software testing activities, determines costs and effectiveness of the testing process, and largely impacts on the quality of the final product. While costs and benefits of automating many testing activities in industrial practice (including managing the quality process, executing large test suites, and managing regression test suites) are well understood and documented, the benefits and obstacles of automatically generating system test suites in industrial practice are not well reported yet, despite the recent progresses of automated test case generation tools. Proprietary tools for automatically generating test cases are becoming common practice in large software organizations, and commercial tools are becoming available for some application domains and testing levels. However, generating system test cases in small and medium-size software companies is still largely a manual, inefficient and ad-hoc activity.This paper reports our experience in introducing techniques for automatically generating system test suites in a medium-size company. We describe the technical and organizational obstacles that we faced when introducing automatic test case generation in the development process of the company, and present the solutions that we successfully experienced in that context. In particular, the paper discusses the problems of automating the generation of test cases by referring to a customized ERP application that the medium-size company developed for a third party multinational company, and presents ABT2.0, the test case generator that we developed by tailoring ABT, a research state-of-the-art GUI test generator, to their industrial environment. This paper presents the new features of ABT2.0, and discusses how these new features address the issues that we faced.

Case Study Investigation of the Fault Detection and Error Locating Effects of Architecture-based Software Testing

For software testing, it is well known that the architecture of a software system can be utilized to enhance testability, fault detection and error locating. However, how much and what effects architecture-based software testing has on software testing have been rarely studied. Thus, this paper undertakes case study investigation of the effects of architecture-based software testing specifically with respect to fault detection and error locating. Through comparing the outcomes with the conventional testing approaches that are not based on test architectures, we confirm the effectiveness of architecture-based software testing with respect to fault detection and error locating. The case studies show that using test architecture can improve fault detection rate by 44.1%–88.5% and reduce error locating time by 3%–65.2%, compared to the conventional testing that does not rely on test architecture. With regard to error locating, the scope of relevant components or statements was narrowed by leveraging test architecture for approximately 77% of the detected faults. We also show that architecture-based testing could provide a means of defining an exact oracle or oracles with range values. This study shows by way of case studies the extent to which architecture-based software testing can facilitate detecting certain types of faults and locating the errors that cause such faults. In addition, we discuss the contributing factors of architecture-based software testing which enable such enhancement in fault detection and error locating.

Dynamic Random Testing: Technique and Experimental Evaluation

A particularly good software testing strategy is to achieve the underlying testing goal while solving the problems of tradeoffs between testing effectiveness and efficiency. To improve the fault detection effectiveness of software testing, the principle of feedback control theory was adopted, which motivated the proposal of dynamic random testing (DRT). The main idea behind DRT is using the testing results to guide the test case selection to increase the selection probabilities of the subdomains with higher fault detection rates. Previous works show that DRT strategy can achieve better effectiveness than random testing strategy and random partition testing strategy, and has significantly lower computational costs than adaptive testing strategy. However, the essential factors that affect the performance of DRT, i.e., adjusting parameters, initial profile, and test case classification have not been thoroughly investigated. Besides, some experimental assumptions are inconsistent with real scenarios. Therefore, this paper gives a series of investigations on DRT with a set of practical subject programs. More specifically, the effectiveness and efficiency of DRT are presented, and the extended experiments on DRT with relevant factors are conducted. The results indicate that the effectiveness of DRT is robust to different initial profiles and affected noticeably by the adjusting parameter settings and test case classification methods.

Efficient Software Testing Technique based on Hybrid Database Approach

In the field of computer science, software testing is referred as a critical process which is executed in order to assess and analyze the performance and risks existing in software applications. There is an emphasis on integrating specific approaches to carry out testing activities in an effective mode; the efficient strategy being explored recently is adopting hybrid database approach. For this purpose, a hybrid algorithm will be proposed to ensure the functionality and outcomes of testing procedure. The technical processes and its impact on current methodology would help to evaluate its effectiveness in software testing through which specific conclusions could be drawn. The findings of the research will elaborate effectiveness of the proposed algorithm that would be used in software testing. It would be suggested that new technology is easier and simple to assess and analyze the reliability of the software. Basically, hybrid database approach comprises of traditional and modern techniques that are deployed in order to achieve testing outcomes. It is explored from various testing methods that challenges have been identified related to focusing on traditional techniques due to which hybrid approach is now being developed in most of the areas. In the light of addressing these concepts, the paper aims to investigate the complexity and efficiency of hybrid database approach in software testing, as well as its scope in the IT industry.

Test Path Prioritization from UML Activity Diagram Using a Hybridized Approach

Software testing is regarded as a pivotal approach to realize a high reliable product. To check for the correctness of results, we require appropriate test cases. UML models are largely used to depict the specifications for software development. Test cases are created independently and based on the sequence of occurrence in the diagrams; they lead to corresponding test paths in the program. In this paper, we have analyzed an activity diagram, consisting of concurrent activities, for generating test paths. The obtained test paths are therefore required to be ranked. We have demonstrated that it is conceivable to apply Genetic Algorithm procedures alongside Ant Colony Optimization technique for not only finding the most critical path but also prioritizing the other paths too for enhancing the effectiveness of software testing.

Structural test data generation using a memetic ant colony optimization based on evolution strategies

Abstract Test data generation is one of the key activities that has a significant impact on the efficiency and effectiveness of software testing. Since manual test data generation is quite inefficient and even impractical, automated test data generation has been realized to produce an appropriate subset of input data to carry out effective software testing in reasonable times. This paper presents a memetic ant colony optimization (ACO) algorithm for structural test data generation. The proposed approach incorporates (1+1)-evolution strategies (ES) to improve the search functionality of ants in local moves and enhance search exploitation. Moreover, we have introduced a novel definition of the pheromone functionality in the way that it discourages ants from choosing mostly covered paths of the program to reinforce search exploration. Given that branch coverage is considered as the coverage criterion, two fitness functions are used accordingly for our proposed algorithm. The first fitness function is a Boolean function which is particularly defined to maximize branch coverage. It outputs one if a given solution is successful in traversing at least a yet uncovered branch; otherwise, it returns zero. The second fitness function is formulated according to the complexity of branches covered. The value of the second fitness function is not taken into account for solutions whose Boolean function value equals one. For these solutions, the decision-making process of ants is merely carried out based on the first fitness function. The experimental results indicate the superiority of our memetic ACO algorithm relative to existing test data generation techniques in terms of both branch coverage and convergence speed.

Software metrics thresholds calculation techniques to predict fault-proneness: An empirical comparison

Context: Nowadays, fault-proneness prediction is an important field of software engineering. It can be used by developers and testers to prioritize tests. This would allow a better allocation of resources, reducing testing time and costs, and improving the effectiveness of software testing. Non-supervised fault-proneness prediction models, especially thresholds-based models, can easily be automated and give valuable insights to developers and testers on the classification performed.Objective: In this paper, we investigated three thresholds calculation techniques that can be used for fault-proneness prediction: ROC Curves, VARL (Value of an Acceptable Risk Level) and Alves rankings. We compared the performance of these techniques with the performance of four machine learning and two clustering based models.Method: Threshold values were calculated on a total of twelve different public datasets: eleven from the PROMISE Repository and another based on the Eclipse project. Thresholds-based models were then constructed using each thresholds calculation technique investigated. For comparison, results were also computed for supervised machine learning and clustering based models. Inter-dataset experimentation between different systems and versions of a same system was performed.Results: Results show that ROC Curves is the best performing method among the three thresholds calculation methods investigated, closely followed by Alves Rankings. VARL method didn’t give valuable results for most of the datasets investigated and was easily outperformed by the two other methods. Results also show that thresholds-based models using ROC Curves outperformed machine learning and clustering based models.Conclusion: The best of the three thresholds calculation techniques for fault-proneness prediction is ROC Curves, but Alves Rankings is a good choice too. In fact, the advantage of Alves Rankings over ROC Curves technique is that it is completely unsupervised and can therefore give pertinent threshold values when fault data is not available.

Adapting automated test generation to GUI testing of industry applications

ContextAutomated test generation promises to improve the effectiveness of software testing and to reduce the involved manual effort. While automated test generation has been successfully applied for code-level API testing, it has not found widespread adoption in practice for testing of graphical user interfaces. Tools for test generation do not support GUI testing out-of-the-box but require dedicated extensions. ObjectiveThis paper explores the applicability of automated test generation for testing GUIs of industry applications. We propose a test adapter approach to bridge the gap between automated test generation tools and industry applications. MethodA multiple case study was conducted in which automated test generation with test adapters has been applied at the unit, integration, and system test level in three industry projects from two different companies. ResultsAutomated test generation via test adapters could be applied at all test levels. It has led to an increase of coverage as well as the detection of new defects that were not found by preceding testing activities in the projects. While test adapters can easily be implemented at the unit test level, their complexity and the corresponding effort for providing adapter implementations rises at higher test levels. ConclusionTest adapters can be used for applying automated test generation for testing GUIs of industry applications. They bridge the gap between automated test generation tools and industry applications. The development of test adapters requires no tool-specific knowledge and can be performed by members of the development team.

Test Cases Generation on Robotics for basis Path Testing using Genetic Algorithm

The paper explores the Genetic Algorithm approach to generate adequate and accurate test data for a specific target path. Software plays an important role in many of the systems, where the usage of software for a variety of purposes in different domains of modern life is rapidly increasing. With advancements in technology, it becomes quite complex whereas, software often contains errors. So testing consumes more money and time, which leads to automation that reduces human effort in finding bugs and errors. Actually, Automation in the last phase of system development is similar to manual testing. In both cases, bugs are detected only after code has been complete. Software testing is the most important component of software development process. Path testing is a popular structural testing method which uses the source code of a program to find every possible executable path. Test data generation is a key problem in software testing and its automation will improve the efficiency and effectiveness of software testing. Genetic Algorithm is an adaptive heuristic search algorithm that premise on evolutionary ideas of natural selection and genetic. In this paper, Genetic Algorithm is used to generate path by converting the program into its corresponding Control Flow Graph and then automatically generates the test data for the target path using different sets of GA operators.

Software Testing Techniques in Information Systems Curricula

Software testing is an important part of information systems (IS) development process. To achieve effectiveness in software testing, the participating IS professionals must apply software testing techniques. A review of the current IS curriculum models reveals that specific pedagogical guidelines are not available for instructing software testing techniques. This paper discusses the importance of software testing to IS development and maintenance, reviews the existing software testing techniques, and provides a pedagogical guideline for instructing software testing techniques in IS curricula.

Using Fuzzy Logic Techniques for Assertion-Based Software Testing Metrics.

Software testing is a very labor intensive and costly task. Therefore, many software testing techniques to automate the process of software testing have been reported in the literature. Assertion-Based automated software testing has been shown to be effective in detecting program faults as compared to traditional black-box and white-box software testing methods. However, the applicability of this approach in the presence of large numbers of assertions may be very costly. Therefore, software developers need assistance while making decision to apply Assertion-Based testing in order for them to get the benefits of this approach at an acceptable level of costs. In this paper, we present an Assertion-Based testing metrics technique that is based on fuzzy logic. The main goal of the proposed technique is to enhance the performance of Assertion-Based software testing in the presence of large numbers of assertions. To evaluate the proposed technique, an experimental study was performed in which the proposed technique is applied on programs with assertions. The result of this experiment shows that the effectiveness and performance of Assertion-Based software testing have improved when applying the proposed testing metrics technique.

New bounds for mixed covering arrays in t-way testing with uniform strength

ContextCombinatorial testing (CT) can increase the effectiveness of software testing by ensuring that all t-way input combinations are covered in a test suite. When software components have different input cardinalities, CT uses a mixed covering array (MCA) to represent the test suite. This study proposes a new methodology for constructing MCAs of t∈{2-6} by using Mixed-Tabu Search (MiTS) as the construction strategy. ObjectiveThe objective of this study is to significantly improve the best bounds of MCAs of t∈{2-6} with uniform strength. MethodThe proposed solution incorporates a new procedure for efficient parameter tuning where statistical testing is used to identify the setting values that significantly affect the performance of MiTS. For validation purposes, we used a robust benchmark that comprised a set of 35 instances of real cases and a set of 95 academic instances, which represented the best bounds reported previously. ResultThe experimental results showed that our MiTS-based methodology improved 93 bounds and matched 36 of them. The Wilcoxon signed-rank test demonstrated that our MiTS-based methodology significantly enhanced the best bounds of MCAs compared with those reported previously with 95% confidence. ConclusionMCAs for t-way testing with a good solution quality (in terms of test size), which involves artificial intelligence-based strategies, may be obtained by following a well-established methodology during the construction process.

面向需求覆盖的航天软件测试用例优化方法

An approach to optimize the aerospace software test cases based on the requirement coverage is researched in this paper to improve the efficiency of the software testing and increase the effectiveness of aerospace software testing. It is composed of the reduction and design of test cases, and has been used in a test of aerospace project. Firstly, the characteristics, testing process, and optimization techniques of aerospace software are analyzed. Then, a method to reduce test requirements is improved to simplify the test requirements. Furthermore, a design of test cases based on the correlative keywords of requirements is given, and the requirement coverage is achieved. Finally, the optimizing approach is applied in the test of aerospace software. The results of application show that the method can not only satisfy the demand of requirement coverage, but also make the workload of designing test cases reduce by 39%. The method ensures testing adequacy, and improves the test efficiency greatly.

An orchestrated survey of methodologies for automated software test case generation

Test case generation is among the most labour-intensive tasks in software testing. It also has a strong impact on the effectiveness and efficiency of software testing. For these reasons, it has been one of the most active research topics in software testing for several decades, resulting in many different approaches and tools. This paper presents an orchestrated survey of the most prominent techniques for automatic generation of software test cases, reviewed in self-standing sections. The techniques presented include: (a) structural testing using symbolic execution, (b) model-based testing, (c) combinatorial testing, (d) random testing and its variant of adaptive random testing, and (e) search-based testing. Each section is contributed by world-renowned active researchers on the technique, and briefly covers the basic ideas underlying the method, the current state of the art, a discussion of the open research problems, and a perspective of the future development of the approach. As a whole, the paper aims at giving an introductory, up-to-date and (relatively) short overview of research in automatic test case generation, while ensuring a comprehensive and authoritative treatment.