Software Testing Tasks Research Articles

Automated Test Case Generation for Path Coverage by Using Multi-Objective Particle Swarm Optimization Algorithm with Reinforcement Learning and Relationship Matrix Strategies

Software testing is an important phase in software life cycle, and test case generation occupies the main part of the testing workload. Path coverage is the most stringent and difficult coverage criteria, so automatic test case generation for path coverage (ATCG-PC) is an important task in software testing. ATCG-PC is usually abstracted as an optimization search problem, and the commonly used solution algorithm is swarm intelligence algorithm. This paper proposes a path coverage test case generation algorithm based on improved multi-objective particle swarm optimization (IMOPSO) algorithm. The algorithm uses the multi-objective strategy to generate test cases for a group of similar paths at a time, and is supplemented by the particle action selection strategy based on reinforcement learning and the secondary search algorithm based on relationship matrix, which makes use of the similarity information between paths to speed up the early convergence and help to quickly reach the optimal solution around the local optimal position. By grouping the paths according to the similarity, the algorithm reduces the test case consumption, and improves the efficiency of test case generation. Experimental results show that, compared with the existing methods, the proposed algorithm can achieve higher path coverage with less test case consumption and is also suitable for large-scale programs.

A Model of How Students Engineer Test Cases With Feedback

Background and Context. Students’ programming projects are often assessed on the basis of their tests as well as their implementations, most commonly using test adequacy criteria like branch coverage, or, in some cases, mutation analysis. As a result, students are implicitly encouraged to use these tools during their development process (i.e., so they have awareness of the strength of their own test suites). Objectives. Little is known about how students choose test cases for their software while being guided by these feedback mechanisms. We aim to explore the interaction between students and commonly used testing feedback mechanisms (in this case, branch coverage and mutation-based feedback). Method. We use grounded theory to explore this interaction. We conducted 12 think-aloud interviews with students as they were asked to complete a series of software testing tasks, each of which involved a different feedback mechanism. Interviews were recorded and transcripts were analyzed, and we present the overarching themes that emerged from our analysis. Findings. Our findings are organized into a process model describing how students completed software testing tasks while being guided by a test adequacy criterion. Program comprehension strategies were commonly employed to reason about feedback and devise test cases. Mutation-based feedback tended to be cognitively overwhelming for students, and they resorted to weaker heuristics in order to address this feedback. Implications. In the presence of testing feedback, students did not appear to consider problem coverage as a testing goal so much as program coverage . While test adequacy criteria can be useful for assessment of software tests, we must consider whether they represent good goals for testing, and if our current methods of practice and assessment are encouraging poor testing habits.

ESSENT: an arithmetic optimization algorithm with enhanced scatter search strategy for automated test case generation

As one of the main research tasks in software testing, automated test case generation based on path coverage (ATCG-PC) aims to achieve maximum path coverage with a minimized set of test cases. In ATCG-PC, the correlation among the dimensions of test cases is widely utilized in academia to minimize the search efforts of the search-based algorithm. Nevertheless, the information related to target path selection is not utilized, which leads to blind decision-making by the search-based algorithm during the target path selection. Therefore, this paper proposes an enhanced scatter search strategy by using opposition-based learning. An arithmetic optimization algorithm is also proposed to solve ATCG-PC based on the enhanced scatter search strategy, namely, ESSENT. The ESSENT algorithm selects the path with the lowest path entropy among the uncovered paths as the target path, and generates new test cases that cover the target path by modifying the dimensions of the existing test cases. The performance of the ESSENT algorithm is evaluated on six iFogSim subprograms and six Stanford coreNLP subprograms. Experiment results show that the ESSENT algorithm achieves a higher convergence rate than other state-of-the-art algorithms. Furthermore, it enables maximum path coverage with fewer test cases.

A comparison of reinforcement learning frameworks for software testing tasks

A comparison of reinforcement learning frameworks for software testing tasks

Automated test case generation for path coverage by using grey prediction evolution algorithm with improved scatter search strategy

Automated test case generation for path coverage (ATCG-PC), as an important task in software testing, aims to achieve the highest path coverage of a tested program by using as little computational overhead as possible. In ATCG-PC, “similar paths are usually executed by similar test cases” is a problem-specific knowledge which was touched by a handful of researchers but still underutilized. Inspired by the problem-specific knowledge, this paper designs a local search strategy by improving a scatter search strategy, and then proposes a grey prediction evolution algorithm with the improved scatter search strategy for ATCG-PC. Here, the improved scatter search strategy could obtain two feasible test cases by exploiting a dimension of a test case covering a certain path. The proposed algorithm is constructed by importing the improved scatter search strategy to the end of the reproduction operation of the grey prediction evolution algorithm holding strong exploration ability. Grey prediction evolution algorithm is first applied to solve ATCG-PC. The performance of the proposed algorithm is evaluated on six fog computing benchmark programs and six natural language processing benchmark programs. The experimental results demonstrate that the proposed algorithm can achieve the highest path coverage with the fewer test cases and running time than some state-of-the-art algorithms.

Role of Machine Learning & Artificial Intelligence Techniques in Software Testing

This study provides information about the use of artificial intelligence (AI) and machine learning (ML) techniques in the field of software testing. The use of AI in software testing is still in its initial stages. Also the automation level is lesser compared to more evolved areas of work.AI and ML can be used to help reduce tediousness and automate tasks in software testing. Testing can be made more efficient and smarter with the help of AI. Researchers recognize potential of AI to bridge the gap between human and machine driven testing capabilities. There are still number of challenges to fully utilize AI and ML techniques in testing but it will definitely enhance the entire testing process and skills of testers and will contribute in business growth. Machine learning research is a subset of overall AI research. The life-cycle of software is increasingly shortening and becoming more complicated. There is a struggle in software development between the competing pressures of developing software and meeting deadlines. AI-powered automated testing makes conducting full test suites in a timely manner on every change. In this article a detailed overview about the various applications of AI in software testing have been demonstrated. Also the implementation of machine learning in software testing has been discussed in detail and use of different machine learning techniques has been explained as well.

Scheduling sequence selection for generating test data to cover paths of MPI programs

Abstract Context: As one of key tasks in software testing, test data generation has been receiving widespread attention in recent years. Message-passing Interface (MPI) programs, which are one representative type of parallel programs, have the characteristic of non-determinism, which is reflected by the non-deterministic execution under different scheduling sequences against the same program input. Previous studies have shown that different difficulties are raised in generating test data under different scheduling sequences, suggesting that selecting appropriate scheduling sequences is beneficial to a high efficiency. Objective: We propose a method of selecting a superior and feasible scheduling sequence for generating test data in the criterion of path coverage against each target path of an MPI program. Method: In the proposed method, a number of program inputs are first sampled by Latin hypercube sampling. Then, the program is executed against each sample under each scheduling sequence, and all the scheduling sequences are sorted according to the similarities between the paths traversed by these samples and the target one. Finally, the feasibility of a scheduling sequence with the best quality is investigated based on the symbolic execution. Results: We apply the proposed method to seven typical MPI programs and compare it with the random one. The experimental results show that test data covering the target path can be generated under the selected scheduling sequence with high success rate and low time consumption. Conclusion: The proposed method takes the influence of scheduling sequences on generating test data into consideration, thus providing a competent way to test parallel programs.

A Survey of Automated Software Fault Localization Approach

Software testing run through the entire software development cycle, which consumes more than 50% of the development and maintenance effort. Fault localization is the most difficult and time-consuming task in software testing. Aimed at locating faults automatically in software, fault localization approach locates faults by running suitable test cases, so the quality of the test cases determines the effectiveness of fault localization. Therefore, the studies on test cases reduction approach for fault localization, automated fault-localization approach with high error-detection ability, and fault comprehension approach to help programmers understand the reason why a faulty statement cause failures have great significance for software testing and development. In this paper, the test cases reduction, fault localization, fault comprehension methods and their limitations are analyzed and summarized, and the future developing trend are discussed at last.

Test Exude: Approach for Test Case Reduction

Software development is a planned and structured process includes research, new development, modification, re-engineering results in software products. The structure of development of software is SDLC. It contains various process and activities to accomplish the task of the software product. In SDLC, testing is the most expensive phase. It is used to validate the software with all possible combinations. Exhaustive testing is impossible task in software testing, as, it is impractical with large software products. Now, automation is applied to generate the test cases for various applications, but the selection and reduction of test cases by automation is the main problem. This target only be achieved, when test cases is contained in large test suites. The technique we propose for test case reduction, called TestExude. It is a technique which gives test cases needed and discard other test cases from the Test Suite. It reduces the redundant test cases and its corresponding elements. This technique works on frequency of test cases in test suite and accepts largest frequency occurred in test suites and discard test cases on the base of frequency selected. Storage is well managed in this technique by reducing the test cases drastically in test suite. Test Suites size, user time, organization cost, storage cost are reduce efficiently. Ultimately, it improves and generate effective test case. This technique is more advantageous than any other techniques as it reduces the cost and time to its minimum extent.

An orchestrated survey of methodologies for automated software test case generation

Test case generation is among the most labour-intensive tasks in software testing. It also has a strong impact on the effectiveness and efficiency of software testing. For these reasons, it has been one of the most active research topics in software testing for several decades, resulting in many different approaches and tools. This paper presents an orchestrated survey of the most prominent techniques for automatic generation of software test cases, reviewed in self-standing sections. The techniques presented include: (a) structural testing using symbolic execution, (b) model-based testing, (c) combinatorial testing, (d) random testing and its variant of adaptive random testing, and (e) search-based testing. Each section is contributed by world-renowned active researchers on the technique, and briefly covers the basic ideas underlying the method, the current state of the art, a discussion of the open research problems, and a perspective of the future development of the approach. As a whole, the paper aims at giving an introductory, up-to-date and (relatively) short overview of research in automatic test case generation, while ensuring a comprehensive and authoritative treatment.

Confidence Interval Estimation of Software Reliability Growth Models based on Ohba’s Inflection S-shaped Model

Software reliability growth models with confidence intervals (SRGMs) are often utilized in software industry in that they provide useful information for software developers to decide the optimal software release time and to refine the quality of software testing tasks. Most SRGMs, nonetheless, do not have transparent explanations for the variance estimation of cumulative software errors. They might not be effective in deducing the confidence interval regarding the mean value function. In such cases, software developers cannot estimate the possible risk variation of software system by using the randomness of mean value function, and it might debase the practicability of applications. In this paper, we utilize Ohba’s Inflection Sshaped model to build the SRGM with confidence intervals that can assist the software developers in determining the optimal release time.

More testers – The effect of crowd size and time restriction in software testing

ContextThe questions of how many individuals and how much time to use for a single testing task are critical in software verification and validation. In software review and usability evaluation contexts, positive effects of using multiple individuals for a task have been found, but software testing has not been studied from this viewpoint. ObjectiveWe study how adding individuals and imposing time pressure affects the effectiveness and efficiency of manual testing tasks. We applied the group productivity theory from social psychology to characterize the type of software testing tasks. MethodWe conducted an experiment where 130 students performed manual testing under two conditions, one with a time restriction and pressure, i.e., a 2-h fixed slot, and another where the individuals could use as much time as they needed. ResultsWe found evidence that manual software testing is an additive task with a ceiling effect, like software reviews and usability inspections. Our results show that a crowd of five time-restricted testers using 10h in total detected 71% more defects than a single non-time-restricted tester using 9.9h. Furthermore, we use F-score measure from the information retrieval domain to analyze the optimal number of testers in terms of both effectiveness and validity of testing results. We suggest that future studies on verification and validation practices use F-score to provide a more transparent view of the results. ConclusionsThe results seem promising for the time-pressured crowds by indicating that multiple time-pressured individuals deliver superior defect detection effectiveness in comparison to non-time-pressured individuals. However, caution is needed, as the limitations of this study need to be addressed in future works. Finally, we suggest that the size of the crowd used in software testing tasks should be determined based on the share of duplicate and invalid reports produced by the crowd and by the effectiveness of the duplicate handling mechanisms.

DYNAMIC SEARCH SPACE TRANSFORMATIONS FOR SOFTWARE TEST DATA GENERATION

Among the tasks in software testing, test data generation is particularly difficult and costly. In recent years, several approaches that use metaheuristic search techniques to automatically obtain the test inputs have been proposed. Although work in this field is very active, little attention has been paid to the selection of an appropriate search space. The present work describes an alternative to this issue. More precisely, two approaches which employ an Estimation of Distribution Algorithm as the metaheuristic technique are explained. In both cases, different regions are considered in the search for the test inputs. Moreover, to depart from a region near to the one containing the optimum, the definition of the initial search space incorporates static information extracted from the source code of the software under test. If this information is not enough to complete the definition, then a grid search method is used. According to the results of the experiments conducted, it is concluded that this is a promising option that can be used to enhance the test data generation process.