Software Testing Techniques Research Articles

Effective State Encoding for Breadth-First Generation of Complex Structures

Generation of complex heap structures is required for many software testing and verification techniques, which are used to enhance software reliability. It enables them to work for programs dependent on such structures. Generating all the valid structures within a size bound is time-consuming, and it is hard to predict an input size completely generatable within a time bound. This is due to the depth-first traversal of the candidate space. However, an efficient breadth-first traversal requires a compact state encoding to store candidates for exploration in the next level. In this paper, we present present a novel state encoding technique that stores an incomplete representation of state that can be recovered during search. We build upon the Korat algorithm—demonstrated to effectively generate tests and find bugs for many software programs—and present iKorat , an incremental algorithm for breadth-first exploration of the search space. Our encoding enables a more efficient implementation of iterative deepening that avoids redundant work. Standard iterative deepening algorithms allow a breadth-first search when the underlying algorithm is depth-first by repeating the work of earlier iterations. iKorat, however, uses information from smaller sizes to avoid redundant work for larger sizes. It also enables a new technique for parallelizing Korat by communicating candidates using our encoding. piKorat generates structures of larger sizes in parallel as soon as incremental information is available from smaller sizes. Our evaluation shows that iKorat-based iterative deepening is more efficient than Korat and that piKorat naturally extends the technique for parallel generation of complex structures.

Developing and Evaluating Objective Termination Criteria for Random Testing

Random testing is a software testing technique through which programs are tested by generating and executing random inputs. Because of its unstructured nature, it is difficult to determine when to stop a random testing process. Faults may be missed if the process is stopped prematurely, and resources may be wasted if the process is run too long. In this article, we propose two promising termination criteria, “All Equivalent” (AEQ) and “All Included in One” (AIO), applicable to random testing. These criteria stop random testing once the process has reached a code-coverage-based saturation point after which additional testing effort is unlikely to provide additional effectiveness. We model and implement them in the context of a general random testing process composed of independent random testing sessions. Thirty-six experiments involving GUI testing and unit testing of Java applications have demonstrated that the AEQ criteria is generally able to stop the process when a code coverage equal or very near to the saturation level is reached, while AIO is able to stop the process earlier in cases it reaches the saturation level of coverage. In addition, the performance of the two criteria has been compared against other termination criteria adopted in the literature.

A Study on Software Process Framework and Testing Techniques

Software engineering is an engineering division connected with development of software product using well-defined scientific principles, techniques and actions. Two different types of activities like software framework activities and umbrella activities are included in standard software process model. Testing acts important job in software development process, i.e. the specified functionality and requirements are tested. The objective of this paper is to provide the software process framework and testing techniques for young researchers.

Employment of multiple algorithms for optimal path-based test selection strategy

Executing various sequences of system functions in a system under test represents one of the primary techniques in software testing. The natural way to create effective, consistent and efficient test sequences is to model the system under test and employ an algorithm to generate the tests that satisfy a defined test coverage criterion. Several criteria of test set optimality can be defined. In addition, to optimize the test set from an economic viewpoint, the priorities of the various parts of the system model under test must be defined. Using this prioritization, the test cases exercise the high priority parts of the system under test more intensely than those with low priority. Evidence from the literature and our observations confirm that finding a universal algorithm that produces an optimal test set for all test coverage and test set optimality criteria is a challenging task. Moreover, for different individual problem instances, different algorithms provide optimal results. In this paper, we present a path-based strategy to perform optimal test selection. The strategy first employs a set of current algorithms to generate test sets; then, it assesses the optimality of each test set by the selected criteria, and finally, chooses the optimal test set. The experimental results confirm the validity and usefulness of this strategy. For individual instances of 50 system under test models, different algorithms provided optimal results; these results varied by the required test coverage level, the size of the priority parts of the model, and the selected test set optimality criteria.

Review Paper on Various Software Testing Techniques & Strategies

Software testing is the process of running an application with the intent of finding software bugs (errors or other defects). Software applications demand has pushed the quality assurance of developed software towards new heights. It has been considered as the most critical stage of the software development life cycle. Testing can analyze the software item to identify the disparity between actual and prescribed conditions and to assess the characteristics of the software. Software testing leads to minimizing errors and cut down software costs. For this purpose, we discuss various software testing techniques and strategies. This paper aims to study diverse as well as improved software testing techniques for better quality assurance purposes.

Genetic algorithm based test data generation for MPI parallel programs with blocking communication

Parallel computing is one of mainstream techniques for high-performance computation in which MPI parallel programs have gained more and more attention. Genetic algorithms (GAs) have been widely employed in automated test data generation, leading to a major family of search-based software testing techniques. However, previous GA-based methods have limitations when testing MPI parallel programs with blocking communication. In this paper, we focus on the path coverage problem for MPI parallel programs with blocking communication, and formulate the problem as an optimization problem with its decision variable being the program input and the execution order of sending nodes. In addition, we develop target amending strategies for candidates when solving the problem using genetic algorithms. The proposed method is evaluated and compared with several state-of-the-art methods through a series of controlled experiments on five typical programs. The experimental results show that the proposed method can effectively and efficiently generate test data for path coverage.

Combining Model-Based Testing and Failure Modes and Effects Analysis for Test Case Prioritization: A Software Testing Approach

Due to The complexity of modern software projects and the increasing size of software systems, it becomes difficult to manually perform tests with limited resources. Also, manual testing cannot assure that the software is tested using all possible combinations of inputs. Therefore, automate software testing activities have become primordial in the Software Development Life Cycle (SDLC). Model-based testing is a prominent validation technique in software testing that uses models of the system under test to automatically generate test cases; this generation leads to a large number of test cases; which cannot exhaustively be executed, due to time and cost constraints. Moreover, the test-cases execution order has an influence on the rate at which faults can be detected. Therefore, it is essential to prioritize test cases in order to execute the most important with limited time and cost. On the other hand, The Failure Mode and Effects Analysis (FMEA) considered as one of the formal techniques to do risk based testing. It offers a structured methodology to identifying the system failure modes, analyzing their effects and setting up controls to risk reducing and improve the quality of systems or products. By applying such a technique to Model-based testing projects, we can benefit from FMEA analysis and the automating in same project. Through this paper, we introduced a new model based testing approach for prioritizing and ranking test cases according to the requirements and failure modes. In the suggested approach we used the Failure Mode and Effect Analysis methodology in model based testing way to automatically generate a set of pair {test case, priority number} to prioritize test cases. It differs from existing approaches in that it generates two types of test cases; requirement-based test cases and failure mode-based test cases. For the purpose of experimental evaluation and analysis, we will compare the suggested approach with some well-known prioritization methods and we will present a case study to illustrate the potential application of the proposed approach in a future work.

Adaptive Partition Testing

Random testing and partition testing are two major families of software testing techniques. They have been compared both theoretically and empirically in numerous studies for decades, and it has been widely acknowledged that they have their own advantages and disadvantages and that their innate characteristics are fairly complementary to each other. Some work has been conducted to develop advanced testing techniques through the integration of random testing and partition testing, attempting to preserve the advantages of both while minimizing their disadvantages. In this paper, we propose a new testing approach, adaptive partition testing , where test cases are randomly selected from some partition whose probability of being selected is adaptively adjusted along the testing process. We particularly develop two algorithms, Markov-chain based adaptive partition testing and reward-punishment based adaptive partition testing , to implement the proposed approach. The former algorithm makes use of Markov matrix to dynamically adjust the probability of a partition to be selected for conducting tests; while the latter is based on a reward and punishment mechanism. We conduct empirical studies to evaluate the performance of the proposed algorithms using ten faulty versions of three large-scale open source programs. Our experimental results show that, compared with two baseline techniques, namely random partition testing (RPT) and dynamic random testing (DRT), our algorithms deliver higher fault-detection effectiveness with lower test case selection overhead. It is demonstrated that the proposed adaptive partition testing is an effective testing approach, taking advantages of both random testing and partition testing.

ABCVS: An Artificial Bee Colony for Generating Variable T-Way Test Sets

To achieve acceptable quality and performance of any software product, it is crucial to assess various software components in the application. There exist various software-testing techniques such as combinatorial testing and covering array. However, problems such as t-way combinatorial explosion is still challenging in any combinatorial testing strategy, as it takes into consideration the entire combinations of input variables. Therefore, to overcome this problem, several optimizations and metaheuristic strategies have been suggested. One of the most effective optimization algorithms based techniques is the Artificial Bee Colony (ABC) algorithm. This paper presents t-way generation strategy for both a uniform and variable strength test suite by applying the ABC strategy (ABCVS) to reduce the size of the test suite and to subsequently enhance the test suite generation interaction. To assess both the effectiveness and performance of the presented ABCVS, several experiments were conducted applying various sets of benchmarks. The results revealed that the proposed ABCVS outweigh the existing based strategies and demonstrated wider interaction between components as opposed to AI-search based and computational based strategies. The results also revealed higher prospect of ABCVS in the aspect of its effectiveness and performance as observed in the majority of case studies.

Improved Bayesian regularisation using neural networks based on feature selection for software defect prediction

Demand for software-based applications has grown drastically in various real-time applications. However, software testing schemes have been developed which include manual and automatic testing. Manual testing requires human effort and chances of error may still affect the quality of software. To overcome this issue, automatic software testing techniques based on machine learning techniques have been developed. In this work, we focus on the machine learning scheme for early prediction of software defects using Levenberg-Marquardt algorithm (LM), Back Propagation (BP) and Bayesian Regularisation (BR) techniques. Bayesian regularisation achieves better performance in terms of bug prediction. However, this performance can be enhanced further. Hence, we developed a novel approach for attribute selection-based feature selection technique to improve the performance of BR classification. An extensive study is carried out with the PROMISE repository where we considered KC1 and JM1 datasets. Experimental study shows that the proposed approach achieves better performance in predicting the defects in software.

Efficient Software Testing Technique based on Hybrid Database Approach

In the field of computer science, software testing is referred as a critical process which is executed in order to assess and analyze the performance and risks existing in software applications. There is an emphasis on integrating specific approaches to carry out testing activities in an effective mode; the efficient strategy being explored recently is adopting hybrid database approach. For this purpose, a hybrid algorithm will be proposed to ensure the functionality and outcomes of testing procedure. The technical processes and its impact on current methodology would help to evaluate its effectiveness in software testing through which specific conclusions could be drawn. The findings of the research will elaborate effectiveness of the proposed algorithm that would be used in software testing. It would be suggested that new technology is easier and simple to assess and analyze the reliability of the software. Basically, hybrid database approach comprises of traditional and modern techniques that are deployed in order to achieve testing outcomes. It is explored from various testing methods that challenges have been identified related to focusing on traditional techniques due to which hybrid approach is now being developed in most of the areas. In the light of addressing these concepts, the paper aims to investigate the complexity and efficiency of hybrid database approach in software testing, as well as its scope in the IT industry.

SOFTWARE TESTING TECHNIQUES AND PRINCIPLES

This paper describes Software testing, need for software testing, Software testing goals and principles. Further it describe about different Software testing techniques and different software testing strategies. Finally it describes the difference between software testing and debugging.To perform testing effectively and efficiently, everyone involved with testing should be familiar with basic software testing goals, principles, limitations and concepts.We further explains different Software testing techniques such as Correctness testing, Performance testing, Reliability testing, Security testing. Further we have discussed the basic principles of black box testing, white box testing and gray box testing. We have surveyed some of the strategies supporting these paradigms, and have discussed their pros and cons. We also describes about different software testing strategies such as unit testing, Integration testing, acceptance testing and system testing.Finally there is comparison between debugging and testing. Testing is more than just debugging .Testing is not only used to locate defects and correct them it is also used in validation, verification process and measurement. A strategy for software Testing integrates software test case design methods into a well planned Series of steps that result in successful Construction of software that result in successful construction of software. Software testing Strategies gives the road map for testing. A software testing Strategy should be flexible enough to promote a customized testing approach at same time it must be right enough. Strategy is generally developed by project managers, software engineer and testing specialist. Software testing is an extremely creative and intellectually challenging task. When testing follows the principles given below, the creative element of test design and execution rivals any of the preceding software development steps, because testing requires high creativity and responsibility only the best personnel must be assigned to design, implement, and analyze test cases, test data and test results.

Experimental Evaluation of Redundancy in Android Mutation Testing

Because of the widespread usage of Android devices, the Android ecosystem has the highest numbers of users, developers, and app downloads. Researchers find that many Android apps are not sufficiently tested, which may lead to crashes, incorrect behaviors, and security vulnerabilities. Mutation testing is a syntax-based software testing technique that is very effective at designing high-quality tests and evaluating pre-existing tests. Our prior research designed and implemented Android mutation testing technique, and then used experiments to assess its strength. However, the high computational cost of Android mutation testing possibly limits its industrial application. This paper presents an experimental evaluation that investigates redundant mutation operators in Android mutation analysis. While maintaining the test quality, our goal is to reduce the cost by excluding redundant mutation operators or improving their design and implementation. In our evaluation, we first generate mutants and design mutation-adequate tests for each mutation operator. Then, we compute redundancy scores for each pair of mutation operators. Our evaluation results indicate that three operators (AODU, AOIU, and LOI) are redundant in Android mutation analysis. Other three operators (FOB, TVD, and ORL) are very hard to kill. One operator (MDL) needs improvement in its design to eliminate trivial mutants. We also identity subsumption relationships among operators (BWS subsumes BWD, ODL subsumes CDL, COD, and VDL).

Fuzzing: State of the Art

As one of the most popular software testing techniques, fuzzing can find a variety of weaknesses in a program, such as software bugs and vulnerabilities, by generating numerous test inputs. Due to its effectiveness, fuzzing is regarded as a valuable bug hunting method. In this paper, we present an overview of fuzzing that concentrates on its general process, as well as classifications, followed by detailed discussion of the key obstacles and some state-of-the-art technologies which aim to overcome or mitigate these obstacles. We further investigate and classify several widely used fuzzing tools. Our primary goal is to equip the stakeholder with a better understanding of fuzzing and the potential solutions for improving fuzzing methods in the spectrum of software testing and security. To inspire future research, we also predict some future directions with regard to fuzzing.

Assessing the Effectiveness of Coverage-Based Fault Localizations Using Mutants

Empirical studies show that coverage-based fault localizations are very effective in testing and debugging software applications. It is also a commonly held belief that no software testing techniques would perform best for all programs with various data structures and complexity. An important research question posed in this paper is whether the type and complexity of faults in a given program has any influence on the performance of these fault localization techniques. This paper investigates the performance of coverage-based fault localizations for different types of faults. We explore and compare the accuracy of these techniques for two large groups of faults often observed in object-oriented programs. First, we explore different types of traditional method-level faults grouped into six categories including those related to arithmetic, relational, conditional, logical, assignment, and shift. We then focus on class-level faults related to object-oriented features and group them into four categories including inheritance, overriding, Java-specific features, and common programming mistakes. The results show that coverage-based fault localizations are less effective for class-level faults associated with object-oriented features of programs. We therefore advocate the needs for designing more effective fault localizations for debugging object-oriented and class-level defects.

Test Suite Minimization in Regression Testing Using Hybrid Approach of ACO and GA

This article describes about the application of search-based techniques in regression testing and compares the performance of various search-based techniques for software testing. Test cases tend to increase exponentially as the software is modified. It is essential to remove redundant test cases from the existing test suite. Regression testing is very costly and must be performed in restricted ways to ensure the validity of the existing software. There exist different methods to improve the quality of test cases in terms of the number of faults covered, opposed to the number of statements covered in a minimum time. Different methods exist for this purpose, such as minimization, test case selection, and test case prioritization. In this article, search-based methods are applied to improve the quality of the test suite in order to select a minimum set of test cases which covers all the statements in a minimum time. The whole approach is named search based regression testing. In this paper, the performance of different metaheuristics for test suite minimization problem is also compared with a hybrid approach of ant colony optimization algorithm and genetic algorithm.

Using Genetic Algorithms in Test Data Generation

Software testing activities account for a considerable portion of systems development cost and, for this reason, many studies have sought to automate these activities. Test data generation has a high cost reduction potential (especially for complex domain systems), since it can decrease human effort. Although several studies have been published about this subject, articles of reviews covering this topic usually focus only on specific domains. This article presents a systematic mapping aiming at providing a broad, albeit critical, overview of the literature in the topic of test data generation using genetic algorithms. The selected studies were categorized by software testing technique (structural, functional, or mutation testing) for which test data were generated and according to the most significantly adapted genetic algorithms aspects. The most used evaluation metrics and software testing techniques were identified. The results showed that genetic algorithms have been successfully applied to simple test data generation, but are rarely used to generate complex test data such as images, videos, sounds, and 3D (three-dimensional) models. From these results, we discuss some challenges and opportunities for research in this area.

Representation and automatic generation of state-transition mapping tree

Among software testing techniques for the detection of software defects, state-transition-based testing which utilizes state diagrams is pretty well known and is being used widely. Such software testing techniques that are based on specifications of a software and perform various tests by utilizing a number of state diagrams are quite well known. According to different testing environment including individuals, applied methods, and the size of a perceived system, the resulting diagrams might highly show the discrepancy regarding some perspectives such as correctness, the quality of representation, and so on. This significantly affects the understanding of a system such that the system under consideration might be interpreted in different ways. The highly complex state diagrams can have a negative effect on the productive and positive utilization of state diagrams in the software system development including the analysis or the state-transition-based test process. Many state diagram-based process conventional approaches for automatic software analysis do not take the complexity problem into deep consideration so much. Moreover, the state diagrams generated by the methods that place a strong emphasis on complexity can generate inappropriate test cases according to the possible misled guidelines or the incorrect objectives. In this paper, we propose methods for resolution of the complexity problem in a state diagram without modification of the states or the transition relationships. In our proposed methods, we provide a representation model in which a software system can be interpreted in the context of a state diagram and an automatic generation method for the proposed representation. We also illustrate a case study for justification of plausibility of the proposed methods.

Risks in Software Testing Techniques A Literature Review

Risks in Software Testing Techniques A Literature Review

ASPECT-ORIENTED SOFTWARE TESTING TECHNIQUES: A REVIEW

- Software testing is a prominent area of research as it ensures the quality and reliability of the software. Testing an aspect-oriented software is also the latest area of research. Although, research in aspect-oriented software testing has reported a couple of new testing techniques. But, there is no work yet indicated in the literature to focus on the basic fundamentals of the aspect-oriented software testing and relationship with traditional software testing techniques. Therefore, this position paper handles this shortcoming and presents the basic concepts of aspect-oriented programming, phases of aspect-oriented software development life cycle and testing phases for new researchers to gather the information on the subject. This paper also presents a comprehensive detail about the relationship of aspect-oriented software testing techniques with other traditional testing techniques.