Test Case Selection Research Articles

Automating TEST Case Design, Selection and Evaluation Report on 10 Editions of A-TESTWorkshop

Trends such as globalisation, standardisation and shorter life-cycles place great demands on the flexibility of the software industry. In order to compete and cooperate on an international scale, a constantly decreasing time to market and an increasing level of quality are essential. Testing is at the moment the most important and mostly used quality assurance technique applied in industry. However, the complexity of software and hence of their development amount is increasing. Modern systems get larger and more complex, as they connect large amounts of components that interact in many different ways and have constantly changing and different types of requirements (functionality, dependability, usability, performance etc.). Data processing that impacts all aspects of our life is increasingly distributed over clouds and devices. This leads to new concerns, such as availability, security, and privacy, which are aspects that also needs to be tested. Consequently, the development of cost-effective and high-quality systems opens new challenges that cannot be faced only with traditional testing approaches, and specifically manual testing is simply insufficient and unreliable to manage the speed needed, and ensure the coverage of ever-changing systems. New techniques for systematization and automation of testing throughout the software and system life-cycle are required.

The Application Of Machine Learning In Test Case Prioritization - A Review

Regression testing is very important but also a very costly and time-consuming activity that ensures the developers that changes in the application will not bring new errors. Retest all, selection of test cases and prioritization of test cases (TCP) approaches are used to enhance the efficiency and effectiveness in regression testing. While test case selection techniques decrease testing time and cost, it can exclude some critical test cases that can detect the faults. On the other hand, test case prioritization considers all test cases and execute them until resources are exhausted or all test cases are executed, while always focusing on the most important ones. Over the years, machine learning has found wide usage in solving different problems in software engineering. Software development and maintenance problems can be defined as learning problems and machine learning techniques have shown to be very effective in solving these problems. In the range of application of machine learning, machine learning techniques have also found usage in solving the test case prioritization problem. In this paper, we investigate the application of machine learning techniques in test case prioritization. We survey some of the most recent studies made in this field and provide information like techniques of machine learning used in TCP process, metrics used to measure the effectiveness of the proposed methods, data used to define the priority of test cases and some advantages or limitations of application of machine learning in TCP.

CSEFuzz: Fuzz Testing Based on Symbolic Execution

Fuzz testing has been successful in finding defects of various software packages. These defects include file parsing, image processing, Internet browsers, and network protocols. However, the quality of the initial seed test cases greatly influences the coverage and defect detection capability of fuzz testing. To address this issue, we propose CSEFuzz, a fuzz testing approach based on symbolic execution for defect detection. First, CSEFuzz generates candidate test cases by symbolic execution and collects coverage information of the test cases. Then, CSEFuzz extracts the test-case templates of the test cases and selects a set of test-case templates according to specific coverage criteria. Finally, CSEFuzz selects test cases according to the selected test-case templates, and the selected test cases are used as initial seed test cases for fuzz testing. Experiments are conducted on 11 open-source programs. The results show that in comparison with afl-cmin, which is the test-case selection command of Kelinci, CSEFuzz with a path coverage criterion reduces the time costs of the initial seed test selection and verification by 94.26%. In addition, compared with afl-cmin, 32 more paths are covered and 16 more defects are detected by CSEFuzz.

Selecting Test Cases for Mechatronic Products with a Variant and Version Management Approach based on a Consistent Toolchain

The number of variants and versions for mechatronic products increases. The high variability poses a challenge for test engineers in selecting suitable test cases upon a change. If a requirement or a feature of a mechatronic product changes, it is not necessary to retest the whole product but only the changed parts. To identify the product features that are directly or indirectly affected by the change, a connection of test, requirement, and variant management is necessary. Therefore, an approach to select test cases based on an occurred change using variant and version knowledge is needed. In this paper, such an approach and its possible application in a toolchain are introduced. The toolchain is built by combining established tools developed by the Parametric Technology Corporation (PTC) that are already used to manage parts of the product life cycle. The resulting PTC Integrity Toolchain and the applicability of the concept on it were evaluated together with industrial experts with positive results.

Towards Pattern-Based Change Verification Framework for Cloud-Enabled Healthcare Component-Based

To survive in the competitive environment, most organizations have adopted component-based software development strategies in the rapid technology advancement era and the proper utilization of cloud-based services. To facilitate the continuous configuration, reduce complexity, and faster system delivery for higher user satisfaction in dynamic scenarios. In cloud services, customers select services from web applications dynamically. Healthcare body sensors are commonly used for diagnosis and monitoring patients continuously for their emergency treatment. The healthcare devices are connected with mobile or laptop etc. on cloud environment with network and frequently change applications. Thus, organizations rely on regression testing during changes and implementation to validate the quality and reliability of the system after the alteration. However, for a large application with limited resources and frequently change component management activities in the cloud computing environment, component-based system verification is difficult and challenging due to irrelevant and redundant test cases and faults. In this study, proposed a test case selection and prioritization framework using a design pattern to increase the faults detection rate. First, we select test cases on frequently accessed components using observer patterns and, secondly, prioritize test cases on adopting some strategies. The proposed framework was validated by an experiment and compared with other techniques (previous faults based and random priority). Hence, experimental results show that the proposed framework successfully verified changes. Subsequently, the proposed framework increases the fault detection rate (i.e., more than 90%) than previous faults based and random priority (i.e., more than 80% respectively).

A Framework for Continuous Regression and Integration Testing in IoT Systems Based on Deep Learning and Search-Based Techniques

Tremendous systems are rapidly evolving based on the trendy Internet of Things (IoT) in various domains. Different technologies are used for communication between the massive connected devices through all layers of the IoT system, causing many security and performance issues. Regression and integration testing are considered repeatedly, in which the vast costs and efforts associated with the frequent execution of these inflated test suites hinder the adequate testing of such systems. This necessitates the focus on exploring innovative scalable testing approaches for large test suites in IoT-based systems. In this paper, a scalable framework for continuous integration and regression testing in IoT-based systems (IoT-CIRTF) is proposed, based on IoT-related criteria for test case prioritization and selection. The framework utilizes search-based techniques to provide an optimized prioritized set of test cases to select from. The selection is based on a trained prediction model for IoT standard components using supervised deep learning algorithms to continuously ensure the overall reliability of IoT-based systems. The experiments are held on two GSM datasets. The experimental results achieved prioritization accuracy up to 90% and 92% for regression testing and integration testing respectively. This provides an enhanced and efficient framework for continuous testing of IoT-based systems, as per IoT-related criteria for the prioritization and selection purposes.

Mobile cloud computing applications penetration testing model design

Mobile cloud computing (MCC) is a promising technology due to its features that mitigate mobile computing limitations and enhances cloud services. However, penetration testing is more challenging when conducted on MCC applications. These applications use offloading, and thus another layer of complexity in generating, selecting and executing test cases, which implies and requires an MCC applications penetration testing offloading-awareness model. To overcome these challenges, a penetration testing model for mobile cloud computing applications is designed. This model defines the process of penetration testing over MCC applications including penetration test preparation, test case generation, selection and execution processes. Key components of this offloading-awareness model are state management and mobile agent while other components are adapted from previous penetration testing models for the web, cloud or mobile applications. This model will enable penetration testers to tackle the mobile cloud computing complexity and uniqueness. Currently, we are preparing the evaluation of the model against these MCC applications.

An Effective Regression Test Case Selection Using Hybrid Whale Optimization Algorithm

Test suite optimization is an ever-demanded approach for regression test cost reduction. Regression testing is conducted to identify any adverse effects of maintenance activity on previously working versions of the software. It consumes almost seventy percent of the overall software development lifecycle budget. Regression test cost reduction is therefore of vital importance. Test suite optimization is the most explored approach to reduce the test suite size to re-execute. This article focuses on test suite optimization as a regression test case selection, which is a proven N-P hard combinatorial optimization problem. The authors have proposed an almost safe regression test case selection approach using a Hybrid Whale Optimization Algorithm and empirically evaluated the same on subject programs retrieved from the Software Artifact Infrastructure Repository with Bat Search and ACO-based regression test case selection approaches. The analyses of the obtained results indicate an improvement in the fault detection ability of the proposed approach over the compared ones with significant reduction in test suite size.

A Fuzzy Rule Based Approach for Test Case Selection Probability Estimation in Regression Testing

Regression testing is a very essential activity during maintenance of software. Due to constraints of time and cost, it is not possible to re-execute every test case with respect to every change occurred. Thus, a technique is required that selects and prioritises the test cases efficiently. This paper proposes a novel fuzzy rule-based approach for selecting and ordering a number of test cases from an existing test suite to predict the selection probability of test cases using multiple factors. The test cases, which have ability to find high fault detection rate with maximum coverage and minimum execution time to test are selected. The results specify the effectiveness of the proposed model for predicting the selection probability of individual test cases.

Wind turbine power curve upgrades: methods for the assessment and test cases study

The research about wind turbine control and blade design optimization has flourished in the latest years and has provided the opportunity of diffusely updating the technology of operating wind turbines. Due to multivariate dependence of wind turbine power on ambient conditions and working parameters, it is complex to estimate the actual impact of power optimization strategies. This problem therefore calls for devoted operation data mining and statistical techniques, which are explored in the present work. In particular, two test cases of multi-MW wind turbines power upgrades are discussed: the former is a combined aerodynamic and control optimization, the latter is the optimization of the yaw control. The assessment of the upgrades impact is performed through the comparison between the post-upgrade measured production and a model estimate of the pre-upgrade production in the same conditions. The wind turbines nearby to the target upgraded ones are employed as references for the operation conditions and their working parameters are employed for a principal component regression of the power of the target wind turbine. The proposed method is general and, for the selected test cases, it arises that the aerodynamic and control optimization improves the Annual Energy Production of the order of the 3%, while the yaw control optimization provides a 1% AEP improvement.

The CMS approach to Analysis Preservation

The CERN analysis preservation portal (CAP) comprises a set of tools and services aiming to assist researchers in describing and preserving all the components of a physics analysis such as data, software and computing environment. Together with the associated documentation, all these assets are kept in one place so that the analysis can be fully or partially reused even several years after the publication of the original scientific results. An experimentspecific submission and retrieval interface has been developed for the CMS Collaboration. It integrates with the CMS internal analysis registry (CADI) to capture all analyses with basic information, complemented with a detailed submission form for full information. The CMS data aggregation system (DAS) is interfaced to the deposit form to assist in filling in exact dataset names used in the analysis to ensure searchability. Efforts are ongoing to describe physics content for an intelligent retrieval, and to interface with container solutions for full reproducibility for selected test cases.

Dynamic Random Testing of Web Services: A Methodology and Evaluation

In recent years, service oriented architecture (SOA) has been increasingly adopted to develop distributed applications in the context of the Internet. To develop reliable SOA-based applications, an important issue is how to ensure the quality of web services. In this article, we propose a dynamic random testing (DRT) technique for web services, which is an improvement over the widely-practiced random testing (RT) and partition testing (PT) approaches. We examine key issues when adapting DRT to the context of SOA, including a framework, guidelines for parameter settings, and a prototype for such an adaptation. Empirical studies are reported where DRT is used to test three real-life web services, and mutation analysis is employed to measure the effectiveness. Our experimental results show that, compared with the three baseline techniques, RT, Adaptive Testing (AT) and Random Partition Testing (RPT), DRT demonstrates higher fault-detection effectiveness with a lower test case selection overhead. Furthermore, the theoretical guidelines of parameter setting for DRT are confirmed to be effective. The proposed DRT and the prototype provide an effective and efficient approach for testing web services.

Forecasting Repair Schedule for Building Components Based on Case-Based Reasoning and Fuzzy-AHP

Building maintenance is closely related to the performance and sustainability of buildings. However, existing approaches to maintenance are limited in terms of estimating required repairs. Therefore, this study proposes a case-based reasoning (CBR)-based model for estimating the time when the first repair will be needed after the completion of construction, even in phases where maintenance-related information is scarce. CBR and fuzzy-analytic hierarchy process (AHP) were employed as research methodologies. A database was established by collecting 257 cases related to maintenance of apartment buildings, and attributes were extracted through literature reviews and expert interviews. Then, attributes were weighted by fuzzy-AHP and case similarities were computed by measuring the Euclidean distance. Similar cases were retrieved based on similarity scores. The model was validated via a comparison of 20 randomly selected test cases with the output of retrieved cases. The results showed that the average case similarities of 3-, 5-, 7-, and 10-nearest neighbors (NN) were 98.05%, 97.86%, 97.73%, and 97.59%, respectively, and mean absolute percentage errors for 3-, 5-, 7-, and 10-NN were mostly lower than 20%, confirming the applicability of the proposed model. The proposed method will help in the preliminary estimation of the repair time of building components.

Hybrid swarm and GA based approach for software test case selection

<span>Being a crucial step and deciding factor for software reliability, software testing has evolved a long way and always attracted researchers due to various inherent challenges. The quality of a software application depends on the effectiveness of the testing carried out during development and maintenance phase. Testing is a crucial but time consuming activity that influences the overall cost of software development. Thus a minimal but efficient test suite selection is the need of the hour. This paper presents a hybrid technique based on swarm based search technique and GA (Genetic Algorithm) for selection of promising test cases to reduce the overall development cost and time of the application. We took component based software into consideration as they offer some inherent advantages over traditional software development paradigms.</span>

Evolutionary Cost-cognizant Test Case Selection and Prioritization for Object-Oriented Programs

Test case prioritization (TCP) is a software testing technique that finds an ideal ordering of test cases for regression testing, so that testers can obtain the maximum benefit of their test suite, even if the testing process is stop at some arbitrary point. The recent trend of software development uses OO paradigm. This paper proposed a cost-cognizant TCP approach for object-oriented software that uses path-based integration testing. Path-based integration testing will identify the possible execution path and extract these paths from the Java System Dependence Graph (JSDG) model of the source code using forward slicing technique. Afterward evolutionary algorithm (EA) was employed to prioritize test cases based on the severity detection per unit cost for each of the dependent faults. The proposed technique was known as Evolutionary Cost-Cognizant Regression Test Case Prioritization (ECRTP) and being implemented as regression testing approach for experiment.

Regression test case selection and prioritization for object oriented software

In software maintenance, after modifying the software a system needs regression testing. Execution of regression testing confirms that any modified code has no adverse effect as well as does not introduce new faults in the existing functionality of the software. When working with object-oriented programming code-based testing is generally expensive. In this study, we proposed a technique for regression testing using unified modeling language (UML) diagrams and code-based analysis for object-oriented software. In this research work, the design and code based technique with an evolutionary approach are presented to select the best possible test cases from the test suite. We used the dependency graph for intermediate representation for the objectoriented program to identify the change. The selection of test cases is done at the design level using the UML model. The models are compared to identify the change between these two models. The proposed approached maximizes the value of APFD.

Computation of optimal transport on discrete metric measure spaces

In this paper we investigate the numerical approximation of an analogue of the Wasserstein distance for optimal transport on graphs that is defined via a discrete modification of the Benamou–Brenier formula. This approach involves the logarithmic mean of measure densities on adjacent nodes of the graph. For this model a variational time discretization of the probability densities on graph nodes and the momenta on graph edges is proposed. A robust descent algorithm for the action functional is derived, which in particular uses a proximal splitting with an edgewise nonlinear projection on the convex subgraph of the logarithmic mean. Thereby, suitable chosen slack variables avoid a global coupling of probability densities on all graph nodes in the projection step. For the time discrete action functional $$\Gamma $$-convergence to the time continuous action is established. Numerical results for a selection of test cases show qualitative and quantitative properties of the optimal transport on graphs. Finally, we use our algorithm to implement a JKO scheme for the gradient flow of the entropy in discrete transportation distance, which is known to coincide with underlying Markov semigroup, and test our results against a classical backward Euler discretization of this discrete heat flow.

A review of object-oriented approach for test case prioritization

<p>The most essential phase in regression testing is Test Case Prioritization (TCP), with its primary objective to increase the fault detection rate at different stages during testing. Prior to achieving the objective, existing evidence of techniques in TCP must be synthesized and analyzed. At present, fault detection for TCP based on object-oriented features only consider statement, module, and class level. The important features of object-oriented (OO) programming like inheritance and polymorphism have not been fully explored for fault detection in TCP. Such OO concepts are important for test case selection and in turn for ranking the test cases (prioritization). This paper reviews various test case prioritization techniques specific to OO systems. This review is hoped to highlight the importance and usage of TCP in relation to object-oriented software development lifecycle.</p>

Analytical and Numerical Propagation Loss Predictions for Gradually Range-Dependent Isospeed Waveguides

An analytical formulation for the calculation of the range and depth-dependence of propagation loss in isospeed water is introduced. The range-dependent bathymetry is handled with Weston's ray invariant approach, requiring a gradually varying water depth. The depth dependence of propagation loss is formulated using a transformation from an incoherent mode sum to an integral over angle, in the adiabatic approximation, and the results obtained in this way are tested by comparison with a full adiabatic normal mode sum. The validity of the adiabatic approximation itself is then investigated by means of a comparison with parabolic equation and coupled normal mode results for selected test cases from the Weston Memorial Workshop, held at the University of Cambridge in April 2010. Propagation loss calculations based on different methods are compared. The proposed analytical approach provides practical, fast and accurate results.

Selection of Best Test Cases using Support Vector Machine for ARPT

Software testing is an essential activity in software industries for quality assurance; subsequently, it can be effectively removing defects before software deployment. Mostly good software testing strategy is to accomplish the fundamental testing objective while solving the trade-offs between effectiveness and efficiency testing issues. Adaptive and Random Partition software Testing (ARPT) approach was a combination of Adaptive Testing (AT) and Random Partition Approach (RPT) used to test software effectively. It has two variants they are ARPT-1 and ARPT-2. In ARPT-1, AT was used to select a certain number of test cases and then RPT was used to select a number of test cases before returning to AT. In ARPT-2, AT was used to select the first m test cases and then switch to RPT for the remaining tests. The computational complexity for random partitioning in ARPT was solved by cluster the test cases using a different clustering algorithm. The parameters of ARPT-1 and ARPT-2 needs to be estimated for different software, it leads to high computation overhead and time consumption. It was solved by Improvised BAT optimization algorithms and this approach is named as Optimized ARPT1 (OARPT1) and OARPT2. By using all test cases in OARPT will leads to high time consumption and computational overhead. In order to avoid this problem, OARPT1 with Support Vector Machine (OARPT1-SVM) and OARPT2-SVM are introduced in this paper. The SVM is used for selection of best test cases for OARPT-1 and OARPT-2 testing strategy. The SVM constructs hyper plane in a multi-dimensional space which is used to separate test cases which have high code and branch coverage and test cases which have low code and branch coverage. Thus, the SVM selects the best test cases for OARPT-1 and OARPT-2. The selected test cases are used in OARPT-1 and OARPT-2 to test software. In the experiment, three different software is used to prove the effectiveness of proposed OARPT1-SVM and OARPT2-SVM testing strategies in terms of time consumption, defect detection efficiency, branch coverage and code coverage.