Test Case Selection Research Articles

ОБ ОЦЕНКЕ ПОГРЕШНОСТИ РАСЧЕТА МОЩНОСТИ МЕТОДОМ АНАЛИЗА НЕОПРЕДЕЛЕННОСТИ И ЧУВСТВИТЕЛЬНОСТИ ПРИ ВЕРИФИКАЦИИ СОПРЯЖЕННЫХ НЕЙТРОННО-ТЕПЛОГИДРАВЛИЧЕСКИХ КОДОВ

This paper describes the capability of sensitivity and uncertainty analysis (SUA) for estimation of power calculation error during the computer code verification compared to the data received in experiments at operating power plants. Estimation of errors in code verification calculation against measurements at operating plants and assessment of conservatism in safety calculations are considered as two interrelated problems. A defining moment in this approach is that one or more test cases are selected; selected test cases are included in matrix of code verification assuming that these test cases involve the same physical phenomena as in simulated accident scenarios. The criterion of test case “representativeness” is similarity between correlation coefficients of model predictions in accident scenario and test case. The example demonstrates how the SUA approach is used in calculations with the VVER-1000 coupled neutronic and thermal-hydraulic simulation model based on the KORSAR/GP and SAPFIR_95&RC_VVER codes.

HSP: A hybrid selection and prioritisation of regression test cases based on information retrieval and code coverage applied on an industrial case study

The usual way to guarantee quality of software products is via testing. This paper presents a novel strategy for selection and prioritisation of Test Cases (TC) for Regression testing. In the lack of code artifacts from where to derive Test Plans, this work uses information conveyed by textual documents maintained by Industry, such as Change Requests. The proposed process is based on Information Retrieval techniques combined with indirect code coverage measures to select and prioritise TCs. The aim is to provide a high coverage Test Plan which would maximise the number of bugs found. This process was implemented as a prototype tool which was used in a case study with our industrial partner (Motorola Mobility). Experiments results revealed that the combined strategy provides better results than the use of information retrieval and code coverage independently. Yet, it is worth mentioning that any of these automated options performed better than the previous manual process deployed by our industrial partner to create test plans.

Can a computer see what an ice expert sees? Multilabel ice objects classification with convolutional neural networks

Abstract Computer-aided scene analysis has drawn much attention, especially in autonomous navigation and advanced navigation assistance systems for surface vessels. In ice-infested waters, multilabel ice object classification and segmentation form the core of these systems, which are required for path-planning and collision avoidance algorithms. This study focuses on the interpretation of ice conditions from close-range optical imagery. It presents a model for multilabel ice object classification that builds on state-of-the-art open source libraries and deep learning platforms. This work explores the generalization ability of open source models to differentiate between nine categories of surface ice features: level ice, deformed ice, broken ice, icebergs, floebergs, floebits, ice floes, pancake ice, and brash ice. The results demonstrate the ability of the models to classify these nine categories from optical close-range images, which were gathered online and during a research cruise to the Fram Strait on the RV Lance in 2012. We tested a variety of classification algorithms on the collected ice imagery and compared the results against randomly selected test cases representing different ice features with different degrees of local texture distortion. In doing so, we can evaluate the effectiveness of the classification of different classes and compare different levels of information presented for the classification. In addition, we provide a model implementation of our model: a GitHub repository, ICEXPERT, that is suitable for ice object classification from close-range ice imagery.

Enhanced regression testing technique for agile software development and continuous integration strategies

To survive in competitive marketplaces, most organizations have adopted agile methodologies to facilitate continuous integration and faster application delivery and rely on regression testing during application development to validate the quality and reliability of the software after changes have been made. Consequently, for large projects with cost and time constraints, it is extremely difficult to determine which test cases to run at the end of each release. In this paper, a test case prioritization and selection approach is proposed to improve the quality of releases. From existing literature, we analyzed prevailing problems and proposed solution relevant to regression testing in agile practices. The proposed approach is based on two phases. First, test cases are prioritized by clustering those test cases that frequently change. In case of a tie, test cases are prioritized based on their respective failure frequencies and coverage criteria. Second, test cases with a higher frequency of failure or coverage criteria are selected. The proposed technique was validated by an empirical study on three industrial subjects. The results show that the method successfully selects an optimal test suite and increases the fault detection rate (i.e., more than 90% in the case of proposed technique and less than 50% in other techniques), which reduces the number of irrelevant test cases and avoids detecting duplicate faults. The results of evaluation metrics illustrate that the proposed technique significantly outperform (i.e., between 91 and 97%) as compared to other existing regression testing techniques (i.e., between 52 and 68%). Therefore, our model enhances the test case prioritization and selection with the ability for earlier and high fault detection. Thus, pruning out irrelevant test cases and redundant faults and enhancing the regression testing process for agile applications.

Automated code-based test selection for software product line regression testing

Regression testing for software product lines (SPLs) is challenging and can be expensive because it must ensure that all the products of a product family are correct whenever changes are made. SPL regression testing can be made efficient through a test case selection method that selects only the test cases relevant to the changes. Some approaches for SPL test case selection have been proposed but either they were not efficient by requiring intervention from human experts or they cannot be used if requirements specifications, architecture and/or traceabilities for test cases are not available or partially eroded. To address these limitations, we propose an automated method of source code-based regression test selection for SPLs. Our method reduces the repetition of the selection procedure and minimizes the in-depth analysis effort for source code and test cases based on the commonality and variability of a product family. Evaluation results of our method using six product lines show that our method reduces the overall time to perform regression testing by 14.8% ∼ 49.1% on average compared to an approach of repetitively applying Ekstazi, which is the state-of-the-art regression test selection method for a single product, to each product of a product family.

Out-of-equilibrium mechanochemistry and self-organization of fluid membranes interacting with curved proteins

The function of biological membranes is controlled by the interaction of the fluid lipid bilayer with various proteins, some of which induce or react to curvature. These proteins can preferentially bind or diffuse towards curved regions of the membrane, induce or stabilize membrane curvature and sequester membrane area into protein-rich curved domains. The resulting tight interplay between mechanics and chemistry is thought to control organelle morphogenesis and dynamics, including traffic, membrane mechanotransduction, or membrane area regulation and tension buffering. Despite all these processes are fundamentally dynamical, previous work has largely focused on equilibrium and a self-consistent theoretical treatment of the dynamics of curvature sensing and generation has been lacking. Here, we develop a general theoretical and computational framework based on a nonlinear Onsager’s formalism of irreversible thermodynamics for the dynamics of curved proteins and membranes. We develop variants of the model, one of which accounts for membrane curving by asymmetric crowding of bulky off-membrane protein domains. As illustrated by a selection of test cases, the resulting governing equations and numerical simulations provide a foundation to understand the dynamics of curvature sensing, curvature generation, and more generally membrane curvature mechano-chemistry.

Dynamic Random Testing: Technique and Experimental Evaluation

A particularly good software testing strategy is to achieve the underlying testing goal while solving the problems of tradeoffs between testing effectiveness and efficiency. To improve the fault detection effectiveness of software testing, the principle of feedback control theory was adopted, which motivated the proposal of dynamic random testing (DRT). The main idea behind DRT is using the testing results to guide the test case selection to increase the selection probabilities of the subdomains with higher fault detection rates. Previous works show that DRT strategy can achieve better effectiveness than random testing strategy and random partition testing strategy, and has significantly lower computational costs than adaptive testing strategy. However, the essential factors that affect the performance of DRT, i.e., adjusting parameters, initial profile, and test case classification have not been thoroughly investigated. Besides, some experimental assumptions are inconsistent with real scenarios. Therefore, this paper gives a series of investigations on DRT with a set of practical subject programs. More specifically, the effectiveness and efficiency of DRT are presented, and the extended experiments on DRT with relevant factors are conducted. The results indicate that the effectiveness of DRT is robust to different initial profiles and affected noticeably by the adjusting parameter settings and test case classification methods.

A new joining by forming process for fixing sheets to tubes

This paper is focused on fixing sheets to tubes away from the tube ends by annular sheet squeezing. The main objectives are the investigation of the physics behind material separation at the cross-section recess corner of the punches and the determination of the appropriate squeezing depth values. The methodology combines experimentation and finite element modelling and the overall presentation is illustrated with selected test cases retrieved from the experimental work plan. Results show that material separation is caused by through-thickness shear, that forces and energy required to form the new surfaces are negligible and that appropriate squeezing depth values result from a compromise between the quality of the mechanical interlocking and the pull-out destructive strength of the joints.

Rotational Diversity in Multi-Cycle Assignment Problems

In multi-cycle assignment problems with rotational diversity, a set of tasks has to be repeatedly assigned to a set of agents. Over multiple cycles, the goal is to achieve a high diversity of assignments from tasks to agents. At the same time, the assignments’ profit has to be maximized in each cycle. Due to changing availability of tasks and agents, planning ahead is infeasible and each cycle is an independent assignment problem but influenced by previous choices. We approach the multi-cycle assignment problem as a two-part problem: Profit maximization and rotation are combined into one objective value, and then solved as a General Assignment Problem. Rotational diversity is maintained with a single execution of the costly assignment model. Our simple, yet effective method is applicable to different domains and applications. Experiments show the applicability on a multi-cycle variant of the multiple knapsack problem and a real-world case study on the test case selection and assignment problem, an example from the software engineering domain, where test cases have to be distributed over compatible test machines.

Generalized framework for PIV-based pressure gradient error field determination and correction

A framework leveraging the governing equations of incompressible flow for the reconstruction and correction of pressure gradient estimation errors from experimental data is extended to incorporate non-zero errors on domain boundaries and Lagrangian pseudo-tracking methods for material acceleration estimation. A second-order system derived from the first-order divergence-curl system governing the pressure gradient error field facilitates handling a variety of error boundary conditions, and the solution can be split into independent Poisson equations, making it straightforward to implement. For known boundary errors, a precise determination of the pressure gradient field is possible, up to the limitations of the numerical method. In practice, the error equations cannot be solved exactly; however, a number of terms may be computed exactly and approximations may be applied on the remaining terms. For the selected test case of a cylinder wake flow in turbulent shedding regime, the analysis of simulated three-dimensional, three-component velocity measurements demonstrates that the errors in pressure estimates can be reduced by up to 50% using a basic finite difference implementation.

Pareto efficient multi-objective black-box test case selection for simulation-based testing

Abstract Context: In many domains, engineers build simulation models (e.g., Simulink) before developing code to simulate the behavior of complex systems (e.g., Cyber-Physical Systems). Those models are commonly heavy to simulate which makes it difficult to execute the entire test suite. Furthermore, it is often difficult to measure white-box coverage of test cases when employing such models. In addition, the historical data related to failures might not be available. Objective: The objective of the approach presented in this paper is to cost-effectively select test cases without making use of white-box coverage information or historical data related to fault detection. Method: We propose a cost-effective approach for test case selection that relies on black-box data related to inputs and outputs of the system. The approach defines in total six effectiveness measures and one cost measure followed by deriving in total 21 objective combinations and integrating them within Non-Dominated Sorting Genetic Algorithm-II (NSGA-II). The proposed six effectiveness metrics are specific to simulation models and are based on anti-patterns and similarity measures. Results: We empirically evaluated our approach with these 21 combinations using six case studies by employing mutation testing to assess the fault revealing capability. We compared our approach with Random Search (RS), two many-objective algorithm, as well as three white-box metrics. The results demonstrated that our approach managed to improve Random Search by up to around 28% in terms of the Hypervolume quality indicator. Similarly, black-box metrics-based test case selection also significantly outperformed those of white-box metrics. Conclusion: We demonstrate that test case selection is a non-trivial problem in the context of simulation models. We also show that the proposed effectiveness metrics performed significantly better than traditional white-box metrics. Thus, we show that black-box test selection approaches are appropriate to solve the test case selection problem within simulation models.

Wind Turbine Yaw Control Optimization and Its Impact on Performance

The optimization of wind energy conversion efficiency has been recently boosting the technology improvement and the scientific comprehension of wind turbines. In this context, the yawing behavior of wind turbines has become a key topic: the yaw control can actually be exploited for optimization at the level of single wind turbine and of wind farm (for example, through active control of wakes). On these grounds, this work is devoted to the study of the yaw control optimization on a 2 MW wind turbine. The upgrade is estimated by analysing the difference between the measured post-upgrade power and a data driven model of the power according to the pre-upgrade behavior. Particular attention has therefore been devoted to the formulation of a reliable model for the pre-upgrade power of the wind turbine of interest, as a function of the operation variables of all the nearby wind turbines in the wind farm: the high correlation between the possible covariates of the model indicates that Principal Component Regression (PCR) is an adequate choice. Using this method, the obtained result for the selected test case is that the yaw control optimization provides a 1% of annual energy production improvement. This result indicates that wind turbine control optimization can non-negligibly improve the efficiency of wind turbine technology.

Preliminary Service Life Estimation Model for MEP Components Using Case-Based Reasoning and Genetic Algorithm

In recent decades, building maintenance has been recognized as an important issue as the number of deteriorating buildings increases around the world. In densely populated cities, building maintenance is essential for ensuring sustainable living and safety for residents. Improper maintenance can not only cause enormous maintenance costs, but also negatively affect residents and their environment. As a first step, the service life of building components needs to be estimated in advance. Mechanical, electrical, and plumbing (MEP) components especially produce many maintenance-related problems compared to other components. In this research, a model was developed that applies the genetic algorithm (GA) and case-based reasoning (CBR) methodologies to estimating the service life of MEP components. The applicability of the model was tested by comparing the outputs of 20 randomly selected test cases with those of retrieved similar cases. The experimental results demonstrated that the overall similarity scores of the retrieved cases were over 90%, and the mean absolute error rate (MAER) of 10-NN was approximately 7.48%. This research contributes to the literature for maintenance management by not only presenting an approach to estimating the service life of building components, but also by helping convert the existing maintenance paradigm from reactive to proactive measures.

Spanwise wake development of a pivoted cylinder undergoing vortex-induced vibrations with elliptic trajectories

The wake development of a pivoted circular cylinder undergoing vortex-induced vibrations with elliptical trajectories is examined experimentally at a fixed Reynolds number of 3027 and mass ratio of 10.8. Simultaneous cylinder displacement measurements and time-resolved, two-component particle image velocimetry in multiple horizontal and vertical planes are used to quantify the structural response and wake development. The selected test cases pertain to $$U^*=U_0/f_\mathrm{n} D=5.48$$ and 7.08, and exhibit different orientations of elliptical cylinder trajectory, both with a clockwise direction of orbiting. Three-dimensional reconstructions of the phase-averaged wake velocity measurements reveal 2S shedding along the span of a stationary cylinder and hybrid shedding for the two vibrating cylinder cases, with planar wake topology transitioning from 2S to P+S to 2S for $$U^*=5.48$$ , and 2S to P+S for 7.08. The observed wake topologies show significant deviation from predictions based on the Morse and Williamson (J Fluids Struct 25(4):697–712, 2009) shedding map. Vortex identification and strength quantification are used to provide insight into vortex dynamics and to propose a model of the dislocations. Examination of the time averaged wake characteristics shows the formation length, wake half-width, and maximum velocity deficit exhibit distinct spanwise trends aligning with the regions associated with specific shedding regimes.

Search-based test case implantation for testing untested configurations

Abstract Context Modern large-scale software systems are highly configurable, and thus require a large number of test cases to be implemented and revised for testing a variety of system configurations. This makes testing highly configurable systems very expensive and time-consuming. Objective Driven by our industrial collaboration with a video conferencing company, we aim to automatically analyze and implant existing test cases (i.e., an original test suite) to test the untested configurations. Method We propose a search-based test case implantation approach (named as SBI) consisting of two key components: 1) Test case analyzer that statically analyzes each test case in the original test suite to obtain the program dependence graph for test case statements and 2) Test case implanter that uses multi-objective search to select suitable test cases for implantation using three operators, i.e., selection, crossover, and mutation (at the test suite level) and implants the selected test cases using a mutation operator at the test case level including three operations (i.e., addition, modification, and deletion). Results We empirically evaluated SBI with an industrial case study and an open source case study by comparing the implanted test suites produced by three variants of SBI with the original test suite using evaluation metrics such as statement coverage (SC), branch coverage (BC), and mutation score (MS). Results show that for both the case studies, the test suites implanted by the three variants of SBI performed significantly better than the original test suites. The best variant of SBI achieved on average 19.3% higher coverage of configuration variable values for both the case studies. Moreover, for the open source case study, the best variant of SBI managed to improve SC, BC, and MS with 5.0%, 7.9%, and 3.2%, respectively. Conclusion SBI can be applied to automatically implant a test suite with the aim of testing untested configurations and thus achieving higher configuration coverage.

Dynamic Partitioning and Additional Branch Coverage for Test Case Selection

Good test cases are those test case with high failure detecting capability. The effectiveness of testing is relied on the test case selection process. The earlier good test cases being selected, the higher the effectiveness of testing. Partitioning is one of the method to improve the effectiveness of test case selection. Dynamic Partitioning is method that has been proved empirically can improve the effectiveness of random selection. This method is proposed based on the intuition that a successful test case has high probability in detecting failure in the next execution. On the other hand, Addition Branch Coverage method has been proposed to improve the effectiveness of testing. It selects test case if it can add the covered branch during test execution. The empirical results show that the Additional Branch Coverage is the best method compare with othe coverage-based method. In this research, the combination of those two methods, Dynamic Partitioning and Additional Branch coverage is proposed with the hypothesis that this combination will combine the advantages of those two methods, and hence produce better method. The effectiveness of the proposed method has been investigated empirically by comparing it with the Random Testing, Dynamic Partitioning and Additional Branch Coverage. The Random Testing usually used as the benchmark in software testing research. The results show that the new method performed more effective that all other studied methods. The combination of Dynamic Partitioning and Additional Branch Coverage improve the effectiveness of test case selection in term of the number of executed test cases to detect all existing failures.

The implementation of ATDD and BDD from Testing Perspectives

Software testing is one of the stages in software development that aims to ensure that the software is built to meet the specifications. Selection of selective test cases has a great chance of finding failure. Black box testing approach is done based on the requirement specification where this approach does not pay attention to the program code but the specification of a software. This approach can be used for testing a software using ATDD (Acceptance Test Driven Development) and BDD (Behavior Driven Development) methods. ATDD is a method of building a software created based on agile principle, acceptance test created by customer, developer and tester. BDD is a growing agile development approach in recent years. Behavior Driven Development (BDD) is built on Test-Driven Development (TDD). This Final Project focuses on the application of ATDD and BDD methods to testing the final three projects of IT Del students tested using the Roboy Framework and Cucumber Framework to find out whether these two methods are effective and be able to improve the software development process. The results obtained are that the application of ATDD and BDD methods are effective and helping to remove the error as soon as possible.

SPH simulation of periodic wave breaking in the surf zone - A detailed fluid dynamic validation

The estimation of wave breaking and run-up on sloped beaches is a relevant issue in different coastal engineering applications. The present study stresses on the capabilities of a Smoothed Particle Hydrodynamics (SPH) solver, with optimal numerical and physical parameters, to accurately simulate the complex flow field in the surf zone and run-up region. Numerical results are compared with high quality experimental measurements of the local flow field in terms of instantaneous and phase averaged values. The selected test case regards the propagation and breaking of regular non-linear waves on a smooth impermeable plane slope. The comparison is based on a complete set of 128 consecutive non-linear regular waves. The level of accuracy of the numerical results and the ability of the model to reproduce the periodic flow in the surf-zone is provided. Current limitations and uncertainty sources are identified and discussed to guide future developments.

Optimal selection and release problem in software testing process: A continuous time stochastic control approach

This paper studies a joint selection of test cases and release problem for a software under test with predetermined classes of test cases and release time deadline. The software test manager can make three alternative choices dynamically during software testing progress before the deadline: continue testing and select a class of test cases, release the software, or scrap the software, with the objective of minimizing the cumulative testing cost plus penalty cost after releasing or scrapping the software. We formulate the problem as a continuous time stochastic control model and provide a mathematically rigorous method to establish the concavity of the optimal cost function. Based on this property, we are able to characterize that the optimal release policy has a threshold structure. Moreover, the thresholds are founded to be monotone in the residual time length in the case of homogeneous release cost. Besides, we put forward a method based on low convex envelope and discover that the optimal selection policy also has a threshold or other simple structure, if the running cost or the removal cost is the same for all classes. Finally, we present an approximation algorithm of computing the optimal cost function, by which some numerical examples are studied to justify our theoretical results and the robustness of our policy. We also conduct a case study to compare our dynamic selection and release testing policy with two other commonly used testing policies and find that our policy is the best in most instances.

Adaptive Partition Testing

Random testing and partition testing are two major families of software testing techniques. They have been compared both theoretically and empirically in numerous studies for decades, and it has been widely acknowledged that they have their own advantages and disadvantages and that their innate characteristics are fairly complementary to each other. Some work has been conducted to develop advanced testing techniques through the integration of random testing and partition testing, attempting to preserve the advantages of both while minimizing their disadvantages. In this paper, we propose a new testing approach, adaptive partition testing , where test cases are randomly selected from some partition whose probability of being selected is adaptively adjusted along the testing process. We particularly develop two algorithms, Markov-chain based adaptive partition testing and reward-punishment based adaptive partition testing , to implement the proposed approach. The former algorithm makes use of Markov matrix to dynamically adjust the probability of a partition to be selected for conducting tests; while the latter is based on a reward and punishment mechanism. We conduct empirical studies to evaluate the performance of the proposed algorithms using ten faulty versions of three large-scale open source programs. Our experimental results show that, compared with two baseline techniques, namely random partition testing (RPT) and dynamic random testing (DRT), our algorithms deliver higher fault-detection effectiveness with lower test case selection overhead. It is demonstrated that the proposed adaptive partition testing is an effective testing approach, taking advantages of both random testing and partition testing.