Test Case Selection Research Articles

Evolving software trace links between requirements and source code

Traceability provides support for diverse software engineering activities including safety analysis, compliance verification, test-case selection, and impact prediction. However, in practice, there...

Investigation of magnetic flux transport and shock formation in a staged Z-pinch

Target preheating is an integral component of magnetized inertial fusion in reducing convergence ratio. In the staged Z-pinch concept, it is achieved via one or more shocks. Previous work [Narkis et al., Phys. Plasmas 23, 122706 (2016)] found that shock formation in the target occurred earlier in higher-Z liners due to faster flux transport to the target/liner interface. However, a corresponding increase in magnitude of magnetic pressure was not observed, and target implosion velocity (and therefore shock strength) remained unchanged. To investigate other means of increasing the magnitude of transported flux, a Korteweg-de Vries-Burgers equation from the 1-D single-fluid, resistive magnetohydrodynamic equations is obtained. Solutions to the nondispersive (i.e., Burgers) equation depend on nondimensional coefficients, whose dependence on liner density, temperature, etc., suggests an increase in target implosion velocity, and therefore shock strength, can be obtained by tailoring the mass of a single-liner gas puff to a double-liner configuration. In the selected test cases of 1-D simulated implosions of krypton on deuterium, the peak Mach number increased from ∼5 to ∼8. While a notable increase was seen, Mach numbers exceeding 10 (implosion velocities exceeding ∼25 cm/μs) are necessary for adequate shock preheating.

An assessment of operational coverage as both an adequacy and a selection criterion for operational profile based testing

While the relation between code coverage measures and fault detection is actively studied, only few works have investigated the correlation between measures of coverage and of reliability. In this work, we introduce a novel approach to measuring code coverage, called the operational coverage, that takes into account how much the program’s entities are exercised so to reflect the profile of usage into the measure of coverage. Operational coverage is proposed as (i) an adequacy criterion, i.e., to assess the thoroughness of a black box test suite derived from the operational profile, and as (ii) a selection criterion, i.e., to select test cases for operational profile-based testing. Our empirical evaluation showed that operational coverage is better correlated than traditional coverage with the probability that the next test case derived according to the user’s profile will not fail. This result suggests that our approach could provide a good stopping rule for operational profile-based testing. With respect to test case selection, our investigations revealed that operational coverage outperformed the traditional one in terms of test suite size and fault detection capability when we look at the average results.

Controller-Centric Combinatorial Wrap-Around Interaction Testing to Evaluate a Stateful PCE-Based Transport Network Architecture

The objective of this paper is to develop a controller-centric combinatorial wrap-around interaction testing methodology for a stateful path computation element (PCE)-based transport network architecture. By exploiting the internal contexts of the controller-centric network in conjunction with combinatorial, wrap-around, and interaction testing methodologies, the proposed testing methodology helps test engineers build a highly configurable testing environment, select high-quality test cases to obtain the best possible combination of interactions, and prune fault cases or useless cases from all possible test cases throughout the entire development process as gray-box testing. The experimental results verify that the combined testing methodology effectively evaluates the controller-centric network for all testing processes in a completely controlled environment.

Requirement-based Selection Model for the Expansion of Regression Test Selection

Issue of maintaining software is to consider which test cases should be kept for the next modification where the size of test suite gets bigger. This makes performance of software development pull out. The objective of proposing requirement-based test case selection model is to improve ability of regression test selection, in particular, to moderate the size of test suite of the modified program, which gets larger after modification regarding the need of specific requirements, including preparing higher ability of removing faults. It comprises five main algorithms, which are finding reused test case, classifying, revising, deleting, and selecting the appropriate test cases. This paper uses six programs run on different four comparative studies, which are selectall, random, and regression test selection. It gives smaller size than the traditional techniques about 49.78% in average. Besides, it offers percent fixing faults that is higher than select-all, random, and regression test selection algorithm as around 0.06–1.32%.

VMS-ROT: A New Module of the Virtual Multifrequency Spectrometer for Simulation, Interpretation, and Fitting of Rotational Spectra.

The Virtual Multifrequency Spectrometer (VMS) is a tool that aims at integrating a wide range of computational and experimental spectroscopic techniques with the final goal of disclosing the static and dynamic physical–chemical properties “hidden” in molecular spectra. VMS is composed of two parts, namely, VMS-Comp, which provides access to the latest developments in the field of computational spectroscopy, and VMS-Draw, which provides a powerful graphical user interface (GUI) for an intuitive interpretation of theoretical outcomes and a direct comparison to experiment. In the present work, we introduce VMS-ROT, a new module of VMS that has been specifically designed to deal with rotational spectroscopy. This module offers an integrated environment for the analysis of rotational spectra: from the assignment of spectral transitions to the refinement of spectroscopic parameters and the simulation of the spectrum. While bridging theoretical and experimental rotational spectroscopy, VMS-ROT is strongly integrated with quantum-chemical calculations, and it is composed of four independent, yet interacting units: (1) the computational engine for the calculation of the spectroscopic parameters that are employed as a starting point for guiding experiments and for the spectral interpretation, (2) the fitting-prediction engine for the refinement of the molecular parameters on the basis of the assigned transitions and the prediction of the rotational spectrum of the target molecule, (3) the GUI module that offers a powerful set of tools for a vis-à-vis comparison between experimental and simulated spectra, and (4) the new assignment tool for the assignment of experimental transitions in terms of quantum numbers upon comparison with the simulated ones. The implementation and the main features of VMS-ROT are presented, and the software is validated by means of selected test cases ranging from isolated molecules of different sizes to molecular complexes. VMS-ROT therefore offers an integrated environment for the analysis of the rotational spectra, with the innovative perspective of an intimate connection to quantum-chemical calculations that can be exploited at different levels of refinement, as an invaluable support and complement for experimental studies.

A Developed Numerical Method for Turbulent Unsteady Fluid Flow in Two-Phase Systems with Moving Interface

This paper presents the development of an accurate and robust numerical modeling of instability of an interface separating two-phase system, such as liquid–gas and/or solid–gas systems. The instability of the interface can be refereed to the buoyancy and capillary effects in liquid–gas system. The governing unsteady Navier–Stokes along with the stress balance and kinematic conditions at the interface are solved separately in each fluid using the finite-volume approach for the liquid–gas system and the Hamilton–Jacobi equation for the solid–gas phase. The developed numerical model represents the surface and the body forces as boundary value conditions on the interface. The adapted approaches enable accurate modeling of fluid flows driven by either body or surface forces. The moving interface is tracked and captured using the level set function that initially defined for both fluids in the computational domain. To asses the developed numerical model and its versatility, a selection of different unsteady test cases including oscillation of a capillary wave, sloshing in a rectangular tank, the broken-dam problem involving different density fluids, simulation of air/water flow, and finally the moving interface between the solid and gas phases of solid rocket propellant combustion were examined. The latter case model allowed for the complete coupling between the gas-phase physics, the condensed-phase physics, and the unsteady nonuniform regression of either liquid or the propellant solid surfaces. The propagation of the unsteady nonplanar regression surface is described, using the Essentially-Non-Oscillatory (ENO) scheme with the aid of the level set strategy. The computational results demonstrate a remarkable capability of the developed numerical model to predict the dynamical characteristics of the liquid–gas and solid–gas flows, which is of great importance in many civilian and military industrial and engineering applications.

SPH with dynamical smoothing length adjustment based on the local flow kinematics

Due to the Lagrangian nature of Smoothed Particle Hydrodynamics (SPH), the adaptive resolution remains a challenging task. In this work, we first analyse the influence of the simulation parameters and the smoothing length on solution accuracy, in particular in high strain regions. Based on this analysis we develop a novel approach to dynamically adjust the kernel range for each SPH particle separately, accounting for the local flow kinematics. We use the Okubo–Weiss parameter that distinguishes the strain and vorticity dominated regions in the flow domain. The proposed development is relatively simple and implies only a moderate computational overhead. We validate the modified SPH algorithm for a selection of two-dimensional test cases: the Taylor–Green flow, the vortex spin-down, the lid-driven cavity and the dam-break flow against a sharp-edged obstacle. The simulation results show good agreement with the reference data and improvement of the long-term accuracy for unsteady flows. For the lid-driven cavity case, the proposed dynamical adjustment remedies the problem of tensile instability (particle clustering).

Improved Time Performance of Adaptive Random Partition Software Testing by Applying Clustering Algorithms

Software testing is generally accepted technique for evaluating and taming software quality. Random testing (RT) is a major software testing strategy and a basic testing technique randomly generates test cases from the set of all possible program inputs. The simplicity of this testing makes it likely the most efficient testing strategy with respect to the time required for test case selection. Though very simple, RT is still considered as one of the state-of-the-art testing techniques, along with other more complicated and systematic testing methods. Its efficacy is notified to be less while considering its capacity of defect detection. This has been proven to pertinently overcome by Adaptive Testing (AT), on the other hand the methodology of AT is comprised of intricate complexity and high computational cost as its main constituents. Adaptive random testing (ART) is one major approach for enhancing RT. Another category of testing techniques is partition testing, which involves dividing the input domain up into a fixed number of disjoint partitions, and choosing test cases from within each partition. Partition testing has powerful, intuitive appeal, and analytical results show that even simple partitioning schemes may be more effective in fault detection than random testing. The existing hybrid approach is a combination of AT and RPT which is called as ARPT strategy, which enhances the AT. The objective of this proposed research is to improve random partition in ARPT strategies by utilizing clustering algorithms like Expectation Maximization (EM) algorithm and Nonnegative Matrix Factorization (NMF) clustering algorithms and the Self-organizing map (SOM) which can be efficiently utilized for partition. In this way random partitioning is improved to reduce the time conception and complexity in ARPT testing strategies.

The impacts of techniques, programs and tests on automated program repair: An empirical study

Manual program repair is notoriously tedious, error-prone, and costly, especially for the modern large-scale projects. Automated program repair can automatically find program patches without much human intervention, greatly reducing the burden of developers as well as accelerating software delivery. Therefore, much research effort has been dedicated to design powerful program repair techniques. To date, although various program repair techniques have been proposed, to our knowledge, there lacks extensive study on the impacts of repair techniques, subject programs, and test suites on the repair effectiveness and efficiency. In this paper, we perform such an extensive study on repairing 180 seeded and real faults from 17 small to large sized programs. We study the impacts of five representative automated program repair techniques, including GenProg, RSRepair, Brute-force-based technique, AE and Kali, on the repair results. We further investigate the impacts of different subject programs and test suites on effectiveness and efficiency of program repair techniques. Our study demonstrates a number of interesting findings: Brute-force-based technique generates the maximum number of patches but is also the most costly technique, while Kali is the most efficient and has medium effectiveness among the studied techniques; techniques that work well with small programs become too costly or ineffective when applied to large sized programs; since tool-reported patches may overfit the selected test cases, we calculate the false positive rates and find that the influence of failed test cases is much larger than that of passed test cases; finally, surprisingly, all the studied techniques except RSRepair can find more than 80% of successful patches within the first 50% of search space.

Software Product Line Testing Based on Feature Model Mutation

The Feature Model (FM) is a fundamental artifact of the Software Product Line (SPL) engineering, used to represent commonalities and variabilities, and also to derive products for testing. However, the test of all features combinations (products) is not always possible in practice. Due to the growing complexity of the applications, only a subset of products is usually selected. The selection is generally based on combinatorial testing, to test features interactions. This kind of selection does not consider different classes of faults that can be present in the FM. The application of a fault-based approach, such as mutation-based testing, can increase the probability of finding faults and the confidence that the SPL products match the requirements. Considering that, this paper introduces a mutation approach to select products for the feature testing of SPLs. The approach can be used similarly to a test criterion in the generation and assessment of test cases. It includes (i) a set of mutation operators, introduced to describe typical faults associated to the feature management and to the FM; and (ii) a testing process to apply the operators. Experimental results show the applicability of the approach. The selected test case sets are capable to reveal other kind of faults, not revealed in the pairwise testing.

Effective Regression Test Case Selection

Regression test case selection techniques attempt to increase the testing effectiveness based on the measurement capabilities, such as cost, coverage, and fault detection. This systematic literature review presents state-of-the-art research in effective regression test case selection techniques. We examined 47 empirical studies published between 2007 and 2015. The selected studies are categorized according to the selection procedure, empirical study design, and adequacy criteria with respect to their effectiveness measurement capability and methods used to measure the validity of these results. The results showed that mining and learning-based regression test case selection was reported in 39% of the studies, unit level testing was reported in 18% of the studies, and object-oriented environment (Java) was used in 26% of the studies. Structural faults, the most common target, was used in 55% of the studies. Overall, only 39% of the studies conducted followed experimental guidelines and are reproducible. There are 7 different cost measures, 13 different coverage types, and 5 fault-detection metrics reported in these studies. It is also observed that 70% of the studies being analyzed used cost as the effectiveness measure compared to 31% that used fault-detection capability and 16% that used coverage.

A Method for Prioritizing Integration Testing in Software Product Lines Based on Feature Model

Testing activities for software product lines should be different from that of single software systems, due to significant differences between software product line engineering and single software system development. The cost of testing in software product line is generally higher compared with single software systems; therefore, there should exist a certain balance between cost, quality of final products, and the time of performing testing activities. As decreasing testing cost is an important challenge in software product line integration testing, the contribution of this paper is in introducing a method for early integration testing in software product lines based on feature model (FM) by prioritizing test cases in order to decrease integration testing costs in SPLs. In this method, we focus on reusing domain engineering artifacts and prioritized selection and execution of integration test cases. It also uses separation of concerns and pruning techniques on FMs to help prioritize the test cases. The method shows to be promising when applied to some case studies in the sense that it decreases the costs of performing integration test by about 82% and also detects about 44% of integration faults in domain engineering.

Thermochemical ablation modeling forward uncertainty analysis—Part I: Numerical methods and effect of model parameters

Next generation spacecraft will bring back heavier payloads from explored planets. Advance in the modeling of the thermo-chemical ablation of carbon-based thermal protection system materials is fundamental to improve the design capabilities of these vehicles. Computational fluid dynamic approaches are extensively used to model the gas-surface interaction phenomena over ablative materials. The advantage of such kind of approaches is the accurate description of the aerothermal environment obtained through the full resolution of the mechanical, thermal, and chemical boundary layers that develop over an ablative surface when exposed to a high-enthalpy flow. This paper is devoted to the assessment of the uncertainties of such kind of thermo-chemical ablation model and to study their effect on the model final outcomes. A sphere of non-pyrolyzing carbon-based material, exposed to conditions similar to those of a typical plasma wind tunnel test, is the selected test case for the analysis. Two forward non-intrusive uncertainty quantification techniques are used to analyze the influence of the defined set of uncertain parameters on the estimate of steady-state mass blowing flux and surface temperature. Our results show that for the selected conditions, and uncertainty ranges, the surface nitridation reaction probability has the strongest impact on the model outcomes.

1D and Q2D thermal resistance analysis of micro channel structure and flat plate heat pipe

Heat pipes and two-phase heat transfer devices are widely used in electronics cooling, and the thermal resistance is a key issue to ensure the system performance and reliability. In this study, the traditional one dimensional (1D) and quasi-two dimensional (Q2D) methods were utilized to analyze the thermal resistance of the previous boiling experiments of evaporator tests with silicon (Si) and copper (Cu) micro channel structures as well as the existing flat-plate heat pipe tests with aluminum micro channels as wick structures. The temperature distributions and variations under different test conditions were collected, and the 1D and Q2D methods were applied to calculate the overall thermal resistance for evaporator tests and heat pipe tests and compared with the experimental data. The results showed that the Q2D method can predict the overall thermal resistance for the evaporator region and the whole heat pipe with a higher accuracy because the spreading resistance was significant and should be considered under small heater area (hot spot) conditions. Detailed distributions of thermal resistance and the calculated errors of each test sample under different heat loads were presented and compared. For present tests, the Q2D method can reach a lower average error (about 10%) and is recommended for calculations. Furthermore, the spreading resistance of Q2D method is further applied to calculate the selected test cases to investigate the wall thickness effects. The spreading resistance of the selected evaporator tests and heat pipe tests may decrease and increase, respectively, as wall thickness increases due to different evaporative heat transfer conditions. The present results demonstrate the applicability of the Q2D methods for both evaporator and heat pipe tests, and the present analyses can be a reference for future thermal management and electronic cooling designs.

Power Ramp Limitation Capabilities of Large PV Power Plants With Active Power Reserves

Power ramp limitation (PRL) is likely to become a requirement for large-scale photovoltaic power plants (LPVPPs) in order to allow the increase of PV penetration levels. Especially in islands with reduced inertia capability, this problem is more stringent: high-power ramp can be caused by either fast irradiance changes or other participant generators for example wind power, or loads. In order to compensate for the power mismatch, LPVPPs must use active power reserve (APR), by either curtailment or auxiliary storage. The paper proposes a PRL control structure for dynamic APR sizing and deployment. The selected test case is the power system of Puerto Rico (PREPA), modeled using the modified IEEE 12 bus benchmark system, with different levels of PV penetration. It is shown that LPVPP with PRL can effectively reduce the ramping rate of the participating generators. Considering that the large area of LPVPPs acts as a filter against fast irradiance changes, the study also reveals the required plant size for which an auxiliary storage is no longer needed in order to comply with PRL requirements—an important economical aspect.

Depletion chain compression method via quantitative significance analysis

The depletion calculation in reactor physics requires depletion chain data. However, the depletion chain that originates from evaluated nuclear data library is unnecessarily detailed for assembly and micro-depletion calculation, while conventional depletion chain compression methods are semi-empirical with limited application scope and accuracy. In this paper, a method that compresses depletion chain based on quantitative significance analysis of each nuclide and reaction channel is proposed. The significance analysis uses fine depletion chain computation results of representative problems as data source, and it is carried out by evaluating the influence on neutron absorption, production and number densities of target nuclides, induced by each basic unit compression. The method is applied to depletion chain compression in the context of PWR assembly calculation. Fine and compressed depletion chain computations of selected test cases are performed. The comparison among obtained numerical results show that, while preserving accuracy requirements, the proposed depletion chain compression method is capable of significantly reducing the complexity of the depletion chain, and the demanded storage and time savings are achieved consequently.

Using Global Constraints to Automate Regression Testing

Nowadays, any communicating or autonomous systems rely on high-quality software-based components. To ensure a sufficient level of quality, these components must be thoroughly verified before being released and being deployed in operational settings. Regression testing is a crucial verification process that executes any new release of a software-based component against previous versions of the component, with existing test cases. However, the selection of test cases in regression testing is challenging as the time available for testing is limited and some selection criteria must be respected. This problem, coined as Test Suite Reduction (TSR), is usually addressed by validation engineers through manual analysis or by using approximation techniques. Even if the underlying optimization problem is untractable in theory, solving it in practice is crucial when there are pressing needs to release high-quality components while at the same time reducing the time-to-market of new software releases. In this paper, we address the TSR problem with sound Artificial intelligence techniques such as Constraint Programming (CP) and global constraints. By associating each test case a cost-value aggregating distinct criteria, such as execution time, priority or importance due to the error-proneness of each test case, we propose several constraint optimization models to find a subset of test cases covering all the test requirements and optimizing the overall cost of selected test cases. Our models are based on a combination of NVALUE, GLOBALCARDINALITY, and SCALAR_PRODUCT, three well-known global constraints that can faithfully encode the coverage relation between test cases and test requirements. Our contribution includes the reuse of existing preprocessing rules to simplify the problem before solving it and the design of structure-aware heuristics, which take into account the notion of costs, associated with test cases. The work presented in this paper has been motivated by an industrial application in the communication domain. Our overall goal is to develop a constraint-based approach of test suite reduction that can be deployed to test a complete product line of conferencing systems in continuous delivery mode. By implementing this approach in a software prototype tool and experimentally evaluated it on both randomly generated instances and industrial instances, we hope to foster a quick adoption of the technology.

GW100: A Plane Wave Perspective for Small Molecules.

In a recent work, van Setten and co-workers have presented a carefully converged G0W0 study of 100 closed shell molecules [ J. Chem. Theory Comput. 2015 , 11 , 5665 - 5687 ]. For two different codes they found excellent agreement to within a few 10 meV if identical Gaussian basis sets were used. We inspect the same set of molecules using the projector augmented wave method and the Vienna ab initio simulation package (VASP). For the ionization potential, the basis set extrapolated plane wave results agree very well with the Gaussian basis sets, often reaching better than 50 meV agreement. In order to achieve this agreement, we correct for finite basis set errors as well as errors introduced by periodically repeated images. For positive electron affinities differences between Gaussian basis sets and VASP are slightly larger. We attribute this to larger basis set extrapolation errors for the Gaussian basis sets. For quasi particle (QP) resonances above the vacuum level, differences between VASP and Gaussian basis sets are, however, found to be substantial. This is tentatively explained by insufficient basis set convergence of the Gaussian type orbital calculations as exemplified for selected test cases.

Regression Testing of Database Applications Under an Incremental Software Development Setting

Software regression testing verifies previous features on a software product when it is modified or new features are added to it. Because of the nature of regression testing it is a costly process. Different approaches have been proposed to reduce the costs of this activity, among which are: minimization, prioritization, and selection of test cases. Recently, soft computing techniques, such as data mining, machine learning, and others have been used to make regression testing more efficient and effective. Currently, in different contexts, to a greater or lesser extent, software products have access to databases (DBs). Given this situation, it is necessary to consider regression testing also for software products such as information systems that are usually integrated with or connected to DBs. In this paper, we present a selection regression testing approach that utilizes a combination of unsupervised clustering with random values, unit tests, and the DB schema to determine the test cases related to modifications or new features added to software products connected to DBs. Our proposed approach is empirically evaluated with two database software applications in a production context. Effectiveness metrics, such as test suite reduction, fault detection capability, recall, precision, and the F-measure are examined. Our results suggest that the proposed approach is enough effective with the resulting clusters of test cases.