Set Of Test Cases Research Articles

Effective fault localization and context‐aware debugging for concurrent programs

SummaryConcurrent programs are difficult to debug because concurrency faults usually occur under specific inputs and thread interleavings. Fault localization techniques for sequential programs are often ineffective because the root causes of concurrency faults involve memory accesses across multiple threads rather than single statements. Previous research has proposed techniques to analyse passing and failing executions obtained from running a set of test cases for identifying faulty memory access patterns. However, stand‐alone access patterns do not provide enough contextual information, such as the path leading to the failure, for developers to understand the bug. We present an approach, Coadec, to automatically generate interthread control flow paths that can link memory access patterns that occurred most frequently in the failing executions to better diagnose concurrency bugs. Coadec consists of two phases. In the first phase, we use feature selection techniques from machine learning to localize suspicious memory access patterns based on failing and passing executions. The patterns with maximum feature diversity information can point to the most suspicious pattern. We then apply a data mining technique and identify the memory access patterns that occurred most frequently in the failing executions. Finally, Coadec identifies faulty program paths by connecting both the frequent patterns and the suspicious pattern. We also evaluate the effectiveness of fault localization using test suites generated from different test adequacy criteria. We introduce and have evaluated Coadec on 10 real‐world multithreaded Java applications. Results indicate that Coadec outperforms state‐of‐the‐art approaches for localizing concurrency faults and that Coadec's context debugging can help developers understand concurrency fault by inspecting a small percentage of code.

A simulation framework for rotorcraft ice accretion and shedding

In this work a novel three-dimensional simulation framework for modelling the safety critical problem of rotorcraft ice formation and shedding is presented. To enable an entirely three-dimensional framework, state-of-the-art numerical modelling techniques are adopted and the inter-dependency between the techniques is discussed. The numerical techniques introduced in this work include models to simulate the rotor flow-field, water droplet trajectories and impingement locations, phase change modelling during the ice accretion, and mesh deformation to account for the moving ice boundary. A set of benchmark test cases are initially used for preliminary validation of the icing framework. This allows for a comparison of the collection efficiency and ice shapes with high-quality experimental measurements. Icing wind tunnel tests conducted on the Spinning Rotor Blade (SRB-II) model rotor are used for an assessment of the numerical predictions specific to rotorcraft. Quantities used for comparisons include the ice thickness and shedding location. Numerical predictions are in good agreement with the measured data at all temperatures. Additionally, the outcome of influential parameters which directly impact rotor ice shapes are assessed. In particular, the model for the temperature profiles within the ice layer, and the centrifugally induced movement of the liquid film.

An Efficient Regression Test Suite Optimization Approach Using Hybrid Spider Monkey Optimization Algorithm

Regression testing validates the modified software and safeguards against the introduction of new errors during modification. A number of test suite optimization techniques relying on meta-heuristic techniques have been proposed to find the minimal set of test cases to execute for regression purposes. This paper proposes a hybrid spider monkey optimization based regression test suite optimization approach and empirically compares its performance with three other approaches based on bat search, ant colony, and cuckoo search. The authors conducted an empirical study with various subjects retrieved from software artifact infrastructure repository. Fault coverage and execution time of algorithm are used as fitness measures to meet the optimization criteria. Extensive experiments are conducted to evaluate the performance of the proposed approach with other search-based approaches under study using various statistical tests like m-way ANOVA and post hoc tests including odds ratio. Results indicate the superiority of the proposed approach in most of the cases and comparable in others.

Test case prioritization based on Artificial Fish School Algorithm

Software testing is considered as an essential and critical part of the software development process. To improve the efficiency of software testing, the test case prioritization (TCP) technique is usually used to preprocess the test case set, which is formulated as a single-objective or multiple-objective optimization problem and solved by swarm intelligence algorithms. In this paper, we adopted one of the state-of-art swarm intelligence algorithms — Artificial Fish School Algorithm to solve the TCP problem. Specifically, the coding method of artificial fish school was designed in combination with the test case set; the Average Percentage of Test-point Coverage and Effective Execution Time were selected to optimize the design of clustering behavior, foraging behavior and tail-chasing behavior of artificial fish school; the optimal solution was found by population iteration. Empirical evaluation was conducted to analyze the performance of the proposed method. Comparison experiments were also carried out, and the experimental results showed that in terms of both single-objective and multiple-objective, the Artificial Fish School Algorithm could better solve the TCP problems and improve the efficiency of software testing.

Simplified Event-Based Load Shedding Scheme for Frequency Stability in an Isolated Power System With High Renewable Penetration. El Hierro: A Case Study

This study presents a novel approach to calculate the load to be shed in El Hierro isolated power system in generation tripping events. The proposed shedding law is based on a linear regression model. The regression model is obtained by means of offline dynamic simulations in a set of representative test cases. The proposed shedding law has been compared to the under-frequency load shedding scheme currently utilized in El Hierro, considering three different implementation schemes. The results of this study demonstrate that the proposed shedding law can contribute to reduce the average amount of load shed in generation tripping events and meet the requirements of the system’s frequency with a moderate investment in smart power management devices.

Multiple-Searching Genetic Algorithm for Whole Test Suites

A test suite is a set of test cases that evaluate the quality of software. The aim of whole test suite generation is to create test cases with the highest coverage scores possible. This study investigated the efficiency of a multiple-searching genetic algorithm (MSGA) for whole test suite generation. In previous works, the MSGA has been effectively used in multicast routing of a network system and in the generation of test cases on individual coverage criteria for small- to medium-sized programs. The performance of the algorithms varies depending on the problem instances. In this experiment were generated whole test suites for complex programs. The MSGA was expanded in the EvoSuite test generation tool and compared with the available algorithms on EvoSuite in terms of the number of test cases, the number of statements, mutation score, and coverage score. All algorithms were evaluated on 14 problem instances with different corpus to satisfy multiple coverage criteria. The problem instances were Java open-source projects. Findings demonstrate that the MSGA generated test cases reached greater coverage scores and detected a larger number of faults in the test class when compared with the others.

Consistent Thermo-Capillarity and Thermal Boundary Conditions for Single-Phase Smoothed Particle Hydrodynamics.

A model for capillary phenomena including temperature-dependency and thermal boundary conditions is presented in the numerical framework of smoothed particle hydrodynamics (SPH). The model requires only a single fluid phase and is therefore computationally more efficient than surface tension schemes which need an explicit fluid-fluid or fluid-gas interface. The model makes use of a surface identification mechanism based on the SPH renormalization tensor. All relevant properties of the continuum surface force (CSF) based approach, i.e., the delta function, normal vector and curvature, are calculated in a consistent manner. The model is parametrized by physical material properties and is successfully validated by means of a large set of analytical test cases. The applicability of the proposed model to more complex scenarios is demonstrated.

A unified hybrid lattice-Boltzmann method for compressible flows: Bridging between pressure-based and density-based methods

A unified expression for high-speed compressible segregated consistent lattice Boltzmann methods, namely, pressure-based and improved density-based methods, is given. It is theoretically proved that in the absence of forcing terms, these approaches are strictly identical and can be recast in a unique form. An important result is that the difference with classical density-based methods lies in the addition of fourth-order term in the equilibrium function. It is also shown that forcing terms used to balance numerical errors in both original pressure-based and improved density-based methods can be written in a generalized way. A hybrid segregated efficient lattice-Boltzmann for compressible flow based on this unified model, equipped with a recursive regularization kernel, is proposed and successfully assessed on a wide set of test cases with and without shock waves.

Dealing with shallow groundwater contexts for the modelling of urban hydrology – A simplified approach to represent interactions between surface hydrology, groundwater and underground structures in hydrological models

Hydrological modelling can be a relevant tool to support sustainable planning decisions. However, urban hydrological models usually rely on simplified underground representation that may limit their applicability in shallow groundwater environments. This paper introduces a set of modules designed to describe the numerous interactions between subsurface processes and surface hydrology that may occur in such contexts. The modules provide a rather straightforward approach to simulate the influence of the groundwater on the surface as well as the influence of several underground structures (sewer systems, building basements, …) on subsurface storage and the volumes to be managed by the sewerage network. The modules are evaluated by comparison with reference models over a set of theoretical test cases and integrated within the URBS hydrological model. An application to a realistic hypothetical watershed is carried out to illustrate the benefits and the feasibility of these focussed development in existing urban hydrological models.

A Multi-Objective Approach for Test Suite Reduction During Testing of Web Applications

During the development and maintenance phases of evolving software, new test cases would be needed for the verification of the accuracy of the modifications as well as for new functionalities leading to an increase in the size of the test suite. Various related objectives are to be kept in mind while reducing the original test suite by removing redundancy and generating a practical representative set of the unique test cases, some of which may need to be maximized and the remaining ones minimized. This paper presents a multi-objective approach for the test suite reduction problem in which one objective is to be minimized and the remaining two maximized. In this study, experiments were performed on diverse versions of four web applications. Various state-of-the-art algorithms and their updated versions were compared with non-dominated sorting genetic algorithm-II (NSGA-II) for performance evaluation. Based on experimental findings, it was concluded that NSGA-II outperforms all other algorithms; moreover, the algorithm attempts to satisfy all the objectives without compromising coverage.

Path Planning on Grids: The Effect of Vertex Placement on Path Length

Video-game designers often tessellate continuous 2-dimensional terrain into a grid of blocked and unblocked square cells. The three main ways to calculate short paths on such a grid are to determine truly shortest paths, shortest vertex paths and shortest grid paths, listed here in decreasing order of computation time and increasing order of resulting path length. We show that, for both vertex and grid paths on both 4-neighbor and 8-neighbor grids, placing vertices at cell corners rather than at cell centers tends to result in shorter paths. We quantify the advantage of cell corners over cell centers theoretically with tight worst-case bounds on the ratios of path lengths, and empirically on a large set of benchmark test cases. We also quantify the advantage of 8-neighbor grids over 4-neighbor grids.

Efficient oscillation detection for verification of mechatronic closed-loop systems using search-based testing

In mechatronic system design, the systematic early-stage verification in simulation environments is a crucial aspect. In particular, the development of highly sophisticated and well performing controllers requires complementary testing by extensive simulations. The assessment and evaluation of the control strategy and its tuning is ideally carried out by specialists and is usually very time consuming. Due to time constraints, often only the nominal configuration and a limited number of other configurations are considered in the simulation-based analysis of the closed-loop system. To cover a more comprehensive set of test cases and speed up the analysis, assistive and automated tools are essential. This paper proposes a method to systematically search for oscillations in closed-loop mechatronic systems, which, in most cases, indicate a poor control performance or even an unstable system behavior. Although oscillation detection methods are very well established in the process industry, these methods do not directly apply to mechatronic systems where frequent changes of the operating points and tracking of fast trajectories have to be taken into account. A recently proposed method which can cope with these challenges uses the Empirical Mode Decomposition (EMD) combined with an assessment of the decomposed modes. This is the basis for the construction of a normalized and continuous oscillation index, which provides a suitable measure for the oscillation strength. The performance of the selected EMD approach compared to other state-of-the-art concepts is illustrated by measurement data of a hydraulic valve. The introduced oscillation index serves as an objective function for a Search-based Testing (SbT) approach to systematically select those parameter combinations which yield a worst-case scenario for the closed-loop system. The high potential of the combination of SbT with an oscillation-based objective function for the automated verification of closed-loop mechatronic systems is demonstrated for the example of a goods lift.

Multifactor Algorithm for Test Case Selection and Ordering

Regression testing being expensive, requires optimization notion. Typically, the optimization of test cases results in selecting a reduced set or subset of test cases or prioritizing the test cases to detect potential faults at an earlier phase. Many former studies revealed the heuristic-dependent mechanism to attain optimality while reducing or prioritizing test cases. Nevertheless, those studies were deprived of systematic procedures to manage tied test cases issue. Moreover, evolutionary algorithms such as the genetic process often help in depleting test cases, together with a concurrent decrease in computational runtime. However, when examining the fault detection capacity along with other parameters, is required, the method falls short. The current research is motivated by this concept and proposes a multifactor algorithm incorporated with genetic operators and powerful features. A factor-based prioritizer is introduced for proper handling of tied test cases that emerged while implementing re-ordering. Besides this, a Cost-based Fine Tuner (CFT) is embedded in the study to reveal the stable test cases for processing. The effectiveness of the outcome procured through the proposed minimization approach is anatomized and compared with a specific heuristic method (rule-based) and standard genetic methodology. Intra-validation for the result achieved from the reduction procedure is performed graphically. This study contrasts randomly generated sequences with procured re-ordered test sequence for over '10' benchmark codes for the proposed prioritization scheme. Experimental analysis divulged that the proposed system significantly managed to achieve a reduction of 35-40% in testing effort by identifying and executing stable and coverage efficacious test cases at an earlier phase.

MPC Tracking Controller Parameters Impacts in Roundabouts

The work is proposed to design a controller of the form known as the roundabout scenario trajectory tracking problem. The road condition is a four-leg, single-lane roundabout; the reference path is given. Due to the decision choice of exits, the MPC tracking controller is used to test the effect of weight parameter and target speed on the performance of the tracking controller. Two sets of test cases are proposed to make the experimental comparison, see the relationship between the control parameters and road conditions (different curvature path), and also see how the weight parameters Q and R and sample time affect the tracking performance. Our work, MPC controller utilization in a roundabout, plays an essential role with the increasing autonomy of vehicles.

Detection of multiple structural damages from drive point and cross electro-mechanical impedance signatures

A damage detection scheme using the electro-mechanical impedance based signatures for detection of multiple damages is proposed. A beam and a stiffened plate are considered as two example structures in this study. Cracks of 1 mm width and varying depth have been considered as damages for the beam structure. On the other hand, disbonds between the plate and stiffener along the stiffener length have been considered as damage in the stiffened plate structure. Drive point and cross conductance at two piezoelectric patch transducers in a chosen frequency range have been obtained using the finite element based software ANSYS. Result of ANSYS has been validated through experimental result. Root mean square deviation (RMSD) and the correlation coefficient (CC) of the conductance of the structure from the baseline conductance have been used as damage features. An artificial neural network with one hidden layer has been trained considering the damage features as input and crack depths/disbond lengths at different locations as output. Accuracy of the trained network in detecting crack depths/disbonds has been observed through a set of test cases.

A Large Dataset With a New Framework for Abandoned Object Detection in Complex Scenarios

Video surveillance is a crucial part of public safety and security systems. State-of-the-art techniques either use back-tracing for owner identification or there is no provision of reporting ownership of the abandoned objects. We provide a novel object association technique which very elegantly tackles this issue. We propose a convolution neural network based framework for abandoned object localization and owner identification in video surveillance systems that performs exceptionally well on publicly available datasets and our newly developed dataset. For testing the usability and robustness of these systems, an extensive set of test cases is a prerequisite, to this end we provide an elaborate surveillance dataset covering many complex cases which are not available in existing datasets. Also, the developed dataset has been tested with various state-of-the-art techniques, showing the complexity and challenging nature of the cases. Results obtained from the proposed framework are highly motivating as it provides event detection along with the time of abandonment and ownership information, which can help further investigation and prevent unwanted incidents.

The numerical implementation of land models: Problem formulation and laugh tests

AbstractThe intent of this paper is to encourage improved numerical implementation of land models. Our contributions in this paper are two-fold. First, we present a unified framework to formulate and implement land model equations. We separate the representation of physical processes from their numerical solution, enabling the use of established robust numerical methods to solve the model equations. Second, we introduce a set of synthetic test cases (the laugh tests) to evaluate the numerical implementation of land models. The test cases include storage and transmission of water in soils, lateral sub-surface flow, coupled hydrological and thermodynamic processes in snow, and cryosuction processes in soil. We consider synthetic test cases as “laugh tests” for land models because they provide the most rudimentary test of model capabilities. The laugh tests presented in this paper are all solved with the Structure for Unifying Multiple Modeling Alternatives model (SUMMA) implemented using the SUite of Nonlinear and DIfferential/Algebraic equation Solvers (SUNDIALS). The numerical simulations from SUMMA/SUNDIALS are compared against (1) solutions to the synthetic test cases from other models documented in the peer-reviewed literature; (2) analytical solutions; and (3) observations made in laboratory experiments. In all cases, the numerical simulations are similar to the benchmarks, building confidence in the numerical model implementation. We posit that some land models may have difficulty in solving these benchmark problems. Dedicating more effort to solving synthetic test cases is critical in order to build confidence in the numerical implementation of land models.

Automated test case generation based on differential evolution with node branch archive

Automatic test case generation (ATCG) is the active research topic in software testing engineering, which can greatly reduce the cost of software testing. In automated test case generation for path coverage (ATCG-PC) problem, since the relationship between test cases and paths is unknown, there are many redundant test cases in the test case set to meet the path coverage criteria. In many previous studies on search-based algorithms for ATCG-PC, researchers have focused on improving the search-based algorithms itself or designing a more suitable fitness function according to the coverage criteria. However, the relationship between test cases and paths can help search-based algorithms cover more paths. The values of some specific test case dimensions change, and offspring individuals may cover different paths. Inspired by this, we proposed a node branch archive strategy, which can record the relationship between node branch direction and the value of test case variables, and cover more paths through this driven search-based algorithms. The experimental results show that compared with other state-of-the-art algorithms, the differential evolution with node branch archive (NBAr-DE) can significantly reduce the number of redundant test cases.

Optimal Pitch Angle Strategy for Energy Maximization in Offshore Wind Farms Considering Gaussian Wake Model

This paper presents a new approach based on the optimization of the blade pitching strategy of offshore wind turbines in order to maximize the global energy output considering the Gaussian wake model and including the effect of added turbulence. A genetic algorithm is proposed as an optimization tool in the process of finding the optimal setting of the wind turbines, which aims to determine the individual pitch of each turbine so that the overall losses due to the wake effect are minimised. The integration of the Gaussian model, including the added turbulence effect, for the evaluation of the wakes provides a step forward in the development of strategies for optimal operation of offshore wind farms, as it is one of the state-of-the-art analytical wake models that allow the evaluation of the energy output of the project in a more reliable way. The proposed methodology has been tested through the execution of a set of test cases that show the ability of the proposed tool to maximize the energy production of offshore wind farms, as well as highlights the importance of considering the effect of added turbulence in the evaluation of the wake.

An expanded benchmark for antibody-antigen docking and affinity prediction reveals insights into antibody recognition determinants

Accurate predictive modeling of antibody-antigen complex structures and structure-based antibody design remain major challenges in computational biology, with implications for biotherapeutics, immunity, and vaccines. Through a systematic search for high-resolution structures of antibody-antigen complexes and unbound antibody and antigen structures, in conjunction with identification of experimentally determined binding affinities, we have assembled a non-redundant set of test cases for antibody-antigen docking and affinity prediction. This benchmark more than doubles the number of antibody-antigen complexes and corresponding affinities available in our previous benchmarks, providing an unprecedented view of the determinants of antibody recognition and insights into molecular flexibility. Initial assessments of docking and affinity prediction tools highlight the challenges posed by this diverse set of cases, which includes camelid nanobodies, therapeutic monoclonal antibodies, and broadly neutralizing antibodies targeting viral glycoproteins. This dataset will enable development of advanced predictive modeling and design methods for this therapeutically relevant class of protein-protein interactions.