Test Case Selection Research Articles

ARMOR: A Recompilation and Instrumentation-Free Monitoring Architecture for Detecting Memory Exploits

Software written in programming languages that permit manual memory management, such as C and C++, are often littered with exploitable memory errors. These memory bugs enable attackers to leak sensitive information, hijack program control flow, or otherwise compromise the system and are a critical concern for computer security. Many runtime monitoring and protection approaches have been proposed to detect memory errors in C and C++ applications, however, they require source code recompilation or binary instrumentation, creating compatibility challenges for applications using proprietary or closed source code, libraries, or plug-ins. This paper introduces a new approach for detecting heap memory errors that does not require applications to be recompiled or instrumented. We show how to leverage the calling convention of a processor to track all dynamic memory allocations made by an application during runtime. We also present a transparent tracking and caching architecture to efficiently verify program heap memory accesses. Performance simulations of our architecture using SPEC benchmarks and real-world application workloads show our architecture achieves hit rates over 95 percent for a 256-entry cache, resulting in only 2.9 percent runtime overhead. Security analysis using a software prototype shows our architecture detects 98 percent of heap memory errors from selected test cases in the Juliet Test Suite and real-world exploits.

Test case selection using multi-objective Evolutionary Algorithms

International Journal of Computer Sciences and Engineering (A UGC Approved and indexed with DOI, ICI and Approved, DPI Digital Library) is one of the leading and growing open access, peer-reviewed, monthly, and scientific research journal for scientists, engineers, research scholars, and academicians, which gains a foothold in Asia and opens to the world, aims to publish original, theoretical and practical advances in Computer Science,Information Technology, Engineering (Software, Mechanical, Civil, Electronics & Electrical), and all interdisciplinary streams of Computing Sciences. It intends to disseminate original, scientific, theoretical or applied research in the field of Computer Sciences and allied fields. It provides a platform for publishing results and research with a strong empirical component. It aims to bridge the significant gap between research and practice by promoting the publication of original, novel, industry-relevant research.

HTTCS: Hybridization Technique for Test Case Selection

One problem found within the process of software maintenance is that the size of the selected test cases is large. This causes the ability of the whole process of software-development life cycle to drop. Particularly, it may be time consuming and cause delays, and the cost may be expensive. The selection of test cases for software maintenance depends more on the criticality of fixing bugs than the criticality of avoiding programming errors. Therefore, selection methods are proposed, such as test-all, random, and regression selection. This includes Technique for Test Case Selection (TTCS) and the improvement of Test Case Selection (TCS). These techniques can provide better results, in particular, giving smaller sizes, reduction rates, and % problem-solving than traditional techniques. However, this paper proposes a new model, which is a combination of using the process of determining an appropriate number of selected test cases regarding TTCS, and TCS with testing-based selection, named the Hybridization Technique for Test Case Selection (HTTCS). Obviously, HTTCS can reduce the size of the selected test cases by about 96.86 - 98.83 %, which is better than TTCS and TCS, by about 0.29 - 16.51 %. Additionally, using HTTCS can increase the % problem-solving by up to 99.98 %, is which higher than others about at most 0.66 %.

Framework for Optimizing Test Cases in Regression Testing

Software once developed is subject to continuous changes. Software Regression Testing thus can be used to reduce the efforts of testing the software by selecting only the required number of test cases and ordering them to test the software after changes have been made to it. In order to improve the fault detection rate, the selection of efficient test cases and order of execution of these tests is important. Here is when the test case selection comes into picture where in, the fault detection rate during the working of any software has to be improved. The test case selection process will find the most efficient test cases which can fully functionally test the software that has been modified. This indeed will contribute to an improved fault detection rate which can provide faster feedback on the system under test and let software engineers begin correcting faults as early as possible. In this paper, an approach for test case selection is proposed which takes into consideration the effect of three parameters History, Coverage and Requirement all together in order to improve the selection process. This will also ensure that the rejection of some efficient test cases is reduced when compared to the selection process in conventional methods, most of them making use of a single parameter for test case selection. These Test cases are further optimized using Genetic Algorithm. Results indicate that the proposed technique is much more efficient in terms of selecting the test cases when compared to conventional techniques, thereby improving fault detection rate.

Test Suite Minimization in Regression Testing Using Hybrid Approach of ACO and GA

This article describes about the application of search-based techniques in regression testing and compares the performance of various search-based techniques for software testing. Test cases tend to increase exponentially as the software is modified. It is essential to remove redundant test cases from the existing test suite. Regression testing is very costly and must be performed in restricted ways to ensure the validity of the existing software. There exist different methods to improve the quality of test cases in terms of the number of faults covered, opposed to the number of statements covered in a minimum time. Different methods exist for this purpose, such as minimization, test case selection, and test case prioritization. In this article, search-based methods are applied to improve the quality of the test suite in order to select a minimum set of test cases which covers all the statements in a minimum time. The whole approach is named search based regression testing. In this paper, the performance of different metaheuristics for test suite minimization problem is also compared with a hybrid approach of ant colony optimization algorithm and genetic algorithm.

Hybridization of Cuckoo-ACO algorithm for test case prioritization

Regression testing is one of the types of maintenance testing and it is performed in case of bug-fixing or whenever there is any new functionality incorporate in the dynamic environment of software development process. Due to which the cost of development process increases because it directly affects the validation process. The addition of the new requirements with limited resources of development process in a time constrained environment definitely increase the cost. Efficient and effective test case selection from the available test suite becomes very critical problem in this scenario and this situation makes it important to applying of some techniques which are prioritization of test cases and selection of test cases for re-arrangement of test cases in a particular schedule and these test cases are selected in a particular sequence order to fulfillment of some selected criteria. This paper suggests Cuckoo Search (CS) algorithm followed by Modified Ant Colony Optimization (M-ACO) algorithm to conclude the test cases in an optimized order in time constrained environment. CS algorithm is inspired by some species of cuckoos having constrained brood lethargy and laying of their eggs in other host bird’s nest. Due to dependency of this algorithm on one single parameter, cuckoo search unlike other optimization algorithms, is more efficient and very easy to implement. But the hybrid Cuckoo-ACO optimization technique is more appropriate for the test case selection and prioritization according to the proposed empirical study.

Experimental and numerical investigation of wave induced forces and motions of partially submerged bodies near a fixed structure in irregular waves

This paper is the third of a series covering investigations of the wave induced force and motion behavior of ice masses near a fixed structure. The objective of this phase of the study was to determine whether the effects observed in regular waves (Sayeed et al. (2017b, 2018)) still held in irregular seas. In the present case, wave characteristics measured in front of the fixed structure showed that the significant wave heights follow a standing wave pattern generated by the superposition of the incident and reflected peak frequency wave. Thus it may be inferred that the significant surge and heave forces and motions will be increased or decreased at the node and antinode locations in front of the structure as previously observed for regular waves. Force spectra generated from regular wave force data (Sayeed et al. 2017b) and wave measurements from the present study showed that beyond the nodal location at Dλp=0.25, the significant surge force goes down and significant heave force goes up as the body gets close to the structure. Direct measurements of irregular wave forces on a fixed berg model at different proximity to the fixed structure demonstrate the change in significant wave forces based on the ratio of separation distance to peak wavelength. When the berg body is positioned closer to the fixed structure, the normalized significant forces in the horizontal direction decrease, and the normalized significant forces in the vertical direction increase. Significant differences were found between the directly measured force spectra and those produced by linear superposition of regular wave force transfer functions. This is attributed to non-linear effects arising from the iceberg profile and low freeboard.Free floating motion experiments showed smaller differences in the surge and heave velocity spectra due to the presence of the structure. This is because there is insufficient motion data at any specific location to capture the motion responses for all the frequencies in the spectrum. The proximity effect on wave loads and wave induced motions in the vertical direction is found to be more pronounced than the same in the horizontal direction.Numerical simulations using RANS based commercial CFD code Flow3D© are compared to the experimental results. Selected test cases from both force and motion experiments are simulated in irregular waves generated from selected experimental wave spectra. Numerical results for irregular waves demonstrate the ability to simulate random waves and resulting forces and motions with trends comparable to the experimental results.

Wind Turbine Power Curve Upgrades

Full-scale wind turbine is a mature technology and therefore several retrofitting techniques have recently been spreading in the industry to further improve the efficiency of wind kinetic energy conversion. This kind of interventions is costly and, furthermore, the energy improvement is commonly estimated under the hypothesis of ideal wind conditions, but real ones can be very different because of wake interactions and/or wind shear induced by the terrain. A precise quantification of the energy gained in real environment is therefore precious. Wind turbines are subjected to non-stationary conditions and therefore it makes little sense to compare energy production before and after an upgrade: the post-upgrade production should rather be compared to a model of the pre-upgrade production under the same conditions. Since the energy improvement is typically of the order of few percents, a very precise model of wind turbine power output is needed and therefore it should be data-driven. Furthermore, the formulation of the model is heavily affected by the features of the available data set and by the nature of the problem. The objective of this work is the discussion of some wind turbine power curve upgrades on the grounds of operational data analysis. The selected test cases are: improved start-up through pitch angle adjustment near the cut-in, aerodynamic blade retrofitting by means of vortex generators and passive flow control devices, and extension of the power curve through a soft cut-out strategy for very high wind speed. The criticality of each test case is discussed and appropriate data-driven models are formulated. These are employed to estimate the energy improvement from each of the upgrades under investigation. The general outcome of this work is a catalog of generalizable methods for studying wind turbine power curve upgrades. In particular, from the study of the selected test cases, it arises that complex wind conditions might affect wind turbine operation such that the production improvement is non-negligibly different from what can be estimated under the hypothesis of ideal wind conditions. A complex wind flow might actually impact on the efficiency of vortex generators and the soft cut-out strategies at high wind speeds. The general lesson is therefore that it is very important to estimate wind turbine upgrades on real environments through operational data.

Test case prioritization and selection technique in continuous integration development environments: a case study

Regression testing is a very important activity in continuous integration development environments. Software engineers frequently integrate new or changed code that involves in a new regression testing. Furthermore, regression testing in continuous integration development environments is together with tight time constraints. It is also impossible to re-run all the test cases in regression testing. Test case prioritization and selection technique are often used to render continuous integration processes more cost-effective. According to multi objective optimization, we present a test case prioritization and selection technique, TCPSCI, to satisfy time constraints and achieve testing goals in continuous integration development environments. Based on historical failure data, testing coverage code size and testing execution time, we order and select test cases. The test cases of the maximize code coverage, the shorter execution time and revealing the latest faults have the higher priority in the same change request. The case study results show that using TCPSCI has a higher cost-effectiveness comparing to the manually prioritization.

Viscous flow simulations at high Reynolds numbers without wall functions: Is [formula omitted] enough for the near-wall cells?

The Reynolds-Averaged Navier Stokes equations supplemented by eddy-viscosity models are still the most common mathematical model for the simulation of wall-bounded viscous flows at high Reynolds numbers. Viscous flow simulations around complex geometries are more accurate with the direct application of the no-slip condition at walls, i.e. avoiding the use of wall functions. Determination of the shear-stress at the wall from its definition requires the use of near-wall cells which are typically requested to present a non-dimensional height of y+≃1. However, the effect of this widespread rule of thumb on the numerical accuracy of the solutions has not been established and it may be dependent on the turbulence model choice.In this paper, we discuss the numerical accuracy of viscous flow simulations performed with a RANS solver supplemented by three eddy-viscosity models in the calculation of statistically steady, incompressible flows: the one-equation model of Spalart & Allmaras and the two-equation models SST k–ω and k–kL. The selected test cases are: a flat plate at Reynolds numbers Re equal to 107, 108 and 109; the NACA 0012 airfoil at angles of attack of 0, 4 and 10 degrees with Re=6×106 and the KVLCC2 tanker at model and full scale Reynolds numbers, Re=4.6×106 and Re=2.03×109. Calculations are performed in sets of geometrically similar grids with different sizes of the near-wall cells height to assess the influence of this choice on the numerical uncertainty of the solutions.The results show that the numerical accuracy of the simulations is dependent on the selected turbulence model. For y+<1, there is no significant effect of the near-wall cell height on the numerical uncertainty of the selected flow quantities for the Spalart & Allmaras and k–kL models and so y+≃1 is sufficient to obtain numerical uncertainties of friction resistance coefficients smaller than 1% . On the other hand, with the SST k–ω model, y+≃1 leads to numerical uncertainties of friction resistance coefficients larger than 5% for all the present test cases. To attain the same uncertainty of the other turbulence models with the SST k–ω, one must use y+≃0.1.

Modeling of bubble flows in vertical pipes with the N-phase compressible algebraic slip model

In the current work, the N-phase compressible algebraic slip model (ASM) framework is presented. The framework is implemented in TransAT© which is a commercial finite-volume CFD solver specialized in the modeling of multiphase flows. It is applied to bubbly flows in vertical pipes where compressibility of the gas phase is important. Effects of compressibility of the gas phase namely, the change in volume of the individual bubbles, the reduction in the liquid volume fraction, and the change in bubble size due to almost constant number density are identified and illustrated with the help of an analytical one-dimensional model. It was shown that the volume effect was the strongest, although the effect of the change in the bubble radius is also important though to a lesser extent.The framework implemented in TransAT© does not solve for the gas number density and assumes a fixed radius for the bubbles. In the first step the implementation is verified with the help of the one-dimensional model. In the next step, the implementation is tested against the selected test cases from the TOPFLOW database for different types of bubbly flow namely the wall-peaked, the transition and the central-peaked. The simulation results are able to predict the overall behavior of the different types of bubbly flow. The radial gas velocity profiles for all the test cases showed a good match with the experimental data. Unrealistic accumulation of small bubbles close to the walls is observed for the wall-peaked test cases. The results showed that the unrealistic behaviour could be removed tuning the wall lubrication force model. The results highlight the inadequacy of the current wall lubrication force models and show that the wall lubrication force should depend on the velocity of the continuous phase probably in an analogous way to the lift force.

Test Case Selection Using Feature Extraction and Clustering

This article explains the selection of important parameters from an execution pattern which brings out the details of the application of test cases. Hence, execution profiles are captured and a new execution profile-based clustering approach is chosen for test case selection, which uses three new features. These are Function frequency, Branches taken and Block percentage. The test cases are clustered using the extracted features. The experiments show that the proposed FBB selects smaller size of more relevant test cases which are more fault revealing compared to the existing Function Call Profile approach.

SPIRITuS: a SimPle Information Retrieval regressIon Test Selection approach

Context:Regression Test case Selection (RTS) approaches aim at selecting only those test cases of a test suite that exercise changed parts of the System Under Test (SUT) or parts affected by changes.Objective:We present SPIRITuS (SimPle Information Retrieval regressIon Test Selection approach). It uses method code coverage information and a Vector Space Model to select test cases to be run. In a nutshell, the extent of a lexical modification to a method is used to decide if a test case has to be selected. The main design goals of SPIRITuS are to be: (i) easy to adapt to different programming languages and (ii) tunable via an easy to understand threshold.Method:To assess SPIRITuS, we conducted a large experiment on 389 faulty versions of 14 open-source programs implemented in Java. We were mainly interested in investigating the tradeoff between the number of selected test cases from the original test suite and fault detection effectiveness. We also compared SPIRITuS against well-known RTS approaches.Results:SPIRITuS selects a number of test cases significantly smaller than the number of test cases the other approaches select at the price of a slight reduction in fault detection capability.Conclusions:SPIRITuS can be considered a viable competitor of existing test case selection approaches especially when the average number of test cases covering a modified method increases (such information can be easily derived before test case selection takes place).

Test case selection using multi-criteria optimization for effective fault localization

As spectra-based fault localization techniques report suspicious statements by analyzing the coverage of test cases, the effectiveness of the results is highly dependent on the composition of test suites. This paper proposes an approach for selecting a subset of the passed test suite when a failure revealed by a failed test case. The goal is to obtain a more effective fault localization using a minimal number of test cases than using the originally given large number of test cases. A novelty is that a prioritization criterion and a selection criterion are defined. Different from previous studies, the failed trace is fully considered. The prioritization criterion partitions statements in the failed trace into more suspicious and less suspicious, and then ranks passed test cases by their ability in distinguishing the more suspicious statements from the less suspicious ones. The selection criterion selects the minimal passed test suite which can maximize the number of coverage equivalent classes in the failed trace, so as to distinguish the suspicious statements and meanwhile reduce the size of the test suite. Another novelty is that our approach turns the test case selection into a multi-criteria optimization to make the prioritization and the selection criteria complement each other. This approach was evaluated with 5 fault localization techniques, 8 subject programs and 35,392 test cases. The results show that the fault localization effectiveness can be significantly improved with less than 5% passed test cases. Our approach has advantages over the statement- based and vector-based test suite reduction approaches in both fault localization effectiveness and test suite reduction rate.

Performance Analysis of BCO-m-GA Technique for Test Case Selection

Objectives: Algorithms analogous to the natural processes have been employed to optimize the fault detection efficiency and reduce software testing time. Test case selection for software testing is the most critical and time consuming part as exhaustive software testing is not possible. Recently, a novel compound technique BCO-m-GA for test case selection has been proposed based on non-sequential combination of stratified sampling, BCO (Bee Colony Optimization) and GA (Genetic Algorithm). This paper makes a performance evaluation of the proposed BCO-m-GA technique. Methods: This paper attempts to analyse the performance of the BCO-m-GA technique for test case selection against individual non-compounded algorithms GA and BCO. Findings: The results underline the improved performance of BCO-m-GA in comparison to GA and BCO in terms of differential of number of Decision Nodes Covered (DNC) and rate of change of Fitness Score (FS) of the optimal solution. Improvements: The insights obtained in this study would help researchers in improved framing of test case selection procedures. Keywords: Bee Colony Optimization, Genetic Algorithm, Software Testing, Test Case Optimization, Test Case Selection

Considerations for Regression Testing Process in Agile Development Environments

Testing is central piece in agile development methodology, that fast is taking roots and at the same time it is becoming challenging, in particular due to short time periods between incremental updates of the software, as such the time for overall testing including regression testing is becoming limited. In regression testing, the changed code as well as certain portion of existing code that is impacted is to be retested. Intuitively, it seems that the best option is to automate the regression testing. However, precisely due to time constraints the automation approach is proving to be challenging for regression testing in agile development environments. As the time to create and update these automation scripts will become limiting factor in case of overnight/weekly releases. Regression testing in agile development environments should be using some prioritization of tests for next cycle to enable timely regression testing. We can use the information of future release plans, defined by the specifications for each release, and prioritize the test cases as we write them. As there can be information available for future release cycles (as there are releases in planned for short intervals) we can have some methods of using this information and make the process of regression testing process in future more effective and efficient. In this paper, we present a technique of using future builds (new enhancements, features, or fixes) to select test cases on a class of software which is used for interconnecting diverse systems in near real time fashion. The results are encouraging and the technique can be used to guide regression testing process in agile development environment.

Energy Consumption Fuzzy Estimation for Object-Oriented Code

The energy consumption (EC) estimation of a computing system is a primitive approach for evaluating its energy efficiency and for further optimization. Existing studies adopt the runtime-dependent approach to measure the EC of software; however, in this paper, the source-level and fuzzy estimation approach are employed to measure the EC of software code, especially object-oriented code, before it is executed. This approach is beneficial to source code quality improvement and EC static optimization. For runtime independence, a fuzzy energy consumption (FEC) model is proposed, in which the code and the EC model, as well as the mappings between them, are first defined; then, the process for estimating the FEC of an arbitrary statement is described, including the statement features, data preparation, fuzzy comprehensive evaluation, and fuzzy pattern matching. Finally, experiments are performed, including the regularities verification, the statement features analysis, the EC and FEC comparison, and the FEC application. The experimental results show that the mean values of EC/FEC for the selected test cases are stable, their standard deviation is approximately 0.00064, and their mean is approximately 0.0059. By FEC, it is feasible to compare the ECs of code statically with reasonable accuracy.

Spectrum-Based and Program Slicing Statistical Fault Localization

Software fault localization is a necessary and time-consuming process in program debugging. An effective fault location method can help developers quickly locate faults and improve debug efficiency .In this paper, a new SIFL (Spectrum-Based Improved Fault Localization) method is proposed to improve the accuracy of fault location, by adding program slicing technology and test case selection technology to traditional spectrum-based fault localization. In the experimental section, this paper uses the public fault data set SIEMENT[1] to test the accuracy of the SIFL method. According to the experimental results, the SIFL method significantly improves the accuracy of fault location compared to SFL method. And using the SIFL method, the average fault location ratio is 3.076% of the program, which showing SIFL method can greatly narrow the scope of fault and improve debug efficiency.

Supporting inheritance hierarchy changes in model-based regression test selection

Models can be used to ease and manage the development, evolution, and runtime adaptation of a software system. When models are adapted, the resulting models must be rigorously tested. Apart from adding new test cases, it is also important to perform regression testing to ensure that the evolution or adaptation did not break existing functionality. Since regression testing is performed with limited resources and under time constraints, regression test selection (RTS) techniques are needed to reduce the cost of regression testing. Applying model-level RTS for model-based evolution and adaptation is more convenient than using code-level RTS because the test selection process happens at the same level of abstraction as that of evolution and adaptation. In earlier work, we proposed a model-based RTS approach called MaRTS to be used with a fine-grained model-based adaptation framework that targets applications implemented in Java. MaRTS uses UML models consisting of class and activity diagrams. It classifies test cases as obsolete, reusable, or retestable based on changes made to UML class and activity diagrams of the system being adapted. However, MaRTS did not take into account the changes made to the inheritance hierarchy in the class diagram and the impact of these changes on the selection of test cases. This paper extends MaRTS to support such changes and demonstrates that the extended approach performs as well as or better than code-based RTS approaches in safely selecting regression test cases. While MaRTS can generally be used during any model-driven development or model-based evolution activity, we have developed it in the context of runtime adaptation. We evaluated the extended MaRTS on a set of applications and compared the results with code-based RTS approaches that also support changes to the inheritance hierarchy. The results showed that the extended MaRTS selected all the test cases relevant to the inheritance hierarchy changes and that the fault detection ability of the selected test cases was never lower than that of the baseline test cases. The extended MaRTS achieved comparable results to a graph-walk code-based RTS approach (DejaVu) and showed a higher reduction in the number of selected test cases when compared with a static analysis code-based RTS approach (ChEOPSJ).

Modelling of dam failure-induced flows over movable beds considering turbulence effects

Waves induced due to partial dam failure in the suddenly enlarged section over a mobile bed are simulated. TVD-WAF scheme combined with HLLC scheme is deployed as a second-order approximate Riemann solver for solving shallow water equations in conjunction with Exner equation. Also, an approach for considering turbulence effects is used based on Boussinesq's turbulent viscosity hypothesis. In the present model, five different bed load movement formulations are used for comparative evaluation of bed load movement. The present model applied for modelling two laboratory tests for dam failure on mobile beds. According to the results of the evaluation, the numerical results of the Ribberink formulation and the Wong and Parker formulation are better than the results of other formulations. In general, the results indicate good agreement between the numerical results and the experimental data both in the water level and bed level simulation in the selected test cases.