Test Data Generation Research Articles

Data-Driven Roll Pass Design of Wire Rod Mills

The classical approach for roll pass design of a wire rod mill employs an iterative technique incorporating spread calculation and rectangular equivalent pass methods. This method comes to its limits in terms of computational efficiency and numerical stability when a complete pass design for a wire rod mill with lots of different final dimensions and materials must be designed. To improve the pass design technique, a fast data-driven method for pass design based on synthetic data generated by the classical pass design model was created. The results are compared to the original training data, as well as newly generated test data. It is shown that the artificial neural network (ANN) is able to predict appropriate oval groove geometries with good precision.

A NOVEL APPROACH FOR TEST DATA GENERATION

Software testing is an essential phase in software design process, accounting for more than half of the total cost due to its rigorous and time-consuming nature. Path test data generation is the most important stage in software testing, and researchers have devised several methods to automate it. In this research, a novel approach based on ant colony optimization and negative selection algorithm (NSA) is projected to automatically create test data for path testing. The most widely used benchmark programs such as triangle classification, dayfinder, minmax and isprime, has been used to test the proposed approach. When compared to random testing, the experimental findings reveal that the proposed method is more efficient in terms of coverage, execution time and more effective in terms of test data creation.

Optimal Model of Software Testing Path Selection Based on Genetic Algorithm and Its Evolutionary Solution

In software testing, the selection of test data is a difficult problem in structural testing. Whether the test data is appropriate or not is directly related to whether the error can be expected to be detected. In the process of software testing, the generation of test data is not only the core problem but also the key and difficulty of software testing. Because of the huge number of test cases and low test efficiency, a powerful optimization algorithm is needed to optimize the initial test cases. As a robust search method, genetic algorithm shows unique advantages and high efficiency in solving high-complexity problems such as large space, multipeak, nonlinear, and global optimization. Based on the application of genetic algorithm, this paper analyzes the optimization path by classifying and calculating the objective function and introducing NSGA-II algorithm, measures the distance between each branch on the processing path sample set, and sorts the path set to obtain the optimal solution. On the basis of the designed model, the experimental results show that the error control rate of the model is 89.4%. Moreover, because of the superiority of NSGA-II algorithm, the probability of comprehensive cross mutation is increased by 56.7%.

Test data generation using flocking of fireflies in software testing

Test data generation using flocking of fireflies in software testing

An automated test data generation method for void pointers and function pointers in C/C++ libraries and embedded projects

Automated test data generation for unit testing C/C++ functions using concolic methods is well-known for improving software quality while reducing human testing effort. However, there have been only a few researches related to generating test data for void pointers and function pointers which are commonly used in C/C++ libraries and embedded projects. This paper proposes a concolic-based method named VFP (Void and Function Pointers test data generation) to generate test data for void pointers and function pointers. The key idea of VFP method is to preprocess the source code of the project under test to find all possible types of void pointers and references of function pointers. These types and references are used in the initial test data generating phase of the concolic testing method. VFP method is implemented in VFP verification tool to test on various C/C++ libraries and embedded projects. The experimental results show that VFP significantly improves the coverage of the generated test data in comparison with existing methods.

PERFORMANCE STUDY OF THE DTU MODEL FOR RELATIONAL DATABASES ON THE AZURE PLATFORM

When solving problems of working with relational databases on cloud platforms, the problem arises of choosing a specific model to ensure the performance of executing queries of varying complexity. The object of research is the processes of implementing various types of queries to relational databases within the framework of the DTU purchase model of the MS Azure platform. The subject is methods for evaluating the performance of work with relational databases based on the timing of query execution and indicators of the load on the resources of the cloud platform. The aim of the study is to develop a system of indicators for monitoring the current state of work with the database for reasonable decision-making on the choice of a certain price category of the DTU model of the MS Azure cloud service, which will optimize the results of working with the database. platforms Achieving the set goals involves the following tasks: to analyze modern tools and services for working with databases, in particular relational databases, on Azure and AWS cloud platforms, the features of their application and implementation; develop software for generating test relational databases of different sizes; test the generated databases on a local resource; taking into account the characteristics of the levels of the Azure DTU model, develop a new system of performance indicators, which includes 2 groups - time indicators and indicators of the load on existing platform resources; develop and implement queries of varying complexity for the generated test database for different levels of the DTU model and analyze the results. Methods. The following methods were used in the research: methods of relational database design; methods of creating queries in SQL-oriented databases with any number of tables; methods of creating and migrating data to cloud platforms; methods of monitoring the results of queries based on time and resource indicators; methods of generating test data for relational databases; system approach for complex assessment and analysis of productivity of work with relational databases. Results. On the basis of the developed scorecard used for the current analysis of the processes of working with relational databases of the MS Azure platform, numerous experiments were carried out for different levels of the model for simple and complex queries to a database with a total volume of 20 GB: loading of DTU model levels when executing various queries, the influence of model levels DTU Azure SQL database on the performance of simple and complex queries, the dependence of the execution time of various queries on the load of the CPU and the speed of write/read operations for different levels of the model. Conclusions. The results of the experiments allow us to conclude that the levels of the DTU model - S3 and S7 - are used to generate test data of various sizes (up to 20 GB) and execute database queries. The practical use of the proposed indicators to evaluate the results of applying the DTU model will improve the efficiency of decision-making on choosing the model level when implementing various queries and generating test data on the Azure cloud platform. The developed set of indicators for working with relational databases on the Azure cloud platform expands the basis of the methodological framework for evaluating the performance of working with relational databases on cloud platforms by analyzing the results of executing the simple and complex database queries on the resources involved.

Calibration, inversion and sensitivity analysis for hydro-morphodynamic models through the application of adjoint methods

The development of reliable, sophisticated hydro-morphodynamic models is essential for protecting the coastal environment against hazards such as flooding and erosion. There exists a high degree of uncertainty associated with the application of these models, in part due to incomplete knowledge of various physical, empirical and numerical closure related parameters in both the hydrodynamic and morphodynamic solvers. This uncertainty can be addressed through the application of adjoint methods. These have the notable advantage that the number and/or dimension of the uncertain parameters has almost no effect on the computational cost associated with calculating the model sensitivities. Here, we develop the first freely available and fully flexible adjoint hydro-morphodynamic model framework. This flexibility is achieved through using the pyadjoint library, which allows us to assess the uncertainty of any parameter with respect to any model functional, without further code implementation. The model is developed within the coastal ocean model Thetis constructed using the finite element code-generation library Firedrake. We present examples of how this framework can perform sensitivity analysis, inversion and calibration for a range of uncertain parameters based on the final bedlevel. These results are verified using so-called dual-twin experiments, where the ‘correct’ parameter value is used in the generation of synthetic model test data, but is unknown to the model in subsequent testing. Moreover, we show that inversion and calibration with experimental data using our framework produces physically sensible optimum parameters and that these parameters always lead to more accurate results. In particular, we demonstrate how our adjoint framework can be applied to a tsunami-like event to invert for the tsunami wave from sediment deposits.

On the Faults Found in REST APIs by Automated Test Generation

RESTful web services are often used for building a wide variety of enterprise applications. The diversity and increased number of applications using RESTful APIs means that increasing amounts of resources are spent developing and testing these systems. Automation in test data generation provides a useful way of generating test data in a fast and efficient manner. However, automated test generation often results in large test suites that are hard to evaluate and investigate manually. This article proposes a taxonomy of the faults we have found using search-based software testing techniques applied on RESTful APIs. The taxonomy is a first step in understanding, analyzing, and ultimately fixing software faults in web services and enterprise applications. We propose to apply a density-based clustering algorithm to the test cases evolved during the search to allow a better separation between different groups of faults. This is needed to enable engineers to highlight and focus on the most serious faults. Tests were automatically generated for a set of eight case studies, seven open-source and one industrial. The test cases generated during the search are clustered based on the reported last executed line and based on the error messages returned, when such error messages were available. The tests were manually evaluated to determine their root causes and to obtain additional information. The article presents a taxonomy of the faults found based on the manual analysis of 415 faults in the eight case studies and proposes a method to support the classification using clustering of the resulting test cases.

Automatic Generation of Acceptance Test Cases From Use Case Specifications: An NLP-Based Approach

Acceptance testing is a validation activity performed to ensure the conformance of software systems with respect to their functional requirements. In safety critical systems, it plays a crucial role since it is enforced by software standards, which mandate that each requirement be validated by such testing in a clearly traceable manner. Test engineers need to identify all the representative test execution scenarios from requirements, determine the runtime conditions that trigger these scenarios, and finally provide the input data that satisfy these conditions. Given that requirements specifications are typically large and often provided in natural language (e.g., use case specifications), the generation of acceptance test cases tends to be expensive and error-prone. In this paper, we present Use Case Modeling for System-level, Acceptance Tests Generation (UMTG), an approach that supports the generation of executable, system-level, acceptance test cases from requirements specifications in natural language, with the goal of reducing the manual effort required to generate test cases and ensuring requirements coverage. More specifically, UMTG automates the generation of acceptance test cases based on use case specifications and a domain model for the system under test, which are commonly produced in many development environments. Unlike existing approaches, it does not impose strong restrictions on the expressiveness of use case specifications. We rely on recent advances in natural language processing to automatically identify test scenarios and to generate formal constraints that capture conditions triggering the execution of the scenarios, thus enabling the generation of test data. In two industrial case studies, UMTG automatically and correctly translated 95 percent of the use case specification steps into formal constraints required for test data generation; furthermore, it generated test cases that exercise not only all the test scenarios manually implemented by experts, but also some critical scenarios not previously considered.

A Research for Executable Path Automatic Generation Method Based on EFSM

Extended finite state machine (EFSM) is currently one of the most widely used model in the field of software testing. EFSM model is an enhanced model based on finite state machine (FSM). Automated test data generation is still a challenging problem due to the complexity of EFSM which extends the input and output parameters, context variables as well as the predicate condition. These reasons lead to conflict of the context variable with the enable conditions in the transition path. In order to avoid infeasible path generation, this paper proposes a method based on modified breadth first search to generate feasible transition path (MBFS-FTP). To solve the problem of state explosion in path generation, this paper converts state diagram to transition diagram on EFSM model. In order to make the EFSM static model can be driven execution, this paper uses UML model and generate executable model, so that implements EFSM execute dynamically. When using breadth-first search algorithm (MBFS) on every target transition to generate an executable transition path, the conflict detection algorithm is utilized to the transition path for conflict decision, avoiding the occurrence of an infeasible transition path. Considering target transition has multiple feasible transition paths, this paper combines suggested penalized value of definition-predicate-use (def-p-use) pair and length of feasible transition path, and then develops the measurement method of feasible transition path, and obtained a set of feasible transition path containing all of transitions. Through the experiment on two actual EFSM model, verifying the effectiveness of MBFS-FTP for feasible transition automatic generation, the experimental results show that MBFS-FTP can reduce the feasible transition path length, and make transition paths are more easily to be triggered at the times of generating test cases lately, furthermore, it can improve the efficiency of generate feasible transition path and save a lot of time.

Interpretation of temperature transient data from coupled reservoir and wellbore model for single phase fluids

This paper presents a non-isothermal, transient coupled reservoir/wellbore model that accounts for the Joule-Thomson (J-T), isentropic expansion, conduction and convection effects for predicting the transient temperature behavior and computing the wellbore temperature at different gauge depths. In this study, single phase fluid flow of oil or geothermal brine from a fully penetrating vertical or inclined well in an infinite-acting homogeneous single layer reservoir is modeled. The coupled simulator solves mass, momentum and energy conservation equations simultaneously for both reservoir and wellbore. We improve solutions by the functional iteration procedure that updates fluid properties based on available correlations as a function of pressure and temperature at a given time step. A comparison of the developed model with a commercial simulator is provided. To understand and identify diagnostic characteristics of temperature transients at gauge locations at the sandface and above the sandface that may arise during a well test, we examine the sensitivity of the model parameters appearing in the coupled non-isothermal reservoir/wellbore model through a synthetically generated test data sets and history matched field application. The drawdown and buildup sandface transient temperature data are obtained from the coupled model and used to interpret and analyze temperature transients. In addition to the J-T coefficient of fluid, history matching transient temperature data provides estimates for the skin zone radius and permeability when analyzed jointly with the conventional pressure test analysis (PTA). An investigation on the effect of gauge location on temperature data shows that the early-time response is influenced by the wellbore phenomena while the J-T effects are clearly identified at later times at typical gauge locations up to 100 m above the top of the producing horizon. Logarithmic derivative of temperature transients are found as a useful diagnostic tool to differentiate the wellbore phenomena from the reservoir response. It is also shown that the temperature transient is more reflective of the properties of the near wellbore region (e.g., skin zone) than the pressure transient. For this reason, analyzing temperature transients together with the pressure transients could add more value to the analysis to better examine near wellbore characteristics. • A fully implicit transient single phase coupled wellbore/reservoir non-isothermal simulator is presented. • New diagnostic plots for temperature transient analysis are provided. • For typical gauge locations (usually up to 100 m above the producing horizon), reservoir response is acquired through drawdown wellbore temperature data. • An effective well radius is introduced for history matching due to the placement of many tools in wellbore during temperature transient analysis. • Reservoir and skin zone parameters are estimated reliably by history matching pressure and temperature field data together.

Comparison of form measurement results for optical aspheres and freeform surfaces

Comparing form measurement data for aspheres and freeform surfaces is an important tool for ensuring the quality and functionality of the devices used to take such measurements and may also allow the underlying measurement methods to be evaluated. However, comparing the highly accurate form measurements of such complex surfaces is a demanding task. It is difficult to analyze measurement results whose accuracies are in the range of several tens of nanometers root-mean-square, especially when comparing data with different, and anisotropic distributions of the 3D measurement points on the surface under test. In this paper, we investigate eight different 3D measurement point distributions that are typical of highly accurate measurement systems currently in use and demonstrate the effects of these distributions on the comparison results by using virtually generated data and applying different evaluation strategies. The results show that, for the examples investigated, the different 3D measurement point distributions can yield different levels of accuracy for the comparison. Furthermore, an improved evaluation procedure is proposed and recommendations on how to significantly reduce the influence of the different 3D measurement point distributions on the comparison result are given. A method of employing virtually generated test data is presented that may be generalized in order to further improve and validate future comparison methods.

BOOMPizer: Minimization and prioritization of CONCOLIC based boosted MC/DC test cases

Recent research evidence indicates that the powerful testing tools, even though generate test inputs automatically for coverage measures, but not up to satisfaction. These tools sometimes achieve high structural coverage, which do not guarantee to have high fault detection ability. These findings lead us to a decisive point that code coverage is merely one factor towards effective test data generation. Thus, we discuss our findings and proposed work on Modified Condition/ Decision Coverage (MC/DC) test case generation and prioritization techniques. This work aims to generate, minimize, and prioritize MC/DC test cases obtained through concolic testing process. This work presents three technical contributions. The first contribution is to propose a greedy algorithm to increase the number of effective test cases for improving MC/DC scores. The second contribution is to minimize the updated test suite size to have only the optimal number of contributing test cases towards forming MC/DC pairs. The third contribution is to prioritize these test cases by considering both their Contribution Index (CI) values and Fault Exposing Potential (FEP) values. The proposed approach is validated by experimenting on eighteen Java programs and achieved on an average ×1.67 times increase in the number of effective test cases that lead to an average increase in MC/DC score by 41.08%. We also achieved on an average 49.00% reduction rate to minimize the test suite size and finally prioritized the test cases, based on their prioritization index values.

An EFSM-Based Test Data Generation Approach in Model-Based Testing

Testing is an integral part of software development. Current fast-paced system developments have rendered traditional testing techniques obsolete. Therefore, automated testing techniques are needed to adapt to such system developments speed. Model-based testing (MBT) is a technique that uses system models to generate and execute test cases automatically. It was identified that the test data generation (TDG) in many existing model-based test case generation (MB-TCG) approaches were still manual. An automatic and effective TDG can further reduce testing cost while detecting more faults. This study proposes an automated TDG approach in MB-TCG using the extended finite state machine model (EFSM). The proposed approach integrates MBT with combinatorial testing. The information available in an EFSM model and the boundary value analysis strategy are used to automate the domain input classifications which were done manually by the existing approach. The results showed that the proposed approach was able to detect 6.62 percent more faults than the conventional MB-TCG but at the same time generated 43 more tests. The proposed approach effectively detects faults, but a further treatment to the generated tests such as test case prioritization should be done to increase the effectiveness and efficiency of testing.

Independent Channel Residual Convolutional Network for Gunshot Detection

The main purpose of this work is to propose a robust approach for dangerous sound events detection (e.g. gunshots) to improve recent surveillance systems. Despite the fact that the detection and classification of different sound events has a long history in signal processing, the analysis of environmental sounds is still challenging. The most recent works aim to prefer the time-frequency 2-D representation of sound as input to feed convolutional neural networks. This paper includes an analysis of known architectures as well as a newly proposed Independent Channel Residual Convolutional Network architecture based on standard residual blocks. Our approach consists of processing three different types of features in the individual channels. The UrbanSound8k and the Free Firearm Sound Library audio datasets are used for training and testing data generation, achieving a 98 % F1 score. The model was also evaluated in the wild using manually annotated movie audio track, achieving a 44 % F1 score, which is not too high but still better than other state-of-the-art techniques.

Automated Software Test Data Generation With Generative Adversarial Networks

With the rapid increase of software scale and complexity, the cost of traditional software testing methods will increase faster than the scale of software. In order to improve test efficiency, it is particularly important to automatically generate high-quality test cases. This paper introduces a framework for automatic test data generation based on the generative adversarial network (GAN). GAN is employed to train a generative model over execution path information to learn the behavior of the software. Then we can use the trained generative model to produce new test data, and select the test data that can improve the branch coverage according to our proposed selection strategy. Compared to prior work, our proposed method is able to handle programs under test with large-scale branches without analyzing branch expressions. In the experiment, we exhibit the performance of our method by using two modules in GNU Scientific Library. In particular, we consider the application of our method in two testing scenarios; unit testing and integration testing, and conduct a series of experiments to compare the performance of three types of GAN models. Results indicate that the WGAN-GP shows the best performance in our framework. Compared with the random testing method, the WGAN-GP based framework improves the test coverage of five functions out of the seven in the unit testing.

Effectiveness Assessment of the Search-Based Statistical Structural Testing

Search-based statistical structural testing (SBSST) is a promising technique that uses automated search to construct input distributions for statistical structural testing. It has been proved that a simple search algorithm, for example, the hill-climber is able to optimize an input distribution. However, due to the noisy fitness estimation of the minimum triggering probability among all cover elements (Tri-Low-Bound), the existing approach does not show a satisfactory efficiency. Constructing input distributions to satisfy the Tri-Low-Bound criterion requires an extensive computation time. Tri-Low-Bound is considered a strong criterion, and it is demonstrated to sustain a high fault-detecting ability. This article tries to answer the following question: if we use a relaxed constraint that significantly reduces the time consumption on search, can the optimized input distribution still be effective in fault-detecting ability? In this article, we propose a type of criterion called fairness-enhanced-sum-of-triggering-probability (p-L1-Max). The criterion utilizes the sum of triggering probabilities as the fitness value and leverages a parameter p to adjust the uniformness of test data generation. We conducted extensive experiments to compare the computation time and the fault-detecting ability between the two criteria. The result shows that the 1.0-L1-Max criterion has the highest efficiency, and it is more practical to use than the Tri-Low-Bound criterion. To measure a criterion’s fault-detecting ability, we introduce a definition of expected faults found in the effective test set size region. To measure the effective test set size region, we present a theoretical analysis of the expected faults found with respect to various test set sizes and use the uniform distribution as a baseline to derive the effective test set size region’s definition.

Modeling Of Subscriber Station Energy Consumption Network 802.16

Worldwide Interoperability for Microwave Access (WiMAX) is a Broadband Wireless Access (BWA) technology that has high speed and wide access for multimedia communication. The reception of quality video or voice packets at the receiver is related to how efficient the energy consumption is sent by the Subscriber Station (SS). The more video or voice packets that are sent, the more energy consumption is needed. One way to calculate energy consumption at a WiMAX subscriber station is by using mathematical modeling. This final project is modeling energy consumption in WiMAX. From the simulation, it is found that the change in the bit rate affects the energy consumption of the subscriber station. Through simulation, training data is generated to obtain a mathematical model.The mathematical model contains the components of the state duration and the level of energy consumption. Mathematical models are then used to predict energy consumption in WiMAX. The model is tested through the generation of test data. The test results through simulation showthe percentage deviation of the average energy consumption of training data with mathematical modeling on average is 0.180%. Meanwhile, the energy consumption of test data and mathematical models has an average deviation of 0.187% and 0.191%. The least energy consumption is generated when the MAC Initialization state (state i0) is 0.001 Joule, while the highest energy consumption is the Uplink Frame state (state i14) of 101.017226 Joule.

Software Testing using an Adaptive Genetic Algorithm

In the structural software test, test data generation is essential. The problem of generating test data is a search problem, and for solving the problem, search algorithms can be used. Genetic algorithm is one of the most widely used algorithms in this field. Adjusting genetic algorithm parameters helps to increase the effectiveness of this algorithm. In this paper, the Adaptive Genetic Algorithm (AGA) is used to maintain the diversity of the population to test data generation based on path coverage criterion, which calculates the rate of recombination and mutation with the similarity between chromosomes and the amount of chromosome fitness during and around each algorithm. Experiments have shown that this method is faster for generating test data than other versions of the genetic algorithm used by others.

SPOT: Testing Stream Processing Programs with Symbolic Execution and Stream Synthesizing

Adoption of distributed stream processing (DSP) systems such as Apache Flink in real-time big data processing is increasing. However, DSP programs are prone to be buggy, especially when one programmer neglects some DSP features (e.g., source data reordering), which motivates development of approaches for testing and verification. In this paper, we focus on the test data generation problem for DSP programs. Currently, there is a lack of an approach that generates test data for DSP programs with both high path coverage and covering different stream reordering situations. We present a novel solution, SPOT (i.e., Stream Processing Program Test), to achieve these two goals simultaneously. At first, SPOT generates a set of individual test data representing each path of one DSP program through symbolic execution. Then, SPOT composes these independent data into various time series data (a.k.a, stream) in diverse reordering. Finally, we can perform a test by feeding the DSP program with these streams continuously. To automatically support symbolic analysis, we also developed JPF-Flink, a JPF (i.e., Java Pathfinder) extension to coordinate the execution of Flink programs. We present four case studies to illustrate that: (1) SPOT can support symbolic analysis for the commonly used DSP operators; (2) test data generated by SPOT can more efficiently achieve high JDU (i.e., Joint Dataflow and UDF) path coverage than two recent DSP testing approaches; (3) test data generated by SPOT can more easily trigger software failure when comparing with those two DSP testing approaches; and (4) the data randomly generated by those two test techniques are highly skewed in terms of stream reordering, which is measured by the entropy metric. In comparison, it is even for test data from SPOT.