JUnit Test Cases Research Articles

Prioritizing JUnit Test Cases Without Coverage Information: An Optimization Heuristics Based Approach

Regression testing is an expensive activity and Test Case Prioritization (TCP) acts as an improvement mechanism for it. TCP techniques for object oriented programs need attention and in our study, we explored prioritization of JUnit test cases. Ten benchmark Java programs with their several mutated versions were studied. As collecting coverage information is a costly effort, we bypassed these steps and used optimization heuristics for ordering JUnit test cases at test method level. Our approach formulated a novel fitness objective which depends on the number of modified lines executed per unit of execution time. As regression testing is performed after some modification is done on an existing program, maximizing the execution of number of modified lines is highly lucrative. The test case prioritization problem was replicated in context of 0/1 Knapsack problem and then it was solved using Genetic Algorithm (GA). Our exploration also included application of Simulated Annealing and Ant Colony Optimization method for determining the best execution ordering of test cases. We examined the usage of Multi-objective GA by building another new fitness metric which aims to maximize the number of inheritance edges covered by a test case. Results indicate the superiority of optimization heuristics over other existing approaches. It appeared that multi-objective GA yielded better result than single objective prioritization. Among the single objective techniques, ACO performed best. To the best of our knowledge, this is the first study which explored all the above mentioned optimization heuristics for ordering JUnit test cases with the newly coined fitness intents.

Predicting different levels of the unit testing effort of classes using source code metrics: a multiple case study on open-source software

Nowadays, the growth in size and complexity of object-oriented software systems bring new software quality assurance challenges. Applying equally testing (quality assurance) effort to all classes of a large and complex object-oriented software system is cost prohibitive and not realistic in practice. So, predicting early the different levels of the unit testing effort required for testing classes can help managers to: (1) better identify critical classes, which will involve a relatively high-testing effort, on which developers and testers have to focus to ensure software quality, (2) plan testing activities, and (3) optimally allocate resources. In this paper, we investigate empirically the ability of a Quality Assurance Indicator (Qi), a synthetic metric that we proposed in a previous work, to predict different levels of the unit testing effort of classes in object-oriented software systems. The unit testing effort of classes is addressed from the perspective of unit test cases construction. We focused particularly on the effort involved in writing the code of unit test cases. To capture the involved unit testing effort of classes, we used four metrics that quantify different characteristics related to the code of corresponding unit test cases. We used Means and K-Means-based categorizations to group software classes into five categories according to the involved unit testing effort. We performed an empirical analysis using data collected from eight open-source Java software systems from different domains, for which the JUnit test cases were available. To evaluate the ability of the Qi metric to predict different levels of the unit testing effort of classes, we used three modeling techniques: the univariate logistic regression, the univariate linear regression, and the multinomial logistic regression. The performance of the models based on the Qi metric has been compared to the performance of the models based on various well-known object-oriented source code metrics. We used different evaluation criteria to compare the prediction models. Results indicate that the models based on the Qi metric have more promising prediction potential than those based on traditional object-oriented metrics.

An Enhanced Approach for Software Bug Localization using Map Reduce Technique based Apriori (MRTBA) Algorithm

Background/Objectives: Software bugs are generally the faults or errors that occur in the code that may leads to incorrect results. It is therefore necessary to find software bugs to increase the quality of software and for making the software to meet the user requirements. Methods/Statistical Analysis: The testing enables assessment of software which ensures the system whether it meets the system requirements. Graph mining is an approach to find software bugs and furthermore testing involves more computational complexity and cost therefore tools are developed for testing the software. The challenging task is to locate and fix bugs automatically. Bug localization using high performance map reducing process is very successful in the recent research. Findings: This proposed technique called Map Reduce Technique Based Apriori (MRTBA) Algorithm based on Graph mining in where edge weights are used to takes the program as the input and computes method calls of the program. Here each node represents method and each edge represents a method call. The algorithm aims for detection of faulty nodes. It uses Hash Map for reducing edge weights. JUnit test cases are performed at last for detecting bugs by giving different inputs. The approach mines all nodes not only at same level but also the nodes present at different levels. Conclusion/Improvements: In this work MRTBA, a dynamic control flow centered approach for bug localization has been presented. In future this technique can be extended with the proposal of steps involves in connecting same nodes at different levels. It also further enhances to find out indirect recursions for both the trivial and non-trivial map reduction in software bug localization.

Predicting Unit Testing Effort Levels of Classes: An Exploratory Study based on Multinomial Logistic Regression Modeling

The study aims at investigating empirically the ability of a Quality Assurance Indicator (Qi), a metric that we proposed in a previous work, to predict different levels of unit testing effort of classes in object-oriented systems. To capture the unit testing effort of classes, we used four metrics to quantify various perspectives related to the code of corresponding unit test cases. Classes were classified, according to the involved unit testing effort, in five categories (levels). We collected data from two open source Java software systems (ANT and JFREECHART) for which JUnit test cases exist. In order to explore the ability of the Qi metric to predict different levels of the unit testing effort of classes, we decided to explore the possibility of using the Multinomial Logistic Regression (MLR) method. The performance of the Qi metric has been compared to the performance of three well-known source code metrics related respectively to size, complexity and coupling. Results suggest that the MLR model based on the Qi metric is able to accurately predict different levels of the unit testing effort of classes.

A metrics suite for JUnit test code: a multiple case study on open source software

The code of JUnit test cases is commonly used to characterize software testing effort. Different metrics have been proposed in literature to measure various perspectives of the size of JUnit test cases. Unfortunately, there is little understanding of the empirical application of these metrics, particularly which metrics are more useful in terms of provided information. This paper aims at proposing a unified metrics suite that can be used to quantify the unit testing effort. We addressed the unit testing effort from the perspective of unit test case construction, and particularly the effort involved in writing the code of JUnit test cases. We used in our study five unit test case metrics, two of which were introduced in a previous work. We conducted an empirical study in three main stages. We collected data from six open source Java software systems, of different sizes and from different domains, for which JUnit test cases exist. We performed in a first stage a Principal Component Analysis to find whether the analyzed unit test case metrics are independent or are measuring similar structural aspects of the code of JUnit test cases. We used in a second stage clustering techniques to determine the unit test case metrics that are the less volatile, i.e. the least affected by the style adopted by developers while writing the code of test cases. We used in a third stage correlation and linear regression analysis to evaluate the relationships between the internal software class attributes and the test case metrics. The main goal of this study was to identify a subset of unit test case metrics: (1) providing useful information on the effort involved to write the code of JUnit test cases, (2) that are independent from each other, and (3) that are the less volatile. Results confirm the conclusions of our previous work and show, in addition, that: (1) the set of analyzed unit test case metrics could be reduced to a subset of two independent metrics maximizing the whole set of provided information, (2) these metrics are the less volatile, and (3) are also the most correlated to the internal software class attributes.

Regression Test Reduction for Object-Oriented Software: A Control Call Graph Based Technique and Associated Tool

This paper presents a selective regression testing technique and an associated tool for object-oriented software. The technique is based on the concept of Control Call Graphs, which are a reduced form of traditional Control Flow Graphs. It uses static analysis of the source code of the program. The developed tool (1) identifies the Control Call Paths potentially impacted by changes, (2) selects, from an existing test suite, the appropriate test cases, and (3) generates new JUnit test cases for control call paths that are not covered by existing tests (new ones, or those whose structure has been modified after changes). In this way, the approach supports an incremental update of the test suite. The selected JUnit test cases, including the new ones, are automatically executed. Three concrete case studies are reported to provide evidence of the feasibility of the approach and its benefits in terms of reduction of regression testing effort.

A Static Approach to Prioritizing JUnit Test Cases

Test case prioritization is used in regression testing to schedule the execution order of test cases so as to expose faults earlier in testing. Over the past few years, many test case prioritization techniques have been proposed in the literature. Most of these techniques require data on dynamic execution in the form of code coverage information for test cases. However, the collection of dynamic code coverage information on test cases has several associated drawbacks including cost increases and reduction in prioritization precision. In this paper, we propose an approach to prioritizing test cases in the absence of coverage information that operates on Java programs tested under the JUnit framework-an increasingly popular class of systems. Our approach, JUnit test case Prioritization Techniques operating in the Absence of coverage information (JUPTA), analyzes the static call graphs of JUnit test cases and the program under test to estimate the ability of each test case to achieve code coverage, and then schedules the order of these test cases based on those estimates. To evaluate the effectiveness of JUPTA, we conducted an empirical study on 19 versions of four Java programs ranging from 2K-80K lines of code, and compared several variants of JUPTA with three control techniques, and several other existing dynamic coverage-based test case prioritization techniques, assessing the abilities of the techniques to increase the rate of fault detection of test suites. Our results show that the test suites constructed by JUPTA are more effective than those in random and untreated test orders in terms of fault-detection effectiveness. Although the test suites constructed by dynamic coverage-based techniques retain fault-detection effectiveness advantages, the fault-detection effectiveness of the test suites constructed by JUPTA is close to that of the test suites constructed by those techniques, and the fault-detection effectiveness of the test suites constructed by some of JUPTA's variants is better than that of the test suites constructed by several of those techniques.

Evaluating the Effect of Control Flow on the Unit Testing Effort of Classes: An Empirical Analysis

The aim of this paper is to evaluate empirically the relationship between a new metric (Quality Assurance Indicator—Qi) and testability of classes in object-oriented systems. The Qi metric captures the distribution of the control flow in a system. We addressed testability from the perspective of unit testing effort. We collected data from five open source Java software systems for which JUnit test cases exist. To capture the testing effort of classes, we used different metrics to quantify the corresponding JUnit test cases. Classes were classified, according to the required testing effort, in two categories: high and low. In order to evaluate the capability of the Qi metric to predict testability of classes, we used the univariate logistic regression method. The performance of the predicted model was evaluated using Receiver Operating Characteristic (ROC) analysis. The results indicate that the univariate model based on the Qi metric is able to accurately predict the unit testing effort of classes.

Empirical Analysis of Object-Oriented Design Metrics for Predicting Unit Testing Effort of Classes

In this paper, we investigate empirically the relationship between object-oriented design metrics and testability of classes. We address testability from the point of view of unit testing effort. We collected data from three open source Java software systems for which JUnit test cases exist. To capture the testing effort of classes, we used metrics to quantify the corresponding JUnit test cases. Classes were classified, according to the required unit testing effort, in two categories: high and low. In order to evaluate the relationship between object-oriented design metrics and unit testing effort of classes, we used logistic regression methods. We used the univariate logistic regression analysis to evaluate the individual effect of each metric on the unit testing effort of classes. The multivariate logistic regression analysis was used to explore the combined effect of the metrics. The performance of the prediction models was evaluated using Receiver Operating Characteristic analysis. The results indicate that: 1) complexity, size, cohesion and (to some extent) coupling were found significant predictors of the unit testing effort of classes and 2) multivariate regression models based on object-oriented design metrics are able to accurately predict the unit testing effort of classes.

An empirical analysis of a testability model for object-oriented programs

We present, in this paper, a metric based testability model for object-oriented programs. The model is, in fact, an adaptation of a model pro-posed in literature for assessing the testability of object-oriented design. The study presented in this paper aims at exploring empirically the capa-bility of the model to assess testability of classes at the code level. We investigate testability from the perspective of unit testing and required testing effort. We designed an empirical study using data collected from two Java software systems for which JUnit test cases exist. To capture testability of classes in terms of required testing effort, we used different metrics to quantify the corresponding JUnit test cases. In order to eva-luate the capability of the model to predict testability of classes (charac-teristics of corresponding test classes), we used statistical tests using correlation.

An empirical study into class testability

In this paper we investigate factors of the testability of object-oriented software systems. The starting point is given by a study of the literature to obtain both an initial model of testability and existing object-oriented metrics related to testability. Subsequently, these metrics are evaluated by means of five case studies of commercial and open source Java systems for which JUnit test cases exist. The goal of this paper is to identify and evaluate a set of metrics that can be used to assess the testability of the classes of a Java system.

Prioritizing JUnit Test Cases: An Empirical Assessment and Cost-Benefits Analysis

Test case prioritization provides a way to run test cases with the highest priority earliest. Numerous empirical studies have shown that prioritization can improve a test suite's rate of fault detection, but the extent to which these results generalize is an open question because the studies have all focused on a single procedural language, C, and a few specific types of test suites. In particular, Java and the JUnit testing framework are being used extensively to build software systems in practice, and the effectiveness of prioritization techniques on Java systems tested under JUnit has not been investigated. We have therefore designed and performed a controlled experiment examining whether test case prioritization can be effective on Java programs tested under JUnit, and comparing the results to those achieved in earlier studies. Our analyses show that test case prioritization can significantly improve the rate of fault detection of JUnit test suites, but also reveal differences with respect to previous studies that can be related to the language and testing paradigm. To investigate the practical implications of these results, we present a set of cost-benefits models for test case prioritization, and show how the effectiveness differences observed can result in savings in practice, but vary substantially with the cost factors associated with particular testing processes.

Enhancing Design by Contract with Knowledge about Equivalence Partitions.

Software testing seems to be a huge struggle for most developers. This is presumably because of the amount of manual work involved in software testing. In this article we propose a testing approach that makes the testing process less manual. The approach combines three well-known concepts Design by Contract, Unit Testing and Equivalence Partitioning into a new approach named Testing by Contract. The first step is to define a new programming language that supports an extended form of Design by Contract. The idea is to enhance the contracts with knowledge of Equivalence Partitions. The second step is to design a tool that exploits the enhanced contracts to build quality test cases automatically. The test cases are built as JUnit test cases, which allow them to be combined with manually written test cases. A prototype Java-like language with support for this extended form of Design by Contract has been developed. A compiler to regular Java-code as well as a tool for automatic generation of test cases is currently being built. This prototype language/tool will be used to further examine and evaluate the ideas described above.