Search-based Techniques Research Articles

A report on the genetic improvement workshop@GECCO 2016

In July I was fortunate enough to attend the 2nd Genetic Improvement workshop in Denver, hosted under the Genetic and Evolutionary Computing Conference (GECCO) umbrella. For those unaware, Genetic Improvement (or simply 'GI') is an emerging area of software engineering that attempts to use search-based techniques to improve code. I myself have spent much of the last two years emerging myself in GI literature and carrying out investigations relevant to the field. Even in this short time I have been privileged to witness it grow from the solving of toy problems to frameworks capable of outperforming human-driven manual alternatives (readers need look no further than the winners of this year's Humie awards). Events such as the GI workshop serve as a platform for ideas, research proposals, and new experimental results to be discussed. They fill a niche for those who wish to discuss research with others in a structured manner while avoiding the restrictive formalities of submitting to a conference or journal. Building on the success of the 1st workshop (hosted in Madrid last year) the event was extended to cover three consecutive sessions at the GECCO conference. Pleasingly attendance was high with an audience full of both familiar faces and curious newcomers.

On the application of search-based techniques for software engineering predictive modeling: A systematic review and future directions

Software engineering predictive modeling involves construction of models, with the help of software metrics, for estimating quality attributes. Recently, the use of search-based techniques have gained importance as they help the developers and project-managers in the identification of optimal solutions for developing effective prediction models. In this paper, we perform a systematic review of 78 primary studies from January 1992 to December 2015 which analyze the predictive capability of search-based techniques for ascertaining four predominant software quality attributes, i.e., effort, defect proneness, maintainability and change proneness. The review analyses the effective use and application of search-based techniques by evaluating appropriate specifications of fitness functions, parameter settings, validation methods, accounting for their stochastic natures and the evaluation of developmental models with the use of well-known statistical tests. Furthermore, we compare the effectiveness of different models, developed using the various search-based techniques amongst themselves, and also with the prevalent machine learning techniques used in literature. Although there are very few studies which use search-based techniques for predicting maintainability and change proneness, we found that the results of the application of search-based techniques for effort estimation and defect prediction are encouraging. Hence, this comprehensive study and the associated results will provide guidelines to practitioners and researchers and will enable them to make proper choices for applying the search-based techniques to their specific situations.

The experimental applications of search-based techniques for model-based testing: Taxonomy and systematic literature review

ContextModel-based testing (MBT) aims to generate executable test cases from behavioral models of software systems. MBT gains interest in industry and academia due to its provision of systematic, automated, and comprehensive testing. Researchers have successfully applied search-based techniques (SBTs) by automating the search for an optimal set of test cases at reasonable cost compared to other more expensive techniques. Thus, there is a recent surge toward the applications of SBTs for MBT because the generated test cases are optimal and have low computational cost. However, successful, future SBTs for MBT applications demand deep insight into its existing experimental applications that underlines stringent issues and challenges, which is lacking in the literature. ObjectiveThe objective of this study is to comprehensively analyze the current state-of-the-art of the experimental applications of SBTs for MBT and present the limitations of the current literature to direct future research. MethodWe conducted a systematic literature review (SLR) using 72 experimental papers from six data sources. We proposed a taxonomy based on the literature to categorize the characteristics of the current applications. ResultsThe results indicate that the majority of the existing applications of SBTs for MBT focus on functional and structural coverage purposes, as opposed to stress testing, regression testing and graphical user interface (GUI) testing. We found research gaps in the existing applications in five areas: applying multi-objective SBTs, proposing hybrid techniques, handling complex constraints, addressing data and requirement-based adequacy criteria, and adapting landscape visualization. Only twelve studies proposed and empirically evaluated the SBTs for complex systems in MBT. ConclusionThis extensive systematic analysis of the existing literature based on the proposed taxonomy enables to assist researchers in exploring the existing research efforts and reveal the limitations that need additional investigation.

An exploratory study for software change prediction in object-oriented systems using hybridized techniques

Variation in software requirements, technological upgrade and occurrence of defects necessitate change in software for its effective use. Early detection of those classes of a software which are prone to change is critical for software developers and project managers as it can aid in efficient resource allocation of limited resources. Moreover, change prone classes should be efficiently restructured and designed to prevent introduction of defects. Recently, use of search based techniques and their hybridized counter-parts have been advocated in the field of software engineering predictive modeling as these techniques help in identification of optimal solutions for a specific problem by testing the goodness of a number of possible solutions. In this paper, we propose a novel approach for change prediction using search-based techniques and hybridized techniques. Further, we address the following issues: (i) low repeatability of empirical studies, (ii) less use of statistical tests for comparing the effectiveness of models, and (iii) non-assessment of trade-off between runtime and predictive performance of various techniques. This paper presents an empirical validation of search-based techniques and their hybridized versions, which yields unbiased, accurate and repeatable results. The study analyzes and compares the predictive performance of five search-based, five hybridized techniques and four widely used machine learning techniques and a statistical technique for predicting change prone classes in six application packages of a popular operating system for mobile--Android. The results of the study advocate the use of hybridized techniques for developing models to identify change prone classes.

Subtle higher order mutants

Context: Research has shown the majority of real faults are complex and cannot be simulated with traditional First Order Mutants (FOMs). Higher Order Mutants (HOMs), which are created by making multiple syntactic changes to the source code (i.e, combining multiple FOMs), can potentially be used to simulate realistic faults.Objective: Our goal is to find subtle HOMs, which we define to be HOMs that are not killed by a reference test suite that kills all the FOMs. Subtle HOMs represent cases where single faults interact by masking each other to produce complex faulty behavior. The fault-detection effectiveness of a test suite can be improved by adding test cases that detect subtle HOMs.Method: We developed six search-based techniques to find subtle HOMs: Genetic Algorithm, Local Search, Data-Interaction Guided Local Search, Test-Case Guided Local Search, Restricted Random Search and Restricted Enumeration. We then performed empirical studies to evaluate the ability of the search techniques to find subtle HOMs and investigated factors that impact the characteristics of the subtle HOMs.Results: Local Search and both the Guided Local Search techniques were more effective than the other techniques at finding subtle HOMs. The majority of subtle HOMs that were found resulted from combining mutated Java primitive operators.Conclusions: The proposed techniques are able to find subtle HOMs. Composing subtle HOMs of lower degrees is an effective way for finding new subtle HOMs of higher degrees.

Search-Based Test Data Generator for Data-Flow Dependencies Using Dominance Concepts, Branch Distance and Elitism

Software testing is a complex and expensive activity of the software development life cycle. Software testing includes test data generation according to a test adequacy criterion. The use of search-based techniques has been the focus of researchers to automate the process of software test data generation for structural control-flow criteria. Automating test data generation remains a challenging problem for more robust adequacy criterion such as satisfying data-flow dependencies of a program. This study proposes a search-based approach that generates test data for data-flow dependencies of a program using dominance concepts, branch distance, and elitism. Genetic algorithm is used for the proposed approach and Gray encoding is used to encode test data. A set of subject programs is taken from the research literature to evaluate efficiency and effectiveness of the proposed approach. For the proposed approach, the measures considered are the mean number of generations and mean percentage coverage achieved. The performance of the proposed approach is evaluated by comparing the results with those of random search and earlier studies on data-flow testing. Over several experiments, it is shown that the proposed approach performed significantly better than random search and earlier studies with respect to data-flow test data generation and optimization. There is an increasing performance gap for more complex subject programs.

Projections onto convex sets parameter estimation through harmony search and its application for image restoration

Image restoration is a research field that attempts to recover a blurred and noisy image. Although we have one-step algorithms that are often fast for image restoration, iterative formulations allow a better control of the trade-off between the enhancement of high frequencies (image details) and noise amplification. Projections onto convex sets (POCS) is an iterative--and parametric-based approach that employs a priori knowledge about the blurred image to guide the restoration process, with promising results in different application domains. However, a proper choice of its parameters is a high computational burden task, since they are continuous-valued and there are an infinity of possible values to be checked. In this paper, we propose to optimize POCS parameters by means of harmony search-based techniques, since they provide elegant and simple formulations for optimization problems. The proposed approach has been validated in synthetic and real images, being able to select suitable parameters in a reasonable amount of time.

Subdomain-based test data generation

Considerable effort is required to test software thoroughly. Even with automated test data generation tools, it is still necessary to evaluate the output of each test case and identify unexpected results. Manual effort can be reduced by restricting the range of inputs testers need to consider to regions that are more likely to reveal faults, thus reducing the number of test cases overall, and therefore reducing the effort needed to create oracles. This article describes and evaluates search-based techniques, using evolution strategies and subset selection, for identifying regions of the input domain (known as subdomains) such that test cases sampled at random from within these regions can be used efficiently to find faults. The fault finding capability of each subdomain is evaluated using mutation analysis, a technique that is based on faults programmers are likely to make. The resulting subdomains kill more mutants than random testing (up to six times as many in one case) with the same number or fewer test cases. Optimised subdomains can be used as a starting point for program analysis and regression testing. They can easily be comprehended by a human test engineer, so may be used to provide information about the software under test and design further highly efficient test suites.

An evolutionary algorithm for the discovery of rare class association rules in learning management systems

Association rule mining, an important data mining technique, has been widely focused on the extraction of frequent patterns. Nevertheless, in some application domains it is interesting to discover patterns that do not frequently occur, even when they are strongly related. More specifically, this type of relation can be very appropriate in e-learning domains due to its intrinsic imbalanced nature. In these domains, the aim is to discover a small but interesting and useful set of rules that could barely be extracted by traditional algorithms founded in exhaustive search-based techniques. In this paper, we propose an evolutionary algorithm for mining rare class association rules when gathering student usage data from a Moodle system. We analyse how the use of different parameters of the algorithm determine the rule characteristics, and provides some illustrative examples of them to show their interpretability and usefulness in e-learning environments. We also compare our approach to other existing algorithms for mining both rare and frequent association rules. Finally, an analysis of the rules mined is presented, which allows information about students' unusual behaviour regarding the achievement of bad or good marks to be discovered.

An assessment of search-based techniques for reverse engineering feature models

Successful software evolves from a single system by adding and changing functionality to keep up with users’ demands and to cater to their similar and different requirements. Nowadays it is a common practice to offer a system in many variants such as community, professional, or academic editions. Each variant provides different functionality described in terms of features. Software Product Line Engineering (SPLE) is an effective software development paradigm for this scenario. At the core of SPLE is variability modelling whose goal is to represent the combinations of features that distinguish the system variants using feature models, the de facto standard for such task. As SPLE practices are becoming more pervasive, reverse engineering feature models from the feature descriptions of each individual variant has become an active research subject. In this paper we evaluated, for this reverse engineering task, three standard search based techniques (evolutionary algorithms, hill climbing, and random search) with two objective functions on 74 SPLs. We compared their performance using precision and recall, and found a clear trade-off between these two metrics which we further reified into a third objective function based on Fβ, an information retrieval measure, that showed a clear performance improvement. We believe that this work sheds light on the great potential of search-based techniques for SPLE tasks.

Prediction of faults-slip-through in large software projects: an empirical evaluation

A large percentage of the cost of rework can be avoided by finding more faults earlier in a software test process. Therefore, determination of which software test phases to focus improvement work on has considerable industrial interest. We evaluate a number of prediction techniques for predicting the number of faults slipping through to unit, function, integration, and system test phases of a large industrial project. The objective is to quantify improvement potential in different test phases by striving toward finding the faults in the right phase. The results show that a range of techniques are found to be useful in predicting the number of faults slipping through to the four test phases; however, the group of search-based techniques (genetic programming, gene expression programming, artificial immune recognition system, and particle swarm optimization---based artificial neural network) consistently give better predictions, having a representation at all of the test phases. Human predictions are consistently better at two of the four test phases. We conclude that the human predictions regarding the number of faults slipping through to various test phases can be well supported by the use of search-based techniques. A combination of human and an automated search mechanism (such as any of the search-based techniques) has the potential to provide improved prediction results.

Integrating Collaborative Filtering and Matching-based Search for Product Recommendations

Currently, recommender systems (RS) have been widely applied in many commercial e-commerce sites to help users deal with the information overload problem. Recommender systems provide personalized recommendations to users and, thus, help in making good decisions about which product to buy from the vast amount of product choices. Many of the current recommender systems are developed for simple and frequently purchased products like books and videos, by using collaborative-filtering and content-based approaches. These approaches are not directly applicable for recommending infrequently purchased products such as cars and houses as it is difficult to collect a large number of ratings data from users for such products. Many of the ecommerce sites for infrequently purchased products are still using basic search-based techniques whereby the products that match with the attributes given in the target user's query are retrieved and recommended. However, search-based recommenders cannot provide personalized recommendations. For different users, the recommendations will be the same if they provide the same query regardless of any difference in their interest. In this article, a simple user profiling approach is proposed to generate user's preferences to product attributes (i.e., user profiles) based on user product click stream data. The user profiles can be used to find similarminded users (i.e., neighbours) accurately. Two recommendation approaches are proposed, namely Round-Robin fusion algorithm (CFRRobin) and Collaborative Filtering-based Aggregated Query algorithm (CFAgQuery), to generate personalized recommendations based on the user profiles. Instead of using the target user's query to search for products as normal search based systems do, the CFRRobin technique uses the attributes of the products in which the target user's neighbours have shown interest as queries to retrieve relevant products, and then recommends to the target user a list of products by merging and ranking the returned products using the Round Robin method. The CFAgQuery technique uses the attributes of the products that the user's neighbours have shown interest in to derive an aggregated query, which is then used to retrieve products to recommend to the target user. Experiments conducted on a real e-commerce dataset show that both the proposed techniques CFRRobin and CFAgQuery perform better than the standard Collaborative Filtering and the Basic Search approaches, which are widely applied by the current e-commerce applications.

Development of a computer program for the path generation of tractor hitch points

In the vertical longitudinal plane of a tractor, the three-point hitch system is a six-bar mechanism that can be designed and modelled as two distinct four-bar linkages sharing two links. A Newton–Raphson solution for the four-bar linkage has been applied to a three-point hitch system of tractor to determine the range of movement of the lift arm and generate the path of motion of lower and upper hitch points in the movement range of the hitch. The algorithm was executed as a program in MATLAB. The program has the ability to generate the path of motion of lower; upper and virtual hitch points, determine the geometric performance parameters of the hitch system, and identify the variations in the quality of force transmission by the linkage system throughout the range of hitch movements for 165 different groups of hitch dimensions available in the Nebraska tractor test reports. Application of the program for searching improved design solutions for the existing hitch system of a tractor was demonstrated. The program has the potential to be used in any search-based techniques for optimisation of the dimensions of three-point hitch systems for tractors. In addition, variations in length of links and spatial location of their pivot points were studied for various categories of hitches. The range of values of link lengths and the spatial location of their pivot points reported here can be used for defining the design space during the optimal dimensional synthesis of tractor three-point hitch systems.

Design-Oriented Analysis of Circuits With Equality Constraints

This paper presents a design-oriented circuit analysis that is augmented with design constraints. This analysis computes the circuit response and also finds the values of circuit parameters (equal to the number of design specifications) that result in a specified circuit performance. An application of this approach is demonstrated for the periodic steady-state analysis with shooting and finite difference formulations. The new analysis with design equality constraints is several times faster than search-based techniques that employ conventional analysis methods.

MIJ2K Optimization using evolutionary multiobjective optimization algorithms

This paper deals with the multiobjective definition of video compression and its optimization. The optimization will be done using NSGA-II, a well-tested and highly accurate algorithm with a high convergence speed developed for solving multiobjective problems. Video compression is defined as a problem including two competing objectives. We try to find a set of optimal, so-called Pareto-optimal solutions, instead of a single optimal solution. The two competing objectives are quality and compression ratio maximization. The optimization will be achieved using a new patent pending codec, called MIJ2K, also outlined in this paper. Video will be compressed with the MIJ2K codec applied to some classical videos used for performance measurement, selected from the Xiph.org Foundation repository. The result of the optimization will be a set of near-optimal encoder parameters. We also present the convergence of NSGA-II with different encoder parameters and discuss the suitability of MOEAs as opposed to classical search-based techniques in this field.

Special Issue on Search‐Based Software Maintenance

Search-based software engineering (SBSE) consists of the application of metaheuristic search techniques—such as genetic algorithms, simulated annealing, hill climbing, and tabu search—to software engineering problems. This idea is based on the observation that some software engineering activities can be thought of as optimization problems, and on the fact that techniques providing exact solutions are, very often, unable to adequately scale to large, real-world software engineering problems. Define fitness functions to guide an automated search for solutions, rather than attempting to construct solutions from scratch by hand. SBSE moves the engineer up the abstraction chain, from the tedious and expensive role of detailed solution construction, to the potentially more productive and more cost-effective area of fitness function design. This approach has been repeatedly demonstrated to have successful applications across the software engineering life cycle from requirements and project planning to maintenance and testing. This special issue focuses on applications of search SBSE to the maintenance phase of the life cycle. After software testing 2, software maintenance is the most widely studied application area for SBSE research. Work on SBSE applications to problems in software maintenance can be traced back to the pioneering work of the Mancoridis et al. 3, which led to the development of the Bunch tool for software re-modularization 4. In the past five years there has been a significant increase in SBSE applications to software maintenance and evolution. Search-based techniques have been applied to post-development code improvement problems, such as object-oriented refactoring 5, 6, and imperative language slicing and transformation 7. Other applications are related to software engineering economics and project management problems, such as release planning 8-12, effort, and cost estimation 13-16, and project staffing and planning 17-20. Last, but not the least, SBSE has been applied to deal with service-oriented architectures, for example for services QoS-aware composition 21, service-level agreement negotiation 22, and testing 23. A complete overview of SBSE with a discussion of future research trends can be found in the recent International Conference on Software Engineering (ICSE) ‘Future of Software Engineering’ paper 24. This issue of the Journal of Software Maintenance and Evolution: Research and Practice features three outstanding papers, dealing with different aspects of software maintenance and evolution: release planning and staffing, re-factoring, and component substitution. The paper ‘Optimized Staffing for Product Releases—Focused Search and its Application at Chartwell Technologies’ by Puneet Kapur, An Ngo-The, Günther Ruhe, and Andrew Smith presents an approach for optimizing the staffing for product releases. The problem they solve can be viewed as part of the more general release planning problem, which assigns features to releases such that technical, resource, risk, and budget constraints are met. In particular, the genetic search-based approach the authors propose aims at assigning human resources to software development tasks, taking them from a pool of developers having different levels of skills. The applicability of the proposed approach is demonstrated with an industrial case study. The paper ‘Search-Based Refactoring: an Empirical Study’ by Mark O'Keffe and Mel Ó Cinnéide presents an approach to automatically re-factor, using search-based techniques, object-oriented systems that undergo repeated addition of functionality. Such addition can, very often, deteriorate the quality of the system's underlying design. Their re-factoring is guided by the QMOOD quality model 25, and the paper compares different optimization techniques, such as Genetic Algorithms, Simulated Annealing, and Multiple Ascent Hill-Climbing. The results of experiments carried out on five software systems suggest that the latter technique outperforms the others. The paper ‘Search-based Many-to-one Component Substitution’ by Nicolas Desnos, Marianne Huchard, Guy Tremblay, Christelle Urtado, and Sylvain Vauttier presents a search-based approach that allows the replacement of an obsolete component with a composition of multiple components. When a component becomes obsolete, the approach searches for a minimal assembly of other components, exposing their functionality by means of ports that preserves the behavior of the original component. The proposed approach has been evaluated on a set of randomly generated components. We hope that readers will enjoy this special issue and find the selected papers insightful and inspiring. We would like to thank all the authors who presented papers for this special issue: there were many good quality papers, but unfortunately we could accept only three of them. Last, but not the least, we would like to thank the reviewers who provided timely, detailed and constructive feedbacks to the authors. Without their precious help this special issue would not have been possible.

A maximum a posteriori MIMO detector using recursive metric computations

A complexity reduction technique for soft-output maximum a posteriori probability detection in wireless multiple-input multiple-output (MIMO) channels using rectangular quadrature amplitude modulation (QAM) constellations is derived. In contrast to existing low-complexity detection schemes that reduce the number of candidate tests, we instead focus on lowering the computational burden per candidate. The search over a large number of hypotheses to generate output L-values is facilitated by a doubly recursive mechanism, exploiting differential updates of the distance metrics from candidate to candidate. Moreover, the search over all transmit candidates is simplified using a hierarchical "triangular" (zig-zag) shaped candidate scanning order. The technique is derived conceptually using mathematical terms, and then illustrated by means of a 2 times 2 MIMO system using 64-QAM modulation. Under certain practical assumptions, the new scheme is shown to reduce the overall number of operations per candidate by a factor of about 50-100. While the concept is demonstrated by means of the exhaustive search case, it can be combined with other search-based MIMO detection techniques such as spherical decoding

A model-based adaptive motion estimation scheme using Renyi’s entropy for wireless video

Efficient motion estimation is an important problem because it determines the compression efficiency and complexity of a video encoder. Motion estimation can be formulated as an optimization problem; most motion estimation algorithms use mean squared error, sum of absolute differences or maximum a posteriori probability as the optimization criterion and apply search-based techniques (e.g., exhaustive search or three-step search) to find the optimum motion vector. However, most of these algorithms do not effectively utilize the knowledge gained in the search process for future search efforts and hence are computationally inefficient. This paper addresses this inefficiency problem by introducing an adaptive motion estimation scheme that substantially reduces computational complexity while yet providing comparable compression efficiency, as compared to existing fast-search methods. Our approach is motivated by the recent developments in using Renyi’s entropy as the optimization criterion for system modeling [D. Erdogmus, J. Principe, Comparison of entropy and mean square error criteria in adaptive system training using higher order, in: Proceedings of the International Workshop on Independent Component Analysis and Signal Separation, Helsinki, Finland, 2000, pp. 75–80]. This scheme is particularly suited for wireless video sensor networks, video conferencing systems and live streaming videos which have stringent computational requirements. Our results show that our scheme reduces the computational complexity by a factor of 9–21, compared to the existing fast algorithms.

Search-based software engineering

This paper claims that a new field of software engineering research and practice is emerging: search-based software engineering. The paper argues that software engineering is ideal for the application of metaheuristic search techniques, such as genetic algorithms, simulated annealing and tabu search. Such search-based techniques could provide solutions to the difficult problems of balancing competing (and some times inconsistent) constraints and may suggest ways of finding acceptable solutions in situations where perfect solutions are either theoretically impossible or practically infeasible. In order to develop the field of search-based software engineering, a reformulation of classic software engineering problems as search problems is required. The paper briefly sets out key ingredients for successful reformulation and evaluation criteria for search-based software engineering.

The SEMINAL workshop

This paper reports on the first international Workshop on Software Engineering using Metaheuristic INnovative ALgorithms.The aim of the workshop was to bring together researchers in search-based metaheuristic techniques with researchers and practitioners in Software Engineering. The workshop sought to support and develop the embryonic community which straddles these two communities and which is working on the application of metaheuristic search-based techniques to problems in Software Engineering.The paper outlines the nature of the nascent field of Search-Based Software Engineering, and briefly outlines the papers presented at the workshop and the discussions which took place.