Multi-objective Genetic Programming Research Articles

MULTICR: Predicting Merged and Abandoned Code Changes in Modern Code Review Using Multi-Objective Search

Modern Code Review (MCR) is an essential process in software development to ensure high-quality code. However, developers often spend considerable time reviewing code changes before being merged into the main code base. Previous studies attempted to predict whether a code change was going to be merged or abandoned soon after it was submitted to improve the code review process. However, these approaches require complex cost-sensitive learning, which makes their adoption challenging since it is difficult for developers to understand the main factors behind the models’ predictions. To address this issue, we introduce in this paper, MULTICR, a multi-objective search-based approach that uses Multi-Objective Genetic Programming (MOGP) to learn early code review prediction models as IF-THEN rules. MULTICR evolves predictive models while maximizing the accuracy of both merged and abandoned classes, eliminating the need for misclassification cost estimation. To evaluate MULTICR, we conducted an empirical study on 146,612 code reviews from Eclipse, LibreOffice, and Gerrithub. The obtained results show that MULTICR outperforms existing baselines in terms of Mathew correlation coefficient (MCC) and F1 scores while learning less complex models compared to decision trees. Our experiments showed also how MULTICR allows identifying the main factors related to abandoned code reviews as well as their associated thresholds, making it a promising approach for early code review prediction with notable performance and interoperability. Additionally, we qualitatively evaluate MULTICR by conducting a user study through semi-structured interviews involving 10 practitioners from different organizations. The obtained results indicate that 90% of the participants find that MULTICR is useful and can help them to improve the code review process. Additionally, the learned IF-THEN rules of MULTICR are transparent.

Analysis and optimization of pre-stressed modal features of ship anchor support parts

In order to solve the problems of excessive weight and unreasonable structure of anchor machine parts caused by traditional design methods, a lightweight optimization method was proposed based on pre-stressed modal analysis. The design variables were determined, and the parameterized model was established by using ANSYS Workbench. Under ultimate load conditions, the strength of wall frame board and lower box bodies was simulated and calculated. Through modal analysis, the discretized natural frequencies under different design variables could be obtained. The multi-objective genetic algorithm and sequence quadratic programming were respectively used to calculate the lightweight analysis model. The results showed that the weight of the supporting components in ship anchor can be reduced by more than 5 % without reducing strength and equivalent stiffness.

Ensemble classifiers using multi-objective Genetic Programming for unbalanced data

Genetic Programming (GP) can be used to design effective classifiers due to its built-in feature selection and feature construction characteristics. Unbalanced data distributions affect the classification performance of GP classifiers. Some fitness functions have been proposed to solve the class imbalance problem of GP classifiers. However, with the evolution of GP, single-objective GP classifiers evaluated by a single fitness function have poor generalization ability. Moreover, using the best evolved GP classifier for decision-making can easily lead to the possibility of misclassification. In this paper, multi-objective GP is used to optimize multiple fitness functions including AUC approximation (Wmw), Distance (Dist), and Complexity to evolve ensemble classifiers, which jointly determines the class labels of unknown instances. Experiments on sixteen datasets show that our multi-objective GP can significantly improve classification performance compared with single-objective GP, and our proposed ensemble classifiers evolved by multi-objective GP can further improve the classification performance than the single best GP classifier. Comparisons with six GP-based and five traditional machine learning algorithms show that our proposed approaches can achieve significantly better classification performance on most cases.

Evolutionary modeling approach based on multiobjective genetic programming for strip quality prediction

In the iron and steel industry, hardness is one of the key indicators of strip quality in the continuous annealing production line (CAPL). However, the complex production process and the strong coupled nonlinearity between process parameters make it difficult to develop accurate mechanism models and pose a challenge for data-driven modeling approaches. More importantly, most of the data-driven learning methods lack interpretability and cannot characterize the mathematical relationship between process parameters and product quality, which in turn makes it extremely hard to understand the process mechanism. Therefore, this paper proposes an interpretable modeling approach (IMA) based on feature decomposition and ensemble to construct interpretable analytical models between process parameters and strip quality. In the IMA, a data-mechanism fusion-based feature decomposition (DM_FD) method is first applied to cope with high-dimensional input feature problems. Then, an improved multiobjective genetic programming algorithm (iMOGP) is developed to construct interpretability sub-models. Finally, a sparse optimization ensemble method (SOE) is used to integrate the sub-models to achieve interpretability and good generalization. Experimental results based on practical strip data demonstrate that the proposed IMA can cope well with high-dimensional input features and achieve model interpretability compared with commonly used machine learning methods and genetic programming (GP)-based modeling methods while ensuring better accuracy and generalization.

A Novel Multiobjective Genetic Programming Approach to High-Dimensional Data Classification.

The development of data sensing technology has generated a vast amount of high-dimensional data, posing great challenges for machine learning models. Over the past decades, despite demonstrating its effectiveness in data classification, genetic programming (GP) has still encountered three major challenges when dealing with high-dimensional data: 1) solution diversity; 2) multiclass imbalance; and 3) large feature space. In this article, we have developed a problem-specific multiobjective GP framework (PS-MOGP) for handling classification tasks with high-dimensional data. To reduce the large solution space caused by high dimensionality, we incorporate the recursive feature elimination strategy based on mining the archive of evolved GP solutions. A progressive domination Pareto archive evolution strategy (PD-PAES), which optimizes the objectives in a specific order according to their objectives, is proposed to evaluate the GP individuals and maintain a better diversity of solutions. Besides, to address the seriously imbalanced class issue caused by traditional binary decomposition (BD) one versus rest (OVR) for multiclass classification problems, we design a method named BD with a similar positive and negative class size (BD-SPNCS) to generate a set of auxiliary classifiers. Experimental results on benchmark and real-world datasets demonstrate that our proposed PS-MOGP outperforms state-of-the-art traditional and evolutionary classification methods in the context of high-dimensional data classification.

A Multi-Objective Genetic Programming Algorithm With α Dominance and Archive for Uncertain Capacitated Arc Routing Problem

The uncertain capacitated arc routing problem is an important combinatorial optimization problem with many applications in the real world. Genetic programming hyper heuristic has been successfully used to automatically evolve routing policies, which can make real-time routing decisions for uncertain capacitated arc routing problems. It is desired to evolve routing policies that are both effective and small/simple to be easily understood. The effectiveness and size are two potentially conflicting objectives. A further challenge is the objective selection bias issue, i.e., it is much more likely to obtain small but ineffective routing policies than the effective ones that are typically large. In this paper, we propose a new multi-objective genetic programming algorithm to evolve effective and small routing policies. The new algorithm employs the α dominance strategy with a newly proposed α adaptation scheme to address the objective selection bias issue. In addition, it contains a new archive strategy to prevent the loss of promising individuals due to the rotation of training instances. The experimental results showed that the newly proposed algorithm can evolve significantly better routing policies than the current state-of-the-art algorithms for uncertain capacitated arc routing problem in terms of both effectiveness and size. We have also analysed the evolved routing policies to show better interpretability.

Multi-objective genetic programming strategies for topic-based search with a focus on diversity and global recall

Topic-based search systems retrieve items by contextualizing the information seeking process on a topic of interest to the user. A key issue in topic-based search of text resources is how to automatically generate multiple queries that reflect the topic of interest in such a way that precision, recall, and diversity are achieved. The problem of generating topic-based queries can be effectively addressed by Multi-Objective Evolutionary Algorithms, which have shown promising results. However, two common problems with such an approach are loss of diversity and low global recall when combining results from multiple queries. This work proposes a family of Multi-Objective Genetic Programming strategies based on objective functions that attempt to maximize precision and recall while minimizing the similarity among the retrieved results. To this end, we define three novel objective functions based on result set similarity and on the information theoretic notion of entropy. Extensive experiments allow us to conclude that while the proposed strategies significantly improve precision after a few generations, only some of them are able to maintain or improve global recall. A comparative analysis against previous strategies based on Multi-Objective Evolutionary Algorithms, indicates that the proposed approach is superior in terms of precision and global recall. Furthermore, when compared to query-term-selection methods based on existing state-of-the-art term-weighting schemes, the presented Multi-Objective Genetic Programming strategies demonstrate significantly higher levels of precision, recall, and F1-score, while maintaining competitive global recall. Finally, we identify the strengths and limitations of the strategies and conclude that the choice of objectives to be maximized or minimized should be guided by the application at hand.

Automatic Design of Dynamic Collaboration Strategies for Machines and Automated Guided Vehicles via Multiobjective Genetic Programming

In the flexible manufacturing system (FMS), the automated guided vehicles (AGVs) have been widely applied to the material logistics. The transporting phases of AGVs and the processing phases of machines are alternately executed and form the production flow. The two kinds of phases will both influence the completing time and cause energy consumption and are difficult to decouple. Therefore, in this paper, we focus on the dynamic collaboration problem between processing machines and AGVs (DCPMA) and establish a multiobjective optimization model to minimize the makespan and the energy consumption of FMS. In order to solve DCPMA, we propose a novel genetic programming (GP) to evolve collaboration strategies. In GP, 10 status statistics related to the handling time and energy consumption are selected into GP terminal set to express the GP tree. During dynamic simulation, each collaboration strategy evaluated by GP will dynamically select the job-machine-AGV scheme combination with the highest priority calculated from the GP tree. In addition, a series of generation operators and selection operators are customized for DCPMA. Finally, the training and testing results show that the proposed GP is superior to 28 combinations of basic collaboration strategies, and has better adaptability and scalability for various scenarios.

A New Multi-Objective Genetic Programming Model for Meteorological Drought Forecasting

Drought forecasting is a vital task for sustainable development and water resource management. Emerging machine learning techniques could be used to develop precise drought forecasting models. However, they need to be explicit and simple enough to secure their implementation in practice. This article introduces a novel explicit model, called multi-objective multi-gene genetic programming (MOMGGP), for meteorological drought forecasting that addresses both the accuracy and simplicity of the model applied. The proposed model considers two objective functions: (i) root mean square error and (ii) expressional complexity during its evolution. While the former is used to increase the model accuracy at the training phase, the latter is assigned to decrease the model complexity and achieve parsimony conditions. The model evolution and verification procedure were demonstrated using the standardized precipitation index obtained for Burdur City, Turkey. The comparison with benchmark genetic programming (GP) and multi-gene genetic programming (MGGP) models showed that MOMGGP provides the same forecasting accuracy with more parsimony conditions. Thus, it is suggested to utilize the model for practical meteorological drought forecasting.

Shear strength assessment of reinforced recycled aggregate concrete beams without stirrups using soft computing techniques

This paper presents a study to predict the shear strength of reinforced recycled aggregate concrete beams without stirrups using soft computing techniques. The methodology involves the development of a Multi-Objective Genetic Algorithm Evolutionary Polynomial Regression (MOGA-EPR) and Gene Expression Programming (GEP) models. The input variables considered are the longitudinal reinforcement ratio, recycled coarse aggregate ratio, beam cross-section dimensions, and concrete compressive strength. Data collected from the literature were used to train and validate the models. The results showed that the MOGA-EPR and GEP models can accurately predict the shear strength of beams without stirrups. The models also performed better than equations from the codes and literature. This study provides an alternative approach to accurately predict the shear strength of reinforced recycled aggregate concrete beams without stirrups.

Automated pattern generation for swarm robots using constrained multi-objective genetic programming

Swarm robotic systems (SRSs), which are widely used in many fields, such as search and rescue, usually comprise a number of robots with relatively simple mechanisms collaborating to accomplish complex tasks. A challenging task for SRSs is to design local interaction rules for self-organization of robots that can generate adaptive patterns to entrap moving targets. Biologically inspired approaches such as gene regulatory network (GRN) models provide a promising solution to this problem. However, the design of GRN models for generating entrapping patterns relies on the expertise of designers. As a result, the design of the GRN models is often a laborious and tedious trial-and-error process. In this study, we propose a modular design automation framework for GRN models that can generate entrapping patterns. The framework employs basic network motifs to construct GRN models automatically without requiring expertise. To this end, a constrained multi-objective genetic programming is utilized to simultaneously optimize the structures and parameters of the GRN models. A multi-criteria decision-making approach is adopted to choose the preferred GRN model for generating the entrapping pattern. Comprehensive simulation results demonstrate that the proposed framework can obtain novel GRN models with simpler structures than those designed by human experts yet better performance in complex and dynamic environments. Proof-of-concept experiments using e-puck robots confirmed the feasibility and effectiveness of the proposed GRN models.

Application of soft computing in predicting the compressive strength of self-compacted concrete containing recyclable aggregate

AbstractSelf-compacting concrete (SCC) is a type of concrete known for its environmental benefits and improved workability. In this study, data-driven approaches were used to anticipate the compressive strength (CS) of self-compacting concrete (SCC) containing recycled plastic aggregates (RPA). A database of 400 experimental data sets was used to assess the capabilities of multi-objective genetic algorithm evolutionary polynomial regression (MOGA-EPR) and gene expression programming (GEP). The analysis results indicated that the proposed equations provided more accurate CS predictions than traditional approaches such as the linear regression model (LRM). The proposed equations achieved lower mean absolute error (MAE) and root mean square error (RMSE) values, a mean close to the optimum value (1.0), and a higher coefficient of determination (R2) than the LRM. As such, the proposed approaches can be utilized to obtain more reliable design calculations and better predictions of CS in SCC incorporating RPA.

Multitask Multiobjective Genetic Programming for Automated Scheduling Heuristic Learning in Dynamic Flexible Job-Shop Scheduling.

Evolutionary multitask multiobjective learning has been widely used for handling more than one multiobjective task simultaneously. However, it is rarely used in dynamic combinatorial optimization problems, which have valuable practical applications such as dynamic flexible job-shop scheduling (DFJSS) in manufacturing. Genetic programming (GP), as a popular hyperheuristic approach, has been used to learn scheduling heuristics for generating schedules for multitask single-objective DFJSS only. Searching in the heuristic space with GP is more difficult than in the solution space, since a small change on heuristics can lead to ineffective or even infeasible solutions. Multiobjective DFJSS is more challenging than single DFJSS, since a scheduling heuristic needs to cope with multiple objectives. To tackle this challenge, we first propose a multipopulation-based multitask multiobjective GP algorithm to preserve the quality of the learned scheduling heuristics for each task. Furthermore, we develop a multitask multiobjective GP algorithm with a task-oriented knowledge-sharing strategy to further improve the effectiveness of learning scheduling heuristics for DFJSS. The results show that the designed multipopulation-based GP algorithms, especially the one with the task-oriented knowledge-sharing strategy, can achieve good performance for all the examined tasks by maintaining the quality and diversity of individuals for corresponding tasks well. The learned Pareto fronts also show that the GP algorithm with task-oriented knowledge-sharing strategy can learn competitive scheduling heuristics for DFJSS on both of the objectives.

BPEL process defects prediction using multi-objective evolutionary search

Web services are becoming increasingly popular technologies for modern organizations to improve their cooperation and collaboration through building new software systems by composing pre-built services. Such services are typically composed and executed through BPEL (Business Process Execution Language) processes. Like any other software artifact, such processes are frequently changed to add new or modify existing functionalities or adapt to environmental changes. However, poorly planned changes may introduce BPEL process design defects known as anti-patterns or defects. The presence of defects often leads to a regression in software quality. In this paper, we introduce an automated approach to predict the presence of defects in BPEL code using Multi-Objective Genetic Programming (MOGP). Our approach consists of learning from real-world instances of each service-based business process defect (i.e., anti-pattern) type to infer prediction rules based on the combinations of process metrics and their associated threshold values. We evaluate our approach based on a dataset of 178 real-world business processes that belong to various application domains, and a variety of BPEL process defect types such as data flow and portability defects. The statistical analysis of the achieved results shows the effectiveness of our approach in identifying defects compared with state-of-the-art techniques with a median accuracy of 91%.

Genetic programming for the prediction of berm breakwaters recession

The response of berm breakwaters to wave forces has been examined with rebuild and cumulative experiments. In rebuild experiments, the breakwaters were reconstructed after each test, whereas in cumulative experiments the structural damages were examined at the end of the experiment. This study presents a new method to investigate the berm breakwaters recession considering datasets collected of both types of experiments. Cumulative experimental results were converted to their equivalent rebuild experimental results by modifying the number of waves for the reported damage. After homogenizing the data, the datasets were divided into the train, validation, and test subsets. The data were analyzed using the Multi-Objective Genetic Programming (MOGP) approach, and a prediction model was created to evaluate the berm breakwater recession. The results obtained from the MOGP model were compared to outcomes computed using implicit formulas available in the literature showing that the MOGP model is accurate (R2 = 0.911 and RMSE = 0.111) with a relatively broader applicability range. The impact of each input parameter on the berm breakwater recession was examined using parametric and sensitivity analyses. The stability number was the most important parameter impacting the damage on the coastal structure. The results are in line with findings reported in previous studies.

Evolutionary convolutional neural network for image classification based on multi-objective genetic programming with leader–follower mechanism

As a popular research in the field of artificial intelligence in the last 2 years, evolutionary neural architecture search (ENAS) compensates the disadvantage that the construction of convolutional neural network (CNN) relies heavily on the prior knowledge of designers. Since its inception, a great deal of researches have been devoted to improving its associated theories, giving rise to many related algorithms with pretty good results. Considering that there are still some limitations in the existing algorithms, such as the fixed depth or width of the network, the pursuit of accuracy at the expense of computational resources, and the tendency to fall into local optimization. In this article, a multi-objective genetic programming algorithm with a leader–follower evolution mechanism (LF-MOGP) is proposed, where a flexible encoding strategy with variable length and width based on Cartesian genetic programming is designed to represent the topology of CNNs. Furthermore, the leader–follower evolution mechanism is proposed to guide the evolution of the algorithm, with the external archive set composed of non-dominated solutions acting as the leader and an elite population updated followed by the external archive acting as the follower. Which increases the speed of population convergence, guarantees the diversity of individuals, and greatly reduces the computational resources. The proposed LF-MOGP algorithm is evaluated on eight widely used image classification tasks and a real industrial task. Experimental results show that the proposed LF-MOGP is comparative with or even superior to 35 existing algorithms (including some state-of-the-art algorithms) in terms of classification error and number of parameters.

Shape-constrained multi-objective genetic programming for symbolic regression

We describe and analyze algorithms for shape-constrained symbolic regression, which allow the inclusion of prior knowledge about the shape of the regression function. This is relevant in many areas of engineering — in particular, when data-driven models, which are based on data of measurements must exhibit certain properties (e.g. positivity, monotonicity, or convexity/concavity). To satisfy these properties, we have extended multi-objective algorithms with shape constraints. A soft-penalty approach is used to minimize both the constraint violations and the prediction error. We use the non-dominated sorting genetic algorithm (NSGA-II) as well as the multi-objective evolutionary algorithm based on decomposition (MOEA/D). The algorithms are tested on a set of models from physics textbooks and compared against previous results achieved with single objective algorithms. Further, we generated out-of-domain samples to test the extrapolation behavior using shape constraints and added a different level of noise on the training data to verify if shape constraints can still help maintain the prediction errors to a minimum and generate valid models. The results showed that the multi-objective algorithms were capable of finding mostly valid models, also when using a soft-penalty approach. Further, we investigated that NSGA-II achieved the best overall ranks on high noise instances.

Evolving ensembles using multi-objective genetic programming for imbalanced classification

Multi-objective Genetic Programming (MGP) plays a prominent role in generating Pareto optimal classifier sets and making trade-offs among multiple classes adaptively. However, the existing MGP algorithms show poor performance and are difficult to implement when dealing with imbalanced classification problems. This work proposes a new MGP-based algorithm designed for imbalanced classification. Firstly, an efficient evolutionary strategy with nondominated sorting, environmental selection, and an archiving mechanism is designed to optimize the false positive rate, the false negative rate and reduce the size of the resulting tree. Then, a weighted ensemble decision is made according to each classifier’s performance in the majority and minority classes to obtain final classification results. Experimental results on 21 binary-class datasets and 17 multi-class datasets show that the proposed method outperforms existing ones in several commonly used imbalanced classification metrics.

Surrogate-assisted automatic evolving of dispatching rules for multi-objective dynamic job shop scheduling using genetic programming

Dispatching rules are simple but efficient heuristics to solve multi-objective job shop scheduling problems, particularly useful to face the challenges of dynamic shop environments. A promising method to automatically evolve non-dominated rules represents multi-objective genetic programming based hyper-heuristic (MO-GP-HH). The aim of such methods is to approximate the Pareto front of non-dominated dispatching rules as good as possible in order to provide a sufficient set of efficient solutions from which the decision maker can select the most preferred one. However, one of the main drawbacks of existing approaches is the computational demanding simulation-based fitness evaluation of the evolving rules. To efficiently allocate the computational budget, surrogate models can be employed to approximate the fitness. Two possible ways, that estimate the fitness either based on a simplified problem or based on samples of fully evaluated individuals making use of machine learning techniques are investigated in this paper. Several representatives of both categories are first examined with regard to their selection accuracy and execution time. Furthermore, we developed a surrogate-assisted MO-GP-HH framework, incorporating a pre-selection task in the NSGA-II algorithm. The most promising candidates are consequently implemented in the framework. Using a dynamic job shop scenario, the two proposed algorithms are compared to the original one without using surrogates. With the aim to minimize the mean flowtime and maximum tardiness, experimental results demonstrate that the proposed algorithms outperform the former. Making use of surrogates leads to a reduction in computational costs of up to 70%. Another interesting finding shows that the enhanced ability to identify duplicates based on the phenotypic characterization of individuals is particularly helpful in increasing diversity within a population. This study illustrates the positive effect of this mechanism on the exploration of the entire Pareto front.

Developing Equations for Free Vibration Parameters of Bistable Composite Plates Using Multi-Objective Genetic Programming.

For the last three decades, bistable composite laminates have gained publicity because of their outstanding features, including having two stable shapes and the ability to change these states. A common challenge regarding the analysis of these structures is the high computational cost of existing analytical methods to estimate their natural frequencies. In the current paper, a new methodology combining the Finite Element Method (FEM) and Multi-Objective Genetic Programming (MOGP) is proposed for the analysis of bistable composite structures, leading to some analytical relations derived to obtain the modal parameters of the shells. To achieve this aim, the data extracted from FEM, consisting of the ratio of the length to width (a/b) and the thickness (t) of the laminate, is split into Train and Validation, and Test, subsets. The former is used in MOGP, and four formulas are proposed for the prediction of the free vibration parameters of bistable laminates. The formulas are checked against the Test subset, and the statistical indices are calculated. An excellent performance is observed for all GP formulas, which indicates the reliability and accuracy of the predictions of these models. Parametric studies and sensitivity analyses are conducted to interpret the trend of input parameters in the GP models and the level of sensitivity of each natural frequency formula to the input parameters. These explicit mathematical expressions can be extended to the other bistable laminates to obtain their natural frequencies on the basis of their geometrical dimensions. The results are validated against the experimental data and verified against FEM outcomes.