Benchmark Test Suite Research Articles

Self-Organizing Migrating Algorithm with narrowing search space strategy for robot path planning

This article introduces a version of the Self-Organizing Migrating Algorithm with a narrowing search space strategy named iSOMA. Compared to the previous two versions, SOMA T3A and Pareto that ranked 3rd and 5th respectively in the IEEE CEC (Congress on Evolutionary Computation) 2019 competition, the iSOMA is equipped with more advanced features with notable improvements including applying jumps in the order, immediate update, narrowing the search space instead of searching on the intersecting edges of hyperplanes, and the partial replacement of individuals in the population when the global best improved no further. Moreover, the proposed algorithm is organized into processes named initialization, self-organizing, migrating, and replacement. We tested the performance of this new version by using three benchmark test suites of IEEE CEC 2013, 2015, and 2017, which, together contain a total of 73 functions. Not only is it superior in performance to other SOMAs, but iSOMA also yields promising results against the representatives of well-known algorithmic families such as Differential Evolution and Particle Swarm Optimization. Moreover, we demonstrate the application of iSOMA for path planning of a drone, while avoiding static obstacles and catching the target.

An Enhanced Naked Mole Rat Algorithm for Optimal Cross-Layer Solution for Wireless Underground Sensor Networks

Nature-inspired algorithms serve as the backbone of modern computing technology, and over the past three decades, the field has grown enormously. Many applications were solved by such algorithms and are replacing the traditional classical optimization processes. A recent naked mole-rat algorithm (NMRA) was proposed based on the mating patterns of naked mole-rats. This algorithm proved its worth in terms of competitiveness and application to various domains of research. The aim was to propose an algorithm based on NMRA, named enhanced NMRA (ENMRA), by mitigating the problems that this algorithm suffers from: slow convergence, poor exploration, and local optima stagnation. To enhance the exploration capabilities of basic NMRA, grey wolf optimization (GWO)-based search equations were employed. Exploitation was improved using population division methods based on local neighborhood search (LNS) and differential evolution (DE) equations. To avoid the local stagnation problem, a neighborhood search strategy around the best individual was utilized. Such improvements help the new variant to solve highly challenging optimization problems in contrast to existing algorithms. The efficacy of ENMRA was evaluated using CEC 2019 benchmark test suite. The results were statistically analyzed by the Wilcoxon rank-sum test and Friedman rank (f-rank) test. The resulting analysis proved that ENMRA is superior to the competitive algorithms for test functions CEC 2019 with overall effectiveness of 60.33%. Moreover, the real-world optimization problem from underground wireless sensor networks for an efficient cross-layer solution was also used to test the efficiency of ENMRA. The results of comparative study and statistical tests affirmed the efficient performance of the proposed algorithm.

A reinforcement learning brain storm optimization algorithm (BSO) with learning mechanism

Brain storm optimization algorithm (BSO), which is inspired by brain storm process of human, has been adopted as an efficient optimizer for various complex problems. A reinforcement learning brain storm optimization algorithm (RLBSO) to improve the performance of BSO is proposed in this paper. Four mutation strategies are designed to enhance the search capability of the algorithm in different stages. Elites are adopted as the guidance to ensure the quality of the population. The global best is utilized to guide the search direction of individuals. The cluster centers are employed to improve the exploitation ability of the algorithm. The historical individuals are utilized to the increase the diversity of the population. The Q-learning mechanism is introduced to guide the selection of strategies according to the historical information fed back by the corresponding strategies. A self-learning mechanism, which is based on the evolutionary state of the population and the experience of previous successful individuals, is utilized to determine the update method of individual. The RLBSO algorithm is tested on the CEC 2017 benchmark test suite and a practical engineering problem. The results show that RLBSO has better performance than the other state of the arts algorithms.

Self-adaptive resources allocation-based differential evolution for constrained evolutionary optimization

When using evolutionary algorithms to address constrained optimization problems, it is important to balance not only the diversity and convergence but also the constraints and objective function. To this end, a self-adaptive resources allocation-based differential evolution (SRADE) is presented in this paper. Specifically, during the evolutionary process, three mutation strategies with distinct focuses are collaboratively employed and adaptively assigned to different individuals based on their performance feedback. That is, most of the computing resources will be consumed by the most efficient strategy at different evolutionary stages to mitigate inefficient search under limited resources. These three collaborative strategies focus on maintaining population diversity, driving the population into feasible regions, and promoting the population toward the objective, respectively. Combining the self-adaptive resources allocation scheme and diverse search strategies is expected to satisfy the requirements of the population for diversity, convergence, constraints, and the objective at a certain iteration. Extensive experiments are performed on three benchmark test suites, including a large number of test functions from IEEE CEC 2006, 2010, and 2017. Compared to other well-designed constrained evolutionary approaches, SRADE exhibits superior or very competitive performance.

Many-Objective Gradient-Based Optimizer to Solve Optimal Power Flow Problems: Analysis and Validations

The growing energy demand and environmental consciousness provoke the conventional single-objective optimization framework no longer satisfies new power system planning and control requirements. The number of optimization objectives being considered in power system optimization is increasing, necessitating the development of many-objective Optimal Power Flow (OPF) problems and the development of solution methods. In this paper, the fitness functions for the Many-Objective OPF (MaO-OPF) problem have been formulated, and a new Many-Objective Gradient-Based Optimizer (MaOGBO) based on reference point strategy is proposed to solve the MaO-OPF problem. The objectives of the MaO-OPF problem is to minimize the Reactive Power Loss (RPL), Active Power Loss (APL), Voltage Magnitude Deviation (VMD), Voltage Stability Indicator (VSI), Total Emission (TE), and the Total Fuel Cost (TFC) by satisfying different complex equality and inequality constraints. In the proposed MaOGBO, a reference point strategy is employed to acquire evenly spread Pareto-optimal solutions in each objective space. In order to improve the effectiveness of Pareto solutions, a mixed-multi-constraint handling approach is also implemented. In order to deal with the complex non-linear constraints, the process utilizes both the repair technique and penalty function. Besides, a fuzzy-based membership function strategy is also applied to locate the Best Compromise Solution (BCS) from the Pareto-optimal analytical solution. In order to validate the effectiveness of the proposed MaOGBO, DTLZ and MaF benchmark test suites are considered. In addition, a standard Institute of Electrical and Electronics Engineers (IEEE) bus test systems, such as IEEE-30/IEEE-57/IEEE-118 with different case studies, are also considered to assess the performance of the proposed algorithm. The obtained results are compared with other state-of-the-art algorithms, and the proposed MaOGBO proves the superiority over other competitors for most of the selected problems, including MaO-OPF. Finally, the proposed MaOGBO is also validated on large-scale Algerian 59-bus power systems and proved its superiority in handling realistic systems. This research is further supported up with extra online service and guidance at https://premkumarmanoharan.wixsite.com/mysite.

An improved model‐based evolutionary algorithm for multi‐objective optimization

AbstractThe basic idea in the estimation of distribution algorithms is the replacement of heuristic operators with machine learning models such as regression models, clustering models, or classification models. So, recently, the model‐based evolutionary algorithms (MBEAs) have been suggested in three groups: The estimation of distribution algorithms (EDAs), surrogate assisted evolutionary algorithms, and the inversed models to map from the objective space to the decision space. In this article, a new approach, based on an inversed model of Gaussian process and random forest framework, is proposed. The main idea is applying the process of random forest variable importance with a random grouping that determines some of the best assignment of decision variables to objective functions in order to form a Gaussian process in inverse models that maps to decision space the rich solutions which are discovered from objective space. Then these inverse models through sampling the objective space generate offspring. The proposed algorithm has been tested on the benchmark test suite for evolutionary algorithms (modified Deb K, Thiele L, Laumanns M, Zitzler E (DTLZ), and Walking Fish Group (WFG)) and indicates that the proposed method is a competitive and promising approach.

Stereotactic radiosurgery commissioning and QA test cases-A TG-119 approach for Stereotactic radiosurgery.

To develop a standardized set of representative clinical treatment cases that pose a range of optimization problems for evaluating the plan quality and dosimetric accuracy within the commissioning process for linac-based stereotactic radiosurgery (SRS). Five test cases with increasing complexity were created to validate delivery accuracy in SRS commissioning similar to the approach used by AAPM TG-119 in developing a test suite for IMRT commissioning. Standardized structure sets, planning goals, and delivery requirements were specified for each case including a small sphere target, irregular target, irregular target placed off-axis, multi-target, and abutting organs-at-risk (OARs). Various VMAT field arrangements including a single arc, two coplanar arcs, full arc and vertex half arc, and four noncoplanar arcs were tested to generate clinically appropriate treatment plans. The small spherical target was 1.0cm in diameter. The irregular target was a clinical cavity (2.3×2.2×1.4cm³) and was shifted 4.5cm for the irregular target off-axis case. The multi-target case used the irregular target and four spherical targets representing metastases ranging 0.9 to 1.6cm in diameter, placed up to 7.5cm off-axis. The abutting OARs case included an acoustic neuroma and target placed near the optic nerve. All spherical targets received 24Gy and the cavity received 18Gy. The abutting OAR cases included a 3.74cc lesion adjacent to the brainstem receiving 13Gy and a 1.11cc lesion adjacent to the optic nerve receiving 12Gy. All plans used a single-isocenter placed at the target center or geometric center of multiple targets. Planning goals for all cases included constraints for the target and brain minus PTV, along with brainstem and optic nerve where applicable. Deliverability was assessed through ion chamber measurements, in addition to composite and per-field planar measurements on Gafchromic film and small-field diode array. A mean and SD for measured versus planned doses of 101.0%±2.9% was observed over the 14 ion chamber measurements. Mean and SD for gamma pass rates were 98.5%±2.2% and 97.1%±4.9% for film and diode array, respectively, for gamma criteria of 2% and 1mm. These cases could provide the preliminary groundwork for a novel benchmark for institutions to evaluate linac-based SRS commissioning and delivery accuracy prior to clinical implementation. The rapid widespread implementation of linac-based SRS, the complexity associated with dosimetry and delivery, and high-profile treatment deviations that have already resulted from its use, highlight the importance of such a benchmark test suite. Comprehensive dosimetric measurements from this standardized set of SRS optimization problems were used to fine-tune and understand the limitations of our SRS planning and delivery system and establish a set of baseline data for comparison with other delivery platforms.

Chaos teaching learning based algorithm for large‐scale global optimization problem and its application

AbstractTeaching learning‐based optimization (TLBO) is a popular stochastic algorithm that has recently been widely applied in a variety of optimization problems since its start. In TLBO algorithm, the concept of chaos not only shows a vital effect in its convergence but also plays a substantial role to balance of exploration and exploitation through evolution. However, TLBO is quickly trapped in local optima and premature convergence seems when applied to sophisticated complex functions. To handle these problems, we introduced an improved TLBO algorithm using chaotic concept. To achieve ability to search for exploration and exploitation, new phase called chaotic phase is added in original TLBO algorithm. The proposed method is thoroughly evaluated on benchmark test suites. The numerical result show that proposed method is relatively effective in adapting the chaotic value regarding original TLBO in terms of solution quality and convergence rate. In addition, performance of proposed method is evaluated on benchmark KDD Cup 99 intrusion dataset. The experimental results demonstrate that proposed method achieves higher predictive accuracy, detection rate, false alarm rate, and provided more significant features as compared with other wrapper techniques.

A simple two-stage evolutionary algorithm for constrained multi-objective optimization

The widespread existence of constrained multi-objective optimization problems (CMOPs) in practical applications encourages researchers to devote more efforts to the development of constrained multi-objective evolutionary algorithms (CMOEAs). An efficient way of handling CMOPs is to balance the satisfaction of constrains and the optimization of objective functions. Although many approaches have been designed to this end, most of them are still unable to handle CMOPs with diverse characteristics effectively. To remedy this issue, a simple and generic two-stage framework is proposed in this paper to achieve better efficiency and versatility for CMOPs. Specifically, the whole search process is simply divided into two stages with different purposes: (i) Stage 1 focuses on approaching the unconstrained Pareto front (PF) and storing the obtained feasible solutions in the archive. During stage 1, the archive is updated to promote convergence and diversity; (ii) Stage 2 is mainly designed to obtain the constrained PF of CMOPs. First, the solutions in the archive are used to form the initial population. Afterwards, the constrained Pareto front (CPF) is extensively explored. Based on this framework, a CMOEA, referred to as C-TSEA, is presented. Fifty-seven instances in five benchmark test suites are chosen to evaluate the performance of our approach. Compared with seven advanced CMOEAs, the results demonstrate the superiority or at least competitiveness of C-TSEA. The generality of the proposed framework is also verified by integrating other methods in the proposed framework.

Dynamic group optimization algorithm with a mean–variance search framework

Dynamic group optimization has recently appeared as a novel algorithm developed to mimic animal and human socialising behaviours. Although the algorithm strongly lends itself to exploration and exploitation, it has two main drawbacks. The first is that the greedy strategy, used in the dynamic group optimization algorithm, guarantees to evolve a generation of solutions without deteriorating than the previous generation but decreases population diversity and limit searching ability. The second is that most information for updating populations is obtained from companions within each group, which leads to premature convergence and deteriorated mutation operators. The dynamic group optimization with a mean–variance search framework is proposed to overcome these two drawbacks, an improved algorithm with a proportioned mean solution generator and a mean–variance Gaussian mutation. The new proportioned mean solution generator solutions do not only consider their group but also are affected by the current solution and global situation. The mean–variance Gaussian mutation takes advantage of information from all group heads, not solely concentrating on information from the best solution or one group. The experimental results on public benchmark test suites show that the proposed algorithm is effective and efficient. In addition, comparative results of engineering problems in welded beam design show the promise of our algorithms for real-world applications.

Large-Scale Evolutionary Strategy Based on Gradient Approximation

For large-scale optimization, CMA-ES has the disadvantages of high complexity and premature stagnation. An improved CMA-ES algorithm called GI-ES was proposed in this paper. For the problem of high complexity, the method in this paper replaces the calculation of a covariance matrix with the modeling of expected fitting degrees for a given covariance matrix. At the same time, to solve the problem of premature stagnation, this paper replaces the historical information of elite individuals with the historical information of all individuals. The information can be seen as approximate gradients. The parameters of the next generation of individuals are generated based on the approximate gradients. The experimental results were tested using CEC 2010 and CEC2013 LSGO benchmark test suite, and the experimental results verified the effectiveness of the algorithm on a number of different tasks.

FSDE: efficient evolutionary optimisation for many-objective aero-engine calibration

Engine calibration aims at simultaneously adjusting a set of parameters to ensure the performance of an engine under various working conditions using an engine simulator. Due to the large number of engine parameters to be calibrated, the performance measurements to be considered, and the working conditions to be tested, the calibration process is very time-consuming and relies on the human knowledge. In this paper, we consider non-convex constrained search space and model a real aero-engine calibration problem as a many-objective optimisation problem. A fast many-objective evolutionary optimisation algorithm with shift-based density estimation, called fSDE, is designed to search for parameters with an acceptable performance accuracy and improve the calibration efficiency. Our approach is compared to several state-of-the-art many- and multi-objective optimisation algorithms on the well-known many-objective optimisation benchmark test suite and a real aero-engine calibration problem, and achieves superior performance. To further validate our approach, the studied aero-engine calibration is also modelled as a single-objective optimisation problem and optimised by some classic and state-of-the-art evolutionary algorithms, compared to which fSDE not only provides more diverse solutions but also finds solutions of high-quality faster.

Hunger games search: Visions, conception, implementation, deep analysis, perspectives, and towards performance shifts

A recent set of overused population-based methods have been published in recent years. Despite their popularity, most of them have uncertain, immature performance, partially done verifications, similar overused metaphors, similar immature exploration and exploitation components and operations, and an insecure tradeoff between exploration and exploitation trends in most of the new real-world cases. Therefore, all users need to extensively modify and adjust their operations based on main evolutionary methods to reach faster convergence, more stable balance, and high-quality results. To move the optimization community one step ahead toward more focus on performance rather than change of metaphor, a general-purpose population-based optimization technique called Hunger Games Search (HGS) is proposed in this research with a simple structure, special stability features and very competitive performance to realize the solutions of both constrained and unconstrained problems more effectively. The proposed HGS is designed according to the hunger-driven activities and behavioural choice of animals. This dynamic, fitness-wise search method follows a simple concept of “Hunger” as the most crucial homeostatic motivation and reason for behaviours, decisions, and actions in the life of all animals to make the process of optimization more understandable and consistent for new users and decision-makers. The Hunger Games Search incorporates the concept of hunger into the feature process; in other words, an adaptive weight based on the concept of hunger is designed and employed to simulate the effect of hunger on each search step. It follows the computationally logical rules (games) utilized by almost all animals and these rival activities and games are often adaptive evolutionary by securing higher chances of survival and food acquisition. This method's main feature is its dynamic nature, simple structure, and high performance in terms of convergence and acceptable quality of solutions, proving to be more efficient than the current optimization methods. The effectiveness of HGS was verified by comparing HGS with a comprehensive set of popular and advanced algorithms on 23 well-known optimization functions and the IEEE CEC 2014 benchmark test suite. Also, the HGS was applied to several engineering problems to demonstrate its applicability. The results validate the effectiveness of the proposed optimizer compared to popular essential optimizers, several advanced variants of the existing methods, and several CEC winners and powerful differential evolution (DE)-based methods abbreviated as LSHADE, SPS_L_SHADE_EIG, LSHADE_cnEpSi, SHADE, SADE, MPEDE, and JDE methods in handling many single-objective problems. We designed this open-source population-based method to be a standard tool for optimization in different areas of artificial intelligence and machine learning with several new exploratory and exploitative features, high performance, and high optimization capacity. The method is very flexible and scalable to be extended to fit more form of optimization cases in both structural aspects and application sides. This paper's source codes, supplementary files, Latex and office source files, sources of plots, a brief version and pseudocode, and an open-source software toolkit for solving optimization problems with Hunger Games Search and online web service for any question, feedback, suggestion, and idea on HGS algorithm will be available to the public at https://aliasgharheidari.com/HGS.html.

MOGBO: A new Multiobjective Gradient-Based Optimizer for real-world structural optimization problems

To handle the multiobjective optimization problems of truss-bar design, this paper introduces a new metaheuristic multiobjective optimization algorithm. The proposed algorithm is based on a recently reported single objective version of the gradient-based optimizer (GBO) inspired by the gradients of Newton’s equations. The proposed algorithm is called as multiobjective gradient-based optimizer (MOGBO), in which two operators, such as local escaping operator and gradient search rule, and few vector sets are utilized in the search phase and the elitist non-dominated sorting mechanism is used for agent sorting to find Pareto optimal solutions. The proposed MOGBO is a posteriori method, and the traditional crowding distance mechanism is employed to confirm the coverage of the best solutions for the objectives of the given problem. The performance of the proposed MOGBO algorithm is verified and validated on different test cases, including 15 unconstraint benchmark test suites and eight constraint multiobjective truss bar design problems. To prove the superiority of the MOGBO algorithm, the performance is compared with state-of-the-art algorithms, such as multiobjective ant lion optimization (MOALO), multiobjective water cycle algorithm (MOWCA), multiobjective colliding bodies optimization (MOCBO), and non-dominated sorting gray wolf optimizer (NSGWO) in terms of metrics, such as hyper-volume, coverage, inverted generational distance, pure diversity, Spacing, Spread, coverage Pareto front, diversity maintenance, generational distance, and runtime. The solutions obtained by the proposed MOGBO algorithm is highly accurate and requires less runtime than the other selected algorithms. The obtained results also show the efficiency of the MOGBO in solving most of all the complex multiobjective problems. This research will be further backed up with external guidance for the future research at https://premkumarmanoharan.wixsite.com/mysite.

A Novel Maximin-Based Multi-Objective Evolutionary Algorithm Using One-by-One Update Scheme for Multi-Robot Scheduling Optimization

With the continuous development of E-commerce, warehouse logistics is also facing emerging challenges, including more batches of orders and shorter order processing cycles. When more orders need to be processed simultaneously, some existing task scheduling methods may not be able to give a suitable plan, which delays order processing and reduces the efficiency of the warehouse. Therefore, the intelligent warehouse system that uses autonomous robots for automated storage and intelligent order scheduling is becoming mainstream. Based on this concept, we propose a multi-robot cooperative scheduling system in the intelligent warehouse. The aim of the multi-robot cooperative scheduling system of the intelligent storage is to drive many robots in an intelligent warehouse to perform the distributed tasks in an optimal (e.g., time-saving and energy-conserved) way. In this paper, we propose a multi-robot cooperative task scheduling model in the intelligent warehouse. For this model, we design a maximin-based multi-objective algorithm, which uses a one-by-one update scheme to select individuals. In this algorithm, two indicators are devised to discriminate the equivalent individuals with the same maximin fitness value in the environmental selection process. The results on benchmark test suite show that our algorithm is indeed a useful optimizer. Then it is applied to settle the multi-robot scheduling problem in the intelligence warehouse. Simulation experiment results demonstrate the efficiency of the proposed algorithm on the real-world scheduling problem.

L-SHADE-E: Ensemble of two differential evolution algorithms originating from L-SHADE

For real parameter single objective optimization, Differential Evolution (DE) performs better than other types of population-based metaheuristic. Nevertheless, in the field of DE, it is impossible for a given combination of operators to perform well in all fitness landscapes. Practice has proved that assembling more than one combinations of operators may lead to improvement in solution. However, some DE algorithms with good performance, e.g., one of the best performers in competitions of real parameter single objective optimization among population-based metaheuristics held by the series of IEEE Congress on Evolutionary Computation - L-SHADE-EpSin, are still not involved in ensemble. Furthermore, ensemble based on similar DE algorithms is rarely reported. Based on such a background, we propose L-SHADE-E, ensemble of L-SHADE-EpSin and L-SHADE-RSP in this paper. The two constituent algorithms are both variants of L-SHADE and belong to similar DE algorithms. In our algorithm, L-SHADE-EpSin or L-SHADE-RSP is selected randomly to run at the beginning. If progress on fitness is low for generations, the other constituent algorithm takes over population immediately. Our experiments are based on the CEC 2014 and CEC 2017 benchmark test suites. Altogether, our algorithm is compared with nine DE algorithms and three population-based metaheuristics other than DE. Experimental results show that our algorithm is very competitive.

Multi-objective regression test suite optimization using three variants of adaptive neuro fuzzy inference system.

In the process of software development, regression testing is one of the major activities that is done after making modifications in the current system or whenever a software system evolves. But, the test suite size increases with the addition of new test cases and it becomes in-efficient because of the occurrence of redundant, broken, and obsolete test cases. For that reason, it results in additional time and budget to run all these test cases. Many researchers have proposed computational intelligence and conventional approaches for dealing with this problem and they have achieved an optimized test suite by selecting, minimizing or reducing, and prioritizing test cases. Currently, most of these optimization approaches are single objective and static in nature. But, it is mandatory to use multi-objective dynamic approaches for optimization due to the advancements in information technology and associated market challenges. Therefore, we have proposed three variants of self-tunable Adaptive Neuro-fuzzy Inference System i.e. TLBO-ANFIS, FA-ANFIS, and HS-ANFIS, for multi-objective regression test suites optimization. Two benchmark test suites are used for evaluating the proposed ANFIS variants. The performance of proposed ANFIS variants is measured using Standard Deviation and Root Mean Square Error. A comparison of experimental results is also done with six existing methods i.e. GA-ANFIS, PSO-ANFIS, MOGA, NSGA-II, MOPSO, and TOPSIS and it is concluded that the proposed method effectively reduces the size of regression test suite without a reduction in the fault detection rate.

Revisiting Facial Age Estimation With New Insights From Instance Space Analysis.

When demonstrating the effectiveness of a new algorithm, researchers are traditionally encouraged to compare their algorithm's performance against existing algorithms on well-studied benchmark test suites. In the absence of more nuanced methodologies, algorithm performance is typically summarized on average across the test suite examples. This paper highlights the potential bias of conclusions drawn by analyzing "on average" performance, and the opportunities offered by a recent testing methodology known as instance space analysis. To illustrate, we revisit our 2007 comparative study of algorithms for facial age estimation, and rigorously stress-test to challenge the original conclusions. The case study demonstrates how powerful visualizations offered by instance space analysis enable greater insights into unique strengths and weaknesses, and which algorithm should be used when and why. Inspired by such insights, a new algorithm is proposed, and its unique advantage is demonstrated. The bias often hidden in well-studied datasets, and the ramifications for drawing biased conclusions, are also illustrated in this case study. While focused on facial age estimation, the methodology and lessons learned from the case study are broadly applicable to any study seeking to draw conclusions about algorithm performance based on empirical results.

Benchmarking materials property prediction methods: the Matbench test set and Automatminer reference algorithm

We present a benchmark test suite and an automated machine learning procedure for evaluating supervised machine learning (ML) models for predicting properties of inorganic bulk materials. The test suite, Matbench, is a set of 13 ML tasks that range in size from 312 to 132k samples and contain data from 10 density functional theory-derived and experimental sources. Tasks include predicting optical, thermal, electronic, thermodynamic, tensile, and elastic properties given a material’s composition and/or crystal structure. The reference algorithm, Automatminer, is a highly-extensible, fully automated ML pipeline for predicting materials properties from materials primitives (such as composition and crystal structure) without user intervention or hyperparameter tuning. We test Automatminer on the Matbench test suite and compare its predictive power with state-of-the-art crystal graph neural networks and a traditional descriptor-based Random Forest model. We find Automatminer achieves the best performance on 8 of 13 tasks in the benchmark. We also show our test suite is capable of exposing predictive advantages of each algorithm—namely, that crystal graph methods appear to outperform traditional machine learning methods given ~104 or greater data points. We encourage evaluating materials ML algorithms on the Matbench benchmark and comparing them against the latest version of Automatminer.

An ensemble learning based prediction strategy for dynamic multi-objective optimization

Prediction strategies are widely-used in dynamic multi-objective evolutionary algorithms (DMOEAs). However, the characteristics of the environmental changes are different and only use one single prediction model cannot react to the changes effectively. The mismatching of the changes and prediction models may make the predicted results inaccurate and unstable. To overcome this shortage, an ensemble learning based prediction strategy (ELPS) is proposed in this paper to help algorithms re-initialize a new population after a change is detected. There are four base prediction models in ELPS, i.e., linear prediction model (LP), knee point-based autoregression model (KP-AR), population-based autoregression model (P-AR) and random re-initialization model (RND). Once a change happens, these four base prediction models are trained by the historical information with ensemble learning and a strong prediction model can be constructed on these four base prediction models. The final re-initialized population is generated by this strong prediction model to react to the new environment. With the help of ELPS, the re-initialized population can adapt different environmental changes and improve the performance on prediction accuracy and robustness. The experimental results show that, compared with other state-of-the-art prediction strategies on benchmark test suite, ELPS has better performance on dealing with dynamic multi-objective optimization problems.