Covariance Matrix Adaptation Research Articles

Covariance matrix adaptation evolution strategy based on ensemble of mutations for parking navigation and maneuver of autonomous vehicles

Previous works have demonstrated the use of human-inspired frameworks to address autonomous vehicle parking. In such frameworks, autonomous vehicle parking is formulated as a two-stage optimization problem where the first stage involves the generation of initial waypoints which are consequently used to reduce the autonomous parking task into a local motion planning task. In the second stage, the local motion planning is solved as a classical optimization problem. However, the overall algorithmic performance in the second stage depends on the planning horizon used to formulate the local planning task. In this work, we show first that the state-of-the-art algorithms often employed in the second stage fail with an increase in the planning horizon. Motivated by the need for algorithms that are scalable to different planning horizons, this work proposes Covariance Matrix Adaptation Evolution Strategy (CMA-ES) with an ensemble of mutations for the second stage. Specifically, the proposed algorithm features an ensemble of Gaussian- and Cauchy-based mutations to facilitate an efficient blend of both exploitation and exploration that is crucial for both short and long horizons. Furthermore, to handle the associated constraints, Superiority of Feasible (SF) constraint handling technique is incorporated into the proposed algorithm. Performance analysis based on 24 parking missions conducted with planning horizons of 2 and 4 shows that the proposed algorithm is scalable to different planning horizons compared with three commonly employed state-of-the-art algorithms.

Predicting and analyzing the algal population dynamics of a grass-type lake with explainable machine learning

Algal blooms, exacerbated by climate change and eutrophication, have emerged as a global concern. In this study, we introduce a novel interpretable machine learning (ML) workflow tailored for investigating the dynamics of algal populations in grass-type lakes, Liangzi lake. Utilizing seven ML methods and incorporating the covariance matrix adaptation evolution strategy (CMA-ES), we predict algal density across three distinct time periods, resulting in the construction of a total of 30 ML models. The CMA-ES-CatBoost model consistently demonstrates superior predictive accuracy and generalization capability across these periods. Through the collective validation of various interpretable tools, we identify water temperature and permanganate index as the two most critical water quality parameters (WQIs) influencing algal density in Liangzi Lake. Additionally, we quantify the independent and interactive effects of WQIs on algal density, pinpointing key thresholds and trends. Furthermore, we determine the minimum combination of WQIs that achieves near-optimal predictive performance, striking a balance between accuracy and cost-effectiveness. These findings offer a scientific and economically efficient foundation for governmental agencies to formulate strategies for water quality management and sustainable development.

Generating Cheap Representative Functions for Expensive Automotive Crashworthiness Optimization

Solving real-world engineering optimization problems, such as automotive crashworthiness optimization, is extremely challenging, because the problem characteristics are oftentimes not well understood. Furthermore, typical hyperparameter optimization approaches that require a large function evaluation budget are computationally hindered, if the function evaluation is expensive, e.g., requires finite element (FE) simulation runs. In this paper, we propose an approach to characterize real-world expensive black-box optimization problems using the exploratory landscape analysis (ELA). Based on these landscape characteristics, we can identify test functions that are fast-to-evaluate and representative for hyperparameter optimization purposes. Focusing on 20 problem instances from automotive crashworthiness optimization, our results reveal that these 20 crashworthiness problems exhibit landscape features different from classical optimization benchmark test suites, such as the widely-used black-box optimization benchmarking (BBOB) problem set. In fact, these 20 problem instances belong to problem classes that are distinct from the BBOB test functions based on the clustering results. Further analysis indicates that, as far as the ELA features concern, they are most similar to problem classes of tree-based test functions. By analyzing the performance of two optimization algorithms with different hyperparameters, namely the covariance matrix adaptation evolutionary strategy (CMA-ES) and Bayesian optimization (BO), we show that the tree-based test functions are indeed representative in terms of predicting the algorithm performances. Following this, such scalable and fast-to-evaluate tree-based test functions have promising potential for automated design of an optimization algorithm for specific real-world problem classes.

An Efficient Differential Grouping Algorithm for Large-Scale Global Optimization

Cooperative co-evolution (CC) is a practical and efficient evolutionary framework for solving large-scale global optimization problems (LSGOPs). The performance of CC depends on how variables are being grouped and can be improved through guided variable decomposition for various optimization problems. However, achieving a proper variable decomposition is computationally expensive. This paper proposes an effective yet efficient differential grouping (EDG) method to reduce the associated computational cost. Our method exploits the historical interrelationship information of previous variable groups to examine interactions between the remnant variable groups. This allows us to spend less computing resources without compromising the accuracy of the final grouping result. Our proposal utilizes the covariance matrix adaptation evolution strategy (CMA-ES) algorithm, in conjunction with EDG, to solve LSGOPs. Further, to reduce time complexity and improve the stability of CMA-ES, we substitute the complex matrix decomposition step with simpler matrix operations to compute the square root of the covariance matrix. Results from our experiments and analysis indicate that EDG is a competitive method to solve LSGOPs and improve the performance of CC. The proposed schemes significantly enhance the searchability of CMA-ES compared to the other large-scale variants of CMA-ES and state-of-the-art large-scale optimizers. Moreover, our EDG could be integrated with evolutionary optimizers of different flavors like Differential Evolution (DE).

Trajectory Correction of the Rocket for Aerodynamic Load Shedding Based on Deep Neural Network and the Chaotic Evolution Strategy With Covariance Matrix Adaptation

Trajectory Correction of the Rocket for Aerodynamic Load Shedding Based on Deep Neural Network and the Chaotic Evolution Strategy With Covariance Matrix Adaptation

Virtual muscles and reflex control generates human-like ankle torques during gait perturbations.

A biologically-inspired actuation system, including muscles, spinal reflexes, and vestibular feedback, may be capable of achieving more natural gait mechanics in powered prostheses or exoskeletons. In this study, we developed a Virtual Muscle Reflex (VMR) system to control ankle torque and tuned it using data from human responses to anteroposterior mechanical perturbations at three walking speeds. The system consists of three Hill-Type muscles, simulated in real time, and uses feedback from ground reaction force and from stretch sensors on the virtual muscle fibers. Controller gains, muscle properties, and reflex/vestibular time delays were optimized using Covariance Matrix Adaptation (CMA) to minimize the difference between the VMR torque output and the torque measured from the experiment. We repeated the procedure using a conventional finite-state impedance controller. For both controllers, the coefficient of determination () and root-mean-square error (RMSE) was calculated as a function of time within the gait cycle. The VMR had lower RMSE than the impedance controller in 70%, and in 60% of the trials, the of the VMR controller was higher than for the impedance controller. We concluded that the VMR system can better reproduce the human responses to perturbations than the impedance controller.

A complex system health state assessment method with reference value optimization for interpretable BRB

Health condition assessment is the basis for formulating and optimizing maintenance strategies of complex systems, which is crucial for ensuring the safe and stable operation of these systems. In complex system health condition assessment, it is not only necessary for the model to handle various uncertainties to ensure the accuracy of assessment results, but also to have a transparent and reasonable assessment process and interpretable, traceable assessment results. belief rule base (BRB) has been widely used as an interpretable modeling method in health condition assessment. However, BRB-based models currently face two issues: (1) inaccuracies in expert-provided parameters that can affect the model's accuracy, and (2) after model optimization, interpretability may be reduced. Therefore, this paper proposes a new method for complex system health condition assessment called interpretable BRB with reference value optimization (I-BRB). Firstly, to address the issue of inaccurate reference values, a reference value optimization algorithm with interpretability constraints is designed, which optimizes the reference values without compromising expert knowledge. Secondly, the remaining parameters are optimized using the projection covariance matrix adaptation evolution strategy (P-CMA-ES) with interpretability constraints to improve the model's accuracy. Finally, a case study evaluating the bearing components of a flywheel system is conducted to validate the proposed method. Experimental results demonstrate that I-BRB achieves higher accuracy in health condition assessment.

Covariance matrix adaptation evolution strategy based on correlated evolution paths with application to reinforcement learning

Proven as an efficient population-based optimization algorithm, Covariance Matrix Adaptation Evolution Strategy (CMA-ES) features two evolution paths, one to update the covariance matrix and the other to adapt its mutation strength. Considering the time and space complexity of CMA-ES, there are several attempts in the literature to realize a single-path algorithm. However, such attempts require altering the original structure of CMA-ES and consequently eliminating some vital features crucial to the overall algorithm performance. In this paper, we show that the two evolution paths of CMA-ES are highly correlated and one can be expressed in terms of the other thus reducing the computational cost of the algorithm while preserving the original algorithmic framework. Based on experimental studies conducted using 30 functions from the IEEE CEC 2014 benchmark suite, the proposed algorithm shows comparable results with the standard CMA-ES as well as five other state-of-the-art CMA-ES variants. Furthermore, it is shown that the proposed algorithm can be applied to policy search in Deep Reinforcement Learning (DRL). Performance results based on selected DRL problems from different application domains prove the efficiency of the proposed algorithm compared to other population-based algorithms often applied for policy search in DRL.

Integration of sales, inventory, and transportation resource planning by dynamic-demand joint replenishment problem with time-varying costs

This paper addresses the joint replenishment problem in the context of sales and operations planning. Unlike traditional ones that assume static demands, unit costs, and transportation capacity, we consider dynamic-deterministic demands from sales plans and time-varying unit costs and transportation capacity to deal with the recent logistics conditions. We then introduce the capacitated dynamic-demand joint replenishment problem with time-varying costs. To efficiently solve this problem, a three-phase approach is proposed: (1) simplifying the problem and determining ideal inventory quantities using mixed integer linear programming, (2) estimating policy variables for each item using the covariance matrix adaptation evolution strategy, and (3) updating the ideal inventory quantities based on evaluated shortages until all demands are satisfied. Our method outperforms conventional approaches with at least 12 times faster solution runtimes in tests with up to 100 items. We also obtain the insight that the cost-efficient replenishment plan changes according to the increase rate of the unit transportation cost.

Transfer learning based covariance matrix adaptation for evolutionary many-objective optimization

Evolutionary Algorithms (EA) have proven successful in solving Multi- and Many-objective Optimization Problems (MOPs/MaOPs) in numerous application areas. However, the various linkages among the decision variables may have posed a challenge for traditional evolutionary algorithms for solving such problems. In this paper, a transfer learning based covariance matrix adaptation algorithm, shortened as TL-M2M-CMA, is proposed to handle MOPs/MaOPs with complex decision variable linkage. In TL-M2M-CMA, the search population is first decomposed into a set of subpopulations and each subpopulation corresponds to a specific part of the Pareto-optimal Front (PF) of the MOP/MaOP. During the search, a covariance matrix adaptation process is utilized to learn the linkage among the decision variables, and then the covariance matrix is incorporated into the crossover and mutation operators for better efficiency. To facilitate performance comparison, a set of scalable MOP test instances with various linkage complexities is constructed for experimental studies. The performance of the proposed TL-M2M-CMA on these constructed instances is verified by comparing it to the performance of five state-of-the-art EA methods.

CMAES-WFD: Adversarial Website Fingerprinting Defense Based on Covariance Matrix Adaptation Evolution Strategy

CMAES-WFD: Adversarial Website Fingerprinting Defense Based on Covariance Matrix Adaptation Evolution Strategy

Application of Covariance Matrix Adaptation‐Evolutionary Strategy for Modified Constrained Optimal Power Flow Problem Incorporating Valve Point and Emission Effect

A prevailing problem in power and energy subsystems is the smooth operation of electric energy systems. This work presents recent, efficient, and reliable evolutionary algorithm for solving the optimal power flow (OPF) analysis. All various practical complex equality and inequality constraints, namely, bus voltages, real powers of the generator buses, tap settings of the transformers and the reactive power generations, shunt compensation, and emission, are considered for the real‐world scenario. Primary feature in a gas power plant that raises a lot of computational shortcomings with nonlinear structure in fuel cost is valve point effect. The existing research works have not factored the valve point effect and lack the accuracy in the fuel cost minimization and do not reflect the various practical complexities such as valve point and emission effects in the OPF problem formulation. This paper, for the first time, introduces modified OPF problem formulation incorporating valve point effect and applies covariance matrix adaptation‐evolution strategy (CMA‐ES) for solving the modified OPF problem. The algorithm is scrutinised and tested on a modified IEEE‐30‐bus platform for various OPF objectives such as cost minimization, transmission loss, and total voltage deviation, subjected to practical constraints. Load flow analysis has been carried out using the Newton–Raphson method. This work aims to lay the foundation in such a way that it can be applicable in a real‐world scenario for any number of buses.

Automatic Temperature Parameter Tuning for Reinforcement Learning Using Path Integral Policy Improvement.

In this article, we propose a novel variant of path integral policy improvement with covariance matrix adaptation ( [Formula: see text] - [Formula: see text] ), which is a reinforcement learning (RL) algorithm that aims to optimize a parameterized policy for the continuous behavior of robots. [Formula: see text] - [Formula: see text] has a hyperparameter called the temperature parameter, and its value is critical for performance; however, little research has been conducted on it and the existing method still contains a tunable parameter, which can be critical to performance. Therefore, tuning by trial and error is necessary in the existing method. Moreover, we show that there is a problem setting that cannot be learned by the existing method. The proposed method solves both problems by automatically adjusting the temperature parameter for each update. We confirmed the effectiveness of the proposed method using numerical tests.

An Enhanced Genetic Algorithm using Directional-Based Crossover and normal mutation For Global Optimization Problems

Global optimization has been employed in many practical modeling processes. Using gradient methods to solve optimization problems may be computationally inefficient and time-consuming, particularly when convexity or differentiability is not guaranteed. On the other hand, nature-inspired techniques offer an effective gradient-free approach for solving complex, non-convex, or non-differentiable problems. Genetic algorithms are one of the most effective and widely used nature-inspired techniques. However, canonical genetic algorithms do not always guarantee convergence to the optimum point owing to the stochastic nature of the genetic operators, and typically require more work to ensure convergence and increase performance. Improving the genetic operators remains an open issue and usually involves a trade-off between the speed of convergence and searchability. In this study, we propose an enhanced genetic algorithm that relies on directional-based crossover and normal mutation operators to increase the speed of convergence while preserving searchability. The proposed algorithm is evaluated using a set of 40 typical benchmark functions in two dimensions. In addition, to examine its performance at higher dimensions, 16 functions from the test set were tested at 10 and 100 dimensions. The evaluation results of the proposed algorithm are compared to the outcomes of three modern optimization algorithms, namely (Whale optimization algorithm, Teacher-Learner based algorithm, and Covariance matrix adaptation evolution strategy). The results revealed that the proposed algorithm outperformed the conventional algorithms at lower dimensions in all test functions and showed a relatively better performance than the other algorithms at higher dimensions.

Clinamen2: Functional-style evolutionary optimization in Python for atomistic structure searches

Clinamen2 is a versatile functional-style Python implementation of the covariance matrix adaptation evolution strategy (CMA-ES) utilizing Cholesky decomposition. On top of a problem-agnostic core algorithm, the software package offers a suite of utilities and library code enabling applications to important atomistic structure searches. Features include massively distributed computation and the BI-Population restart scheme. This article details the general code structure and introduces examples that illustrate some relevant applications for the materials science and chemistry worlds, including interfacing to density-functional-theory codes and machine-learned surrogate models. The functional design renders the code modular and adaptable, and makes the creation of interfaces to other atomistic software straightforward. Program summaryProgram Title: Clinamen2CPC Library link to program files:https://doi.org/10.17632/x7syr2txsd.1Developer's repository link:https://github.com/Madsen-s-research-group/clinamen2-public-releasesCode Ocean capsule:https://codeocean.com/capsule/4950229Licensing provisions: Apache-2.0Programming language: PythonSupplementary material:Nature of problem: Find optimal atomistic structures retaining full flexibility in the choice of the optimization target, the methodological approach and its implementation (e.g. CPU- vs. GPU-heavy calculations). Enable interfacing with relevant software, including but not limited to density-functional-theory (DFT) codes and machine-learning (ML) solutions.Solution method: The covariance matrix adaptation evolution strategy (CMA-ES) algorithm is implemented using Cholesky decompositions for efficiency. The core algorithm and application examples for specific problems are implemented in functional-style Python free of side effects, with data classes to keep track of the state of the evolution. Dask is used for job control to enable highly distributed workflows. Advanced strategies like BI-Population CMA-ES are easy to implement and illustrated in the examples.

Multi-objective optimization of permanent magnet motors using deep learning and CMA-ES

This paper proposes a multi-objective optimization method for permanent magnet motors using a fast optimization algorithm, Covariance Matrix Adaptation Evolution Strategy (CMA-ES), and deep learning. Multi-objective optimization with topology optimization is effective in the design of permanent magnet motors. Although CMA-ES needs fewer population size than genetic algorithm for single objective problems, this is not evident for multi-objective problems. For this reason, the proposed method generates training data by solving the single-objective optimization multiple times using CMA-ES, and constructs a deep neural network (NN) based on the data to predict performance from motor images at high speed. The deep NN is then used for fast solution of multi-objective optimization problems. Numerical examples demonstrate the effectiveness of the proposed method.

Performance and Techno-Economic Analysis of Optimal Hybrid Renewable Energy Systems for the Mining Industry in South Africa

This paper presents an exploration of the potential of hybrid renewable energy systems (HRESs), combining floating solar photovoltaics (FPV), wind turbines, and vanadium redox flow (VRF) battery energy storage systems (BESSs) to expedite the transition from conventional to renewable energy for the mining sector in South Africa. The feasibility study assesses how to enhance the overall efficiency and minimize greenhouse gas emissions from an economic standpoint by using the Hybrid Optimization of Multiple Energy Resources (HOMER) grid software version 1.11.1 and PVsyst version 7.4. Furthermore, the BESS Covariance Matrix Adaptation Evolution Strategy (CMA-ES) dispatch algorithm is proposed to make the most of the battery storage capacity and capability, aligning it with the dynamic energy demand and supply patterns of an HRES. The proposed HRES includes a highly efficient SFPV with a performance ratio of 0.855 and an annual energy production of 15,835 MWh; a wind turbine (WT) operating for 2977 h annually, achieving a 25% wind penetration rate; and a dynamic VRF-BESS with a 15,439 kWh life throughput and a 3 s dispatch response time. This HRES has a CapEx of R172 million, a 23.5% Internal Rate of Return (IRR), and an investment payback period of 4.9 years. It offers a low Levelized Cost of Energy (LCoE) at 4.27 R/kWh, a competitive Blended Cost of Energy (BCoE) at 1.91 R/kWh, and a positive net present cost (NPC), making it economically advantageous without external subsidies. Moreover, it annually reduces CO2 emissions by 1,715,468 kg, SO2 emissions by 7437 kg, and NOx emissions by 3637 kg, contributing to a significant environmental benefit.

A supervised parallel optimisation framework for metaheuristic algorithms

A Supervised Parallel Optimisation (SPO) is presented. The proposed framework couples different optimisation algorithms to solve single-objective optimisation problems. The supervision balances the exploration and exploitation capabilities of the distinct optimisers included, providing a general framework to solve problems with diverse characteristics. In this work, five optimisation algorithms are included in the ensemble: Particle Swarm Optimisation (PSO), Genetic Algorithm (GA), Covariance Matrix Adaption-Evolution Strategy (CMA-ES), Differential Evolution (DE), and Modified Cuckoo Search (MCS). A geometric path-finding problem with numerous local minima is used to demonstrate the advantage of SPO. The effectiveness of the approach is compared with that of stand-alone incidences of the integrated optimisation strategies and with state-of-the-art algorithms. In addition, a benchmark test suit composed of engineering applications is utilised to validate the general applicability of SPO with respect to a variety of problems. The good solutions generated by SPO are shown to be generally reproducible, while isolated algorithms, at best, render good solutions only occasionally.

Interactive Robot Task Learning: Human Teaching Proficiency With Different Feedback Approaches

The deployment of versatile robot systems in diverse environments requires intuitive approaches for humans to flexibly teach them new skills. In our present work, we investigate different user feedback types to teach a real robot a new movement skill. We compare feedback as star ratings on an absolute scale for single roll-outs versus preference-based feedback for pairwise comparisons with respective optimization algorithms (i.e., a variation of co-variance matrix adaptation -evolution strategy (CMA-ES) and random optimization) to teach the robot the game of skill cup-and-ball. In an experimental investigation with users, we investigated the influence of the feedback type on the user experience of interacting with the different interfaces and the performance of the learning systems. While there is no significant difference for the subjective user experience between the conditions, there is a significant difference in learning performance. The preference-based system learned the task quicker, but this did not influence the users’ evaluation of it. In a follow-up study, we confirmed that the difference in learning performance indeed can be attributed to the human users’ performance.

Breast simulation pipeline: From medical imaging to patient-specific simulations

BackgroundBreast-conserving surgery is the most acceptable operation for breast cancer removal from an invasive and psychological point of view. Before the surgical procedure, a preoperative MRI is performed in the prone configuration, while the surgery is achieved in the supine position. This leads to a considerable movement of the breast, including the tumor, between the two poses, complicating the surgeon's task. MethodsIn this work, a simulation pipeline allowing the computation of patient-specific geometry and the prediction of personalized breast material properties was put forward. Through image segmentation, a finite element model including the subject-specific geometry is established. By first computing an undeformed state of the breast, the geometrico-material model is calibrated by surface acquisition in the intra-operative stance. FindingsUsing an elastic corotational formulation, the patient-specific mechanical properties of the breast and skin were identified to obtain the best estimates of the supine configuration. The final results are a shape-fitting closest point residual of 4.00 mm for the mechanical parameters Ebreast=0.32 kPa and Eskin=22.72 kPa, congruent with the current state-of-the-art. The Covariance Matrix Adaptation Evolution Strategy optimizer converges on average between 5 to 30 min depending on the initial parameters, reaching a simulation speed of 20 s. To our knowledge, our model offers one of the best compromises between accuracy and speed. InterpretationSatisfactory results were obtained for the estimation of breast deformation from preoperative to intra-operative configuration. Furthermore, we have demonstrated the clinical feasibility of such applications using a simulation framework that aims at the smallest disturbance of the actual surgical pipeline.