Evolution Strategy Algorithm Research Articles

Uncertainty analysis of digital twin model of mine ventilation system

The digital twin model of mine ventilation system (DTMVS) plays an important role in intelligent safety management. However, the uncertainty of the ventilation resistance coefficient, which is the core parameter of the model, makes it challenging to accurately construct a DTMVS. In this study, Latin Hypercube Sampling (LHS) and ventilation resistance coefficient estimation models (VRCEMs) are used to analyze the uncertainty. First, the LHS method was used to explore the effect of uncertainty in the simulated airflow by continuously increasing the level of uncertainty in the ventilation resistance coefficients. Subsequently, the ventilation resistance coefficients were estimated using the VRCEMs, and the uncertainty of the ventilation resistance coefficient and the simulated airflow was analyzed. The results showed that the ventilation resistance coefficients with a 5% coefficient of variation can cause the DTMVS to lose 34% of the real airflow data points. The degree of uncertainty in the ventilation resistance coefficients estimated by the VRCEM-GA (VRCEM using genetic algorithm) and VRCEM-DE (VRCEM using differential evolution algorithm) methods was enhanced by 27.4% and 4.4%, respectively, compared with VRCEM-ES (VRCEM using evolutionary strategy algorithm). The VRCEM-ES model had the least influence on the uncertainty of the simulated airflow of DTMVS. The simulated airflow of the DTMVS constructed based on VRCEMs fluctuated normally within the confidence interval. VRCEMs had a higher sensitivity to the ventilation resistance coefficients of branches with low coefficients of variation.

Real-Time Policy Optimization for UAV Swarms Based on Evolution Strategies

Multi-agent decision-making faces many challenges such as non-stationarity and sparse rewards, while the complexity and randomness of the real environment further complicate policy development. This paper addresses the high-dimensional policy optimization problems of unmanned aerial vehicle (UAV) swarms. By modeling the problem scenario as a Markov decision process, a real-time policy optimization algorithm based on evolution strategy (ES) pre-training is proposed. This approach combines decision-time planning with background planning to evaluate and integrate different sets of policy parameters in a temporal context. In the experimental phase, the policy network is trained using both ES and REINFORCE algorithms on a constructed simulation platform. Comparative experiments demonstrate the effectiveness of using ES for policy pre-training. Finally, the proposed real-time policy optimization algorithm further improves the performance of the swarm by approximately 10% in simulations, offering a feasible solution for adversarial games between swarms and extending the research scope of evolutionary algorithms.

High-Frequency Quantitative Trading of Digital Currencies Based on Fusion of Deep Reinforcement Learning Models with Evolutionary Strategies

High-frequency quantitative trading in the emerging digital currency market poses unique challenges due to the lack of established methods for extracting trading information. This paper proposes a deep evolutionary reinforcement learning (DERL) model that combines deep reinforcement learning with evolutionary strategies to address these challenges. Reinforcement learning is applied to data cleaning and factor extraction from a high-frequency, microscopic viewpoint to quantitatively explain the supply and demand imbalance and to create trading strategies. In order to determine whether the algorithm can successfully extract the significant hidden features in the factors when faced with large and complex high-frequency factors, this paper trains the agent in reinforcement learning using three different learning algorithms, including Q-learning, evolutionary strategies, and policy gradient. The experimental dataset, which contains data on sharp up, sharp down, and continuous oscillation situations, was chosen to test Bitcoin in January-February, September, and November of 2022. According to the experimental results, the evolutionary strategies algorithm achieved returns of 59.18%, 25.14%, and 22.72%, respectively. The results demonstrate that deep reinforcement learning based on the evolutionary strategies outperforms Q-learning and policy gradient concerning risk resistance and return capability. The proposed approach offers a robust and adaptive solution for high-frequency trading in the digital currency market, contributing to the development of effective quantitative trading strategies.

Aviation fuel pump health state assessment based on evidential reasoning and random forests

AbstractAs the power source of the engine, the Fuel Pump(FP) plays a vital role in the safe operation of the aircraft. Due to the complexity of the working mechanism of Aviation Fuel Pumps (AFP) and the strong correlation between the components, the assessment model cannot reflect the health state of the FPs better, while the initial parameters in the assessment model will affect the assessment effect of the model. Therefore, this paper proposes a health status assessment model that can fully integrate monitoring data. To improve the accuracy of the model parameters, the Random Forest algorithm is used to give the feature weights to make up for the limitation of relying on expert knowledge, and the model parameters are optimized by the Covariance Matrix Adaptive Evolutionary Strategy algorithm, which achieves an accurate assessment of the state. Finally, the AFP test bed was built and the AFP was tested. Compared with other methods, the accuracy of the proposed method in this question reaches 96%, which is greatly superior to other methods and verifies the effectiveness of the proposed method. It also provides an outlook on future research directions for health state assessment.

Efficient Sizing and Layout Optimization of Truss Benchmark Structures Using ISRES Algorithm

This paper presents a comprehensive investigation into the application of the Improved Stochastic Ranking Evolution Strategy (ISRES) algorithm for the sizing and layout optimization of truss benchmark structures. Truss structures play a crucial role in engineering and architecture, and optimizing their designs can lead to more efficient and cost-effective solutions. The ISRES algorithm, known for its effectiveness in multi-objective optimization, is adapted for the single-objective optimization of truss designs with multiple design constraints. This study encompasses a wide range of truss benchmark structures, including 10-bar, 15-bar, 18-bar, 25-bar, and 72-bar configurations, each subjected to distinct loading conditions and stress constraints. The objective is to minimize the truss weight while ensuring stress and displacement limits are met. Through extensive experimentation, the ISRES algorithm demonstrates its ability to efficiently explore the solution space and converge to optimal solutions for each truss benchmark structure. The algorithm effectively handles the complexity of the problems, which involve numerous design variables, stress constraints, and nodal displacement limits. A comparative analysis is conducted to assess the performance of the ISRES algorithm against other state-of-the-art optimization methods reported in the literature. The comparison evaluates the quality of the solutions and the computational efficiency of each method. Furthermore, the optimized truss designs are subjected to finite element analysis to validate their structural integrity and stability. The verification process confirms that the designs adhere to the imposed constraints, ensuring the safety and reliability of the final truss configurations. The results of this study demonstrate the efficacy of the ISRES algorithm in providing practical and reliable solutions for the sizing and layout optimization of truss benchmark structures. The algorithm’s competitive performance and robustness make it a valuable tool for structural engineers and designers, offering a versatile and powerful approach for complex engineering optimization tasks. Overall, the findings contribute to the advancement of optimization techniques in structural engineering, promoting the development of more efficient and cost-effective truss designs for a wide range of engineering and architectural applications. The study’s insights empower practitioners to make informed decisions in selecting appropriate optimization strategies for complex truss-design scenarios, fostering advancements in structural engineering and sustainable design practices.

Joint contrastive learning and belief rule base for named entity recognition in cybersecurity

Named Entity Recognition (NER) in cybersecurity is crucial for mining information during cybersecurity incidents. Current methods rely on pre-trained models for rich semantic text embeddings, but the challenge of anisotropy may affect subsequent encoding quality. Additionally, existing models may struggle with noise detection. To address these issues, we propose JCLB, a novel model that Joins Contrastive Learning and Belief rule base for NER in cybersecurity. JCLB utilizes contrastive learning to enhance similarity in the vector space between token sequence representations of entities in the same category. A Belief Rule Base (BRB) is developed using regexes to ensure accurate entity identification, particularly for fixed-format phrases lacking semantics. Moreover, a Distributed Constraint Covariance Matrix Adaptation Evolution Strategy (D-CMA-ES) algorithm is introduced for BRB parameter optimization. Experimental results demonstrate that JCLB, with the D-CMA-ES algorithm, significantly improves NER accuracy in cybersecurity.

The spds*p*+Δ tight binding model for 3C-SiC

In this study, we propose a novel model for crystals containing second-period elements. Adopting the sp3d5s*p*3+∆ tight-binding model, which accounts for the influence of spin–orbit coupling on 3C-SiC, including carbon atoms as second-period elements, we calculate the energy band structure. The Slater–Koster parameters used in the calculations were optimized to experimental values, such as the band gap energy and effective mass, using the covariance matrix adaptation evolution strategy algorithm, a black-box optimization method suitable for such applications. The optimized energy band structure accurately represents the experimental data, confirming the significant impact of p* orbitals near the band gap through projected density of states calculations.

Multi-axis fields boost SABRE hyperpolarization

The inherently low signal-to-noise ratio of NMR and MRI is now being addressed by hyperpolarization methods. For example, iridium-based catalysts that reversibly bind both parahydrogen and ligands in solution can hyperpolarize protons (SABRE) or heteronuclei (X-SABRE) on a wide variety of ligands, using a complex interplay of spin dynamics and chemical exchange processes, with common signal enhancements between 103 and 104. This does not approach obvious theoretical limits, and further enhancement would be valuable in many applications (such as imaging mM concentration species in vivo). Most SABRE/X-SABRE implementations require far lower fields (μT-mT) than standard magnetic resonance (>1T), and this gives an additional degree of freedom: the ability to fully modulate fields in three dimensions. However, this has been underexplored because the standard simplifying theoretical assumptions in magnetic resonance need to be revisited. Here, we take a different approach, an evolutionary strategy algorithm for numerical optimization, multi-axis computer-aided heteronuclear transfer enhancement for SABRE (MACHETE-SABRE). We find nonintuitive but highly efficient multiaxial pulse sequences which experimentally can produce a sevenfold improvement in polarization over continuous excitation. This approach optimizes polarization differently than traditional methods, thus gaining extra efficiency.

A fault diagnosis method for wireless sensor network nodes based on a belief rule base with adaptive attribute weights

Due to the harsh operating environment and ultralong operating hours of wireless sensor networks (WSNs), node failures are inevitable. Ensuring the reliability of the data collected by the WSN necessitates the utmost importance of diagnosing faults in nodes within the WSN. Typically, the initial step in the fault diagnosis of WSN nodes involves extracting numerical features from neighboring nodes. A solitary data feature is often assigned a high weight, resulting in the failure to effectively distinguish between all types of faults. Therefore, this study introduces an enhanced variant of the traditional belief rule base (BRB), called the belief rule base with adaptive attribute weights (BRB-AAW). First, the data features are extracted as input attributes for the model. Second, a fault diagnosis model for WSN nodes, incorporating BRB-AAW, is established by integrating parameters initialized by expert knowledge with the extracted data features. Third, to optimize the model's initial parameters, the projection covariance matrix adaptive evolution strategy (P-CMA-ES) algorithm is employed. Finally, a comprehensive case study is designed to verify the accuracy and effectiveness of the proposed method. The results of the case study indicate that compared with the traditional BRB method, the accuracy of the proposed model in WSN node fault diagnosis is significantly improved.

Waveform design through the trade-off relationship between the MI criterion and the SINR criterion

Aiming at the diversity requirements of cognitive radar monitoring tasks, a joint optimization design criterion that comprehensively considers the mutual information (MI) and signal-to-interference-to-noise ratio (SINR) between the target and the echo is proposed. In view of the challenges brought by the traditional water-filling algorithm, this paper further studies how to effectively solve the new optimization criteria to improve the overall performance of the system. Specifically, this paper proposes a PCMA-ES algorithm that combines an adaptive penalty function with the Covariance Matrix Adaptive Evolutionary Strategy (CMA-ES) algorithm. The penalty function aims to prioritize feasible solutions by assigning them the highest fitness. For infeasible solutions with lower constraint violations, the fitness is slightly lower, allowing for better utilization of information from infeasible solutions. The simulation results show that the PCMA-ES algorithm has lower time complexity or better performance than the traditional water-filling algorithm, and can solve more complex transmission waveforms. In addition, the waveform designed with a joint optimization criterion outperforms that based on a single optimization criterion. The radar detection focus can be adjusted to meet the specific requirements of diverse detection tasks.

An Efficient Differential Grouping Algorithm for Large-Scale Global Optimization

Cooperative co-evolution (CC) is a practical and efficient evolutionary framework for solving large-scale global optimization problems (LSGOPs). The performance of CC depends on how variables are being grouped and can be improved through guided variable decomposition for various optimization problems. However, achieving a proper variable decomposition is computationally expensive. This paper proposes an effective yet efficient differential grouping (EDG) method to reduce the associated computational cost. Our method exploits the historical interrelationship information of previous variable groups to examine interactions between the remnant variable groups. This allows us to spend less computing resources without compromising the accuracy of the final grouping result. Our proposal utilizes the covariance matrix adaptation evolution strategy (CMA-ES) algorithm, in conjunction with EDG, to solve LSGOPs. Further, to reduce time complexity and improve the stability of CMA-ES, we substitute the complex matrix decomposition step with simpler matrix operations to compute the square root of the covariance matrix. Results from our experiments and analysis indicate that EDG is a competitive method to solve LSGOPs and improve the performance of CC. The proposed schemes significantly enhance the searchability of CMA-ES compared to the other large-scale variants of CMA-ES and state-of-the-art large-scale optimizers. Moreover, our EDG could be integrated with evolutionary optimizers of different flavors like Differential Evolution (DE).

A hybridization of evolution strategies with iterated greedy algorithm for no-wait flow shop scheduling problems

This study investigates the no-wait flow shop scheduling problem and proposes a hybrid (HES-IG) algorithm that utilizes makespan as the objective function. To address the complexity of this NP-hard problem, the HES-IG algorithm combines evolution strategies (ES) and iterated greedy (IG) algorithm, as hybridizing algorithms helps different algorithms mitigate their weaknesses and leverage their respective strengths. The ES algorithm begins with a random initial solution and uses an insertion mutation to optimize the solution. Reproduction is carried out using (1 + 5)-ES, generating five offspring from one parent randomly. The selection process employs (µ + λ)-ES, allowing excellent parent solutions to survive multiple generations until a better offspring surpasses them. The IG algorithm’s straightforward search mechanism aids in further improving the solution and avoiding local minima. The destruction operator randomly removes d-jobs, which are then inserted one by one using a construction operator. The local search operator employs a single insertion approach, while the acceptance–rejection criteria are based on a constant temperature. Parameters of both ES and IG algorithms are calibrated using the Multifactor analysis of variance technique. The performance of the HES-IG algorithm is calibrated with other algorithms using the Wilcoxon signed test. The HES-IG algorithm is tested on 21 Nos. Reeves and 30 Nos. Taillard benchmark problems. The HES-IG algorithm has found 15 lower bound values for Reeves benchmark problems. Similarly, the HES-IG algorithm has found 30 lower bound values for the Taillard benchmark problems. Computational results indicate that the HES-IG algorithm outperforms other available techniques in the literature for all problem sizes.

Clinamen2: Functional-style evolutionary optimization in Python for atomistic structure searches

Clinamen2 is a versatile functional-style Python implementation of the covariance matrix adaptation evolution strategy (CMA-ES) utilizing Cholesky decomposition. On top of a problem-agnostic core algorithm, the software package offers a suite of utilities and library code enabling applications to important atomistic structure searches. Features include massively distributed computation and the BI-Population restart scheme. This article details the general code structure and introduces examples that illustrate some relevant applications for the materials science and chemistry worlds, including interfacing to density-functional-theory codes and machine-learned surrogate models. The functional design renders the code modular and adaptable, and makes the creation of interfaces to other atomistic software straightforward. Program summaryProgram Title: Clinamen2CPC Library link to program files:https://doi.org/10.17632/x7syr2txsd.1Developer's repository link:https://github.com/Madsen-s-research-group/clinamen2-public-releasesCode Ocean capsule:https://codeocean.com/capsule/4950229Licensing provisions: Apache-2.0Programming language: PythonSupplementary material:Nature of problem: Find optimal atomistic structures retaining full flexibility in the choice of the optimization target, the methodological approach and its implementation (e.g. CPU- vs. GPU-heavy calculations). Enable interfacing with relevant software, including but not limited to density-functional-theory (DFT) codes and machine-learning (ML) solutions.Solution method: The covariance matrix adaptation evolution strategy (CMA-ES) algorithm is implemented using Cholesky decompositions for efficiency. The core algorithm and application examples for specific problems are implemented in functional-style Python free of side effects, with data classes to keep track of the state of the evolution. Dask is used for job control to enable highly distributed workflows. Advanced strategies like BI-Population CMA-ES are easy to implement and illustrated in the examples.

Drag Reduction Analysis of Optimal Three-dimensional Bumps on the M6 Wing

Based on Isight optimization software and evolutionary strategy algorithm, this paper builds a bump optimization design platform by integrating the three-dimensional bump shape generator program, grid generation software Gridgen and flow field simulation software Fluent. This platform is successfully validated on the RAE2822 supercritical infinite straight wing, which reduces the drag coefficient by 11 counts and increases the lift-drag ratio by 11.48 percent. Through this platform, the drag reduction characteristics of optimal three-dimensional bumps on the M6 wing are numerical studied. The results show that the three-dimensional bumps can weaken shock waves and have spanwise characteristics, the stronger the shock waves, the more obvious the drag reduction effect of the three-dimensional bumps, and the wing with optimal three-dimensional bumps can improve flow field characteristics and prevent flow separations.

Inversion of mine ventilation resistance coefficients enhanced by deep reinforcement learning

The ventilation resistance coefficients of a mine is vital to ventilation system safety management, diagnosis, and intelligentization. Airflow typically serves as the basis for inverting ventilation resistance coefficients. However, the issue of non-uniqueness in conventional nonlinear optimization methods affects the accuracy of inversion. Therefore, this study introduces a novel optimization approach based on deep reinforcement learning (DRL) to invert resistance coefficients. In this methodology, inversion is regarded as a Markov decision process, with the ventilation network solving model (VNSM) embedded within the DRL environment. We design an agent utilizing deep neural networks, which dynamically adjusts the resistance coefficient state variables by interacting with the VNSM to enhance the consistency between theoretical and measured airflow. The consistency corresponds to the agent’s reward. The proximal policy optimization is employed to optimize the agent’s policy. In field experiment, the MAE between the airflow calculated and the measured airflow is 0.354, with MSE of 0.287, RMSE of 0.536, and MRE of 0.013. Compared with the standard genetic algorithm, differential evolution algorithm, and evolution strategy algorithm, the DRL method shows lower MRE, MAE, RMSE, and MSE values by 23.5%, 15.3%, 14.1%, and 26.4%, respectively. Additionally, DRL method exhibits smaller sensitivity differences for different roadways than others algorithm.

A behavior prediction method for complex system based on belief rule base with structural adaptive

Predicting the behavior of complex systems and taking appropriate measures for system management is of paramount importance for decision-makers. Belief rule base (BRB) is an effective method for modeling complex systems, and its construction relies on expert knowledge. However, in certain complex system prediction problems, deriving the structure and parameters of BRB from limited expert knowledge or existing models is a challenge, and there may even be a lack of available expert knowledge. Data mining plays a crucial role in obtaining the parameters for model construction in the design of decision support systems (DSSs). Therefore, this paper proposes a method for behavior prediction of complex systems called the structural adaptive BRB (SA-BRB). First, to reduce the randomness of K-means+ +, this paper introduces an error constraint and employs this algorithm to mine historical data for constructing a reference value set. Second, a model ensemble construction process is designed to create different model structures. Subsequently, the evidence reasoning (ER) algorithm is applied to derive the models, and the projection covariance matrix adaptive evolution strategy (P-CMA-ES) algorithm is used for model optimization. Finally, a model evaluation method is established, allowing adaptive adjustment of the model structure according to the needs of engineering practice and the preferences of decision-makers. Furthermore, the effectiveness of the proposed method is validated through two case studies: one focusing on predicting the health status of a flywheel system and the other on forecasting full-load power generation in a combined power plant.

Instance Weighted Incremental Evolution Strategies for Reinforcement Learning in Dynamic Environments.

Evolution strategies (ESs), as a family of black-box optimization algorithms, recently emerge as a scalable alternative to reinforcement learning (RL) approaches such as Q-learning or policy gradient and are much faster when many central processing units (CPUs) are available due to better parallelization. In this article, we propose a systematic incremental learning method for ES in dynamic environments. The goal is to adjust previously learned policy to a new one incrementally whenever the environment changes. We incorporate an instance weighting mechanism with ES to facilitate its learning adaptation while retaining scalability of ES. During parameter updating, higher weights are assigned to instances that contain more new knowledge, thus encouraging the search distribution to move toward new promising areas of parameter space. We propose two easy-to-implement metrics to calculate the weights: instance novelty and instance quality. Instance novelty measures an instance's difference from the previous optimum in the original environment, while instance quality corresponds to how well an instance performs in the new environment. The resulting algorithm, instance weighted incremental evolution strategies (IW-IESs), is verified to achieve significantly improved performance on challenging RL tasks ranging from robot navigation to locomotion. This article thus introduces a family of scalable ES algorithms for RL domains that enables rapid learning adaptation to dynamic environments.

A novel mathematical model and a hybrid grouping evolution strategy algorithm for an automated last mile delivery system considering wind effect

Efficient and timely delivery of goods in last-mile logistics is a key challenge for e-commerce companies. With the rise of drone technology, drone-based delivery has emerged as a promising solution to address the challenges of traditional last-mile logistics. In light of the environmental and social challenges faced by last-mile delivery, integrating sustainability into logistics planning has become a vital requirement rather than a mere option. Drones have been identified as a potential solution to reduce greenhouse gas emissions and improve the sustainability of last-mile logistics. In this study, we present a mathematical model for the multi-depot multi-period drone delivery problem, which considers wind patterns and multi-periodicity to optimize the routing of multiple drones from multiple depots over a planning horizon of one day or shift. We included the wind profiles in our modeling to achieve a more realistic drone route planning. To address the complexity of the problem, we employ the Dantzig-Wolfe decomposition method to obtain a lower bound for the problem. Since we have a grouping problem, a novel metaheuristic algorithm, based on a grouping evolution strategy algorithm is developed for efficient drone routing. The algorithm uses the structural information along with the problem with the aid of a proposed constructive heuristic based on the well-known best fit strategy. Results show that proposed approach, achieved a solution with a gap of 2.99% to the lower bound on average, demonstrating its effectiveness in addressing the complexity of the multi-depot multi-period drone delivery problem while considering the impact of wind patterns.

Non-magnetic regenerator material of silver oxide for 4 K cryocoolers

In regenerative cryocoolers, magnetic specific heat is often utilized for heat regeneration at cryogenic temperature, such as below 20 K. Magnetic regenerator materials of rare earth compound which have sufficient specific heat enabled cryocooler to reach below 4 K. However, the magnetic noise emitted from the magnetic materials during the refrigeration cycle affects the performance of noise-sensitive devices. In this study, we propose a non-magnetic regenerator material having “optical phonon degree of freedom” even at cryogenic temperatures, in which silver oxide (Ag2O) is selected. Ag2O decomposes at temperature above 410 K in ambient atmosphere, whereas we succeeded in fabricating a sintered Ag2O bulk and small particles of the size of 0.5 mm with high-density more than 90%. Specific heat measurement down to 100 mK was performed for the sintered bulk sample of Ag2O. Below 10 K, a non-Debye behavior was observed, suggesting the contribution from optical phonon modes. Calculation of the phonon density of state (phDOS) performed by the Evolution Strategy algorithm in addition to the first-principles calculation reveals that phDOS has several distinct peaks at low energies, and the low-energy optical phonons contribute to the non-Debye behavior. The low temperature specific heat of Ag2O enhanced by the optical phonons is larger than conventional non-magnetic regenerator materials when compared in terms of specific heat per volume. Moreover, the calculation of cooling performance of cryocooler using Ag2O particles is found to be reached below 4.2 K. This study shows that non-magnetic material with low temperature specific heat enhanced by optical phonon mode can be used as a regenerator material for regenerative cryocoolers.

Evolutionary Approaches for Adversarial Attacks on Neural Source Code Classifiers

As the prevalence and sophistication of cyber threats continue to increase, the development of robust vulnerability detection techniques becomes paramount in ensuring the security of computer systems. Neural models have demonstrated significant potential in identifying vulnerabilities; however, they are not immune to adversarial attacks. This paper presents a set of evolutionary techniques for generating adversarial instances to enhance the resilience of neural models used for vulnerability detection. The proposed approaches leverage an evolution strategy (ES) algorithm that utilizes as the fitness function the output of the neural network to deceive. By starting from existing instances, the algorithm evolves individuals, represented by source code snippets, by applying semantic-preserving transformations, while utilizing the fitness to invert their original classification. This iterative process facilitates the generation of adversarial instances that can mislead the vulnerability detection models while maintaining the original behavior of the source code. The significance of this research lies in its contribution to the field of cybersecurity by addressing the need for enhanced resilience against adversarial attacks in vulnerability detection models. The evolutionary approach provides a systematic framework for generating adversarial instances, allowing for the identification and mitigation of weaknesses in AI classifiers.