Covariance Matrix Adaptation Research Articles

Multi-Lane Differential Variable Speed Limit Control via Deep Neural Networks Optimized by an Adaptive Evolutionary Strategy.

In advanced transportation-management systems, variable speed limits are a crucial application. Deep reinforcement learning methods have been shown to have superior performance in many applications, as they are an effective approach to learning environment dynamics for decision-making and control. However, they face two significant difficulties in traffic-control applications: reward engineering with delayed reward and brittle convergence properties with gradient descent. To address these challenges, evolutionary strategies are well suited as a class of black-box optimization techniques inspired by natural evolution. Additionally, the traditional deep reinforcement learning framework struggles to handle the delayed reward setting. This paper proposes a novel approach using covariance matrix adaptation evolution strategy (CMA-ES), a gradient-free global optimization method, to handle the task of multi-lane differential variable speed limit control. The proposed method uses a deep-learning-based method to dynamically learn optimal and distinct speed limits among lanes. The parameters of the neural network are sampled using a multivariate normal distribution, and the dependencies between the variables are represented by a covariance matrix that is optimized dynamically by CMA-ES based on the freeway's throughput. The proposed approach is tested on a freeway with simulated recurrent bottlenecks, and the experimental results show that it outperforms deep reinforcement learning-based approaches, traditional evolutionary search methods, and the no-control scenario. Our proposed method demonstrates a 23% improvement in average travel time and an average of a 4% improvement in CO, HC, and NOx emission.Furthermore, the proposed method produces explainable speed limits and has desirable generalization power.

Short-term solar radiation forecasting using hybrid deep residual learning and gated LSTM recurrent network with differential covariance matrix adaptation evolution strategy

Developing an accurate and robust prediction of long-term average global solar irradiation plays a crucial role in industries such as renewable energy, agribusiness, and hydrology. However, forecasting solar radiation with a high level of precision is historically challenging due to the nature of this source of energy. Challenges may be due to the location constraints, stochastic atmospheric parameters, and discrete sequential data. This paper reports on a new hybrid deep residual learning and gated long short-term memory recurrent network boosted by a differential covariance matrix adaptation evolution strategy (ADCMA) to forecast solar radiation one hour-ahead. The efficiency of the proposed hybrid model was enriched using an adaptive multivariate empirical mode decomposition (MEMD) algorithm and 1+1EA-Nelder–Mead simplex search algorithm. To compare the performance of the hybrid model to previous models, a comprehensive comparative deep learning framework was developed consisting of five modern machine learning algorithms, three stacked recurrent neural networks, 13 hybrid convolutional (CNN) recurrent deep learning models, and five evolutionary CNN recurrent models. The developed forecasting model was trained and validated using real meteorological and Shortwave Radiation (SRAD1) data from an installed offshore buoy station located in Lake Michigan, Chicago, United States, supported by the National Data Buoy Centre (NDBC). As a part of pre-processing, we applied an autoencoder to detect the outliers in improving the accuracy of solar radiation prediction. The experimental results demonstrate that, firstly, the hybrid deep residual learning model performed best compared with other machine learning and hybrid deep learning methods. Secondly, a cooperative architecture of gated recurrent units (GRU) and long short-term memory (LSTM) recurrent models can enhance the performance of Xception and ResNet. Finally, using an effective evolutionary hyper-parameters tuner (ADCMA) reinforces the prediction accuracy of solar radiation.

Epigenetic opportunities for evolutionary computation

Evolutionary computation is a group of biologically inspired algorithms used to solve complex optimization problems. It can be split into evolutionary algorithms, which take inspiration from genetic inheritance, and swarm intelligence algorithms, that take inspiration from cultural inheritance. However, much of the modern evolutionary literature remains relatively unexplored. To understand which evolutionary mechanisms have been considered, and which have been overlooked, this paper breaks down successful bioinspired algorithms under a contemporary biological framework based on the extended evolutionary synthesis, an extension of the classical, genetics focused, modern synthesis. Although the idea of the extended evolutionary synthesis has not been fully accepted in evolutionary theory, it presents many interesting concepts that could provide benefits to evolutionary computation. The analysis shows that Darwinism and the modern synthesis have been incorporated into evolutionary computation but the extended evolutionary synthesis has been broadly ignored beyond: cultural inheritance, incorporated in the sub-set of swarm intelligence algorithms, evolvability, through covariance matrix adaptation evolution strategy (CMA-ES), and multilevel selection, through multilevel selection genetic algorithm (MLSGA). The framework shows a gap in epigenetic inheritance for evolutionary computation, despite being a key building block in modern interpretations of evolution. This leaves a diverse range of biologically inspired mechanisms as low hanging fruit that should be explored further within evolutionary computation and illustrates the potential of epigenetic based approaches through the recent benchmarks in the literature.

Multiple Strategies Boosted Orca Predation Algorithm for Engineering Optimization Problems

This paper proposes an enhanced orca predation algorithm (OPA) called the Lévy flight orca predation algorithm (LFOPA). LFOPA improves OPA by integrating the Lévy flight (LF) strategy into the chasing phase of OPA and employing the greedy selection (GS) strategy at the end of each optimization iteration. This enhancement is made to avoid the entrapment of local optima and to improve the quality of acquired solutions. OPA is a novel, efficient population-based optimizer that surpasses other reliable optimizers. However, owing to the low diversity of orcas, OPA is prone to stalling at local optima in some scenarios. In this paper, LFOPA is proposed for addressing global and real-world optimization challenges. To investigate the validity of the proposed LFOPA, it is compared with seven robust optimizers, including the improved multi-operator differential evolution algorithm (IMODE), covariance matrix adaptation evolution strategy (CMA-ES), gravitational search algorithm (GSA), grey wolf optimizer (GWO), moth-flame optimization algorithm (MFO), Harris hawks optimization (HHO), and the original OPA on 10 unconstrained test functions linked to 2020 IEEE Congress on Evolutionary Computation (CEC’20). Furthermore, four different design engineering issues, including the welded beam, the tension/compression spring, the pressure vessel, and the speed reducer, are solved using the proposed LFOPA, to test its applicability. It was also employed to address node localization challenges in wireless sensor networks (WSNs) as an example of real-world applications. Results and tests of significance show that the proposed LFOPA performs much better than OPA and other competitors. LFOPA simulation results on node localization challenges are much superior to other competitors in terms of minimizing squared errors and localization errors.

Stokastik Fraktal Arama Algoritması ve CMA-ES tabanlı yeni bir hibrit algoritma

In this study, a novel hybridization approach, which is called CMASFS and is based on the covariance matrix adaptation evolution strategy (CMA-ES) and the stochastic fractal search (SFS) algorithms. To make the proposed algorithm dynamic, Gaussian walk equations involved in the diffusion process of SFS have been updated and the algorithm decide to use which the Gaussian walk equations. The effectiveness of the proposed algorithm is tested using CEC2017 benchmark functions having unimodal, multimodal, hybrid, and composition functions in 10, 30, 50, and 100 dimensions. The performance of the CMASFS algorithm is compared with 17 metaheuristic algorithms given in the literature over the CEC2017 benchmark functions. According to the results, it is seen that CMASFS is generally obtained better mean error values. Moreover, to show the superiority of the proposed algorithm, Friedman analysis and the Wilcoxon rank-sum test are applied to the test results of the algorithms. The results of the Wilcoxon signed-rank test show that the improvement with the CMASFS algorithm is statistically significant on the majority of the CEC2017. The results of Friedman test verify that the CMASFS is obtained the best rank compared to both the original SFS and other compared algorithms.

Structural optimisation for controlled deflections of additively manufactured single material beams

Closely controlling the mechanical behaviour and characterization of the deflection of a beam structure is a well-known and widely studied engineering problem. The progress in additive manufacturing methods and the possibilities to closely control the material property variations with the controlled placement of materials further widen the opportunities to achieve given beam deflection criteria. The multi-material additive manufacturing solutions suffer from the lack of real engineering material options, and the quality and performance of the printed parts are usually unsuitable for producing functional parts. A novel cellular structured solution is proposed here, which utilises optimisation of geometries of individual cells of a single material structured beam to obtain deflection profiles closely matched with preset conditions under different loading conditions. The cellular geometry of the structured beam is continually altered for searching and converging on the optimal structure of the cells by the covariance matrix adaptation evolution strategy algorithm in an iterative manner. The optimised beam structures could also be physically produced with single material additive manufacturing methods and the experimental and numerical beam deflection responses correlated closely.

Accurate estimation of the optical properties of nanofluids for solar energy harvesting using the null-collision forward Monte Carlo method

Nanofluid is commonly used in solar energy systems for its excellent physical properties. Its optical properties play an important role in enhancing the performance of the solar energy system. To obtain the optical properties of nanofluids accurately, we proposed a measurement technique as the combination of the null-collision forward Monte Carlo (NC-FMC) and the covariance matrix adaptation evolution strategy based on a restart strategy. An energy partition branch and a criterion are introduced to improve the performance of the classic NC-FMC. A multi-angle model is developed for alleviating the ill-posedness of the model by analyzing the confidence intervals for the estimated parameters. Numerical experiments of several nanofluids with different particle diameters and base fluids are operated to analyze the accuracy and efficiency of the measurement model. A normalized merit function is applied to test the proposed model's ability to predict the performance of a PV/T system with additional nanofluids. The results show that the proposed measurement method has good accuracy and robustness when applied to forecast the performance of PV/T system.

Inversion of potential fields by interactive optimization of 3D subsurface models using a spring-based space warping and evolution strategy

The rapid development of dynamic data-integrative modeling of geologic processes and subsurface structures is an important factor for the sustainable utilization of natural resources. One of the current gaps in data-integrative modeling is the interactive construction of realistic 3D models of the earth’s underground. We have developed here a methodology of 3D interactive processing of potential fields — gravity and magnetics — as well as their potentials and derivatives, combining forward and inverse modeling. Forward computations are based on the approximation of geologic subsurface structures by polyhedra with triangulated surfaces. Inverse computations for the model geometry are performed by means of the covariance matrix adaptation evolution strategy (CMA-ES), which is proved to be efficient in the case of a strongly nonlinear problem and high-dimensional parameter space, as in potential field models. The main disadvantage is related to triangulation, as certain algorithmic constraints must be applied during the approximation of geologic body shapes. To avoid topology distortions, we use a concept of warping the space containing the model, rather than the model itself. However, the optimized lengths of grid sides are dependent on each other, which degrades the self-adaptation of the CMA-ES. The elegant solution is to introduce a system of virtual elastic springs connecting the grid nodes. We develop a numerical formulation of this system and provide a proof of the concept rather than an overview of the theoretical concepts of the inversion schemes. The new workflow is tested on the 3D SEG Advanced Modeling Program model and applied to a real case study of salt dome modeling in the Northwest German Basin. In the context of the inversion procedure described here, we determine how an interpreter can visually control and influence the quality of the inversion on a timeline.

GCP-HOLO: Generating High-Order Linkage Graphs for Path Synthesis

Abstract One degrees-of-freedom (1DOF) linkages are persistent in mechanical systems. However, designing linkages to follow a desired path, known as path synthesis, is challenging due to non-linearities, combinatorial nature, and strict geometric constraints. Current state-of-the-art algorithms cannot scale well to linkages with higher-order linkage graphs, which are required to satisfy more complicated paths for new mechanical systems, such as hopping and flying robots. One reason for this is that state-of-the-art algorithms spend the majority of the time exploring constraint-violating designs. This work uses an Assur group 0DOF linkage as a graph grammar rule to modify both linkage graph and spatial parameters, ensuring all designs are valid 1DOF linkages. Using this graph grammar, this paper formulates linkage path synthesis as a tree search and uses a deep reinforcement learning (DRL) agent to search the space of kinematically feasible planar 1DOF linkages. This paper introduces a method using a graph convolution policy for high-order linkage graph optimization (GCP-HOLO). An anytime algorithm, GCP-HOLO outputs linkages with 1–8 loops (4–16 bars) efficiently. When comparing the GCP-HOLO formulation to a recent state-of-the-art paper that solves a mixed integer conic program, GCP-HOLO generates sets of solutions of varying linkage complexities to eight test trajectories in a quarter of the time. Extending GCP-HOLO with a global node optimization, such as covariance matrix adaptation evolutionary strategy, the results quickly converge to finding better solutions for 4/8 tests, with the whole pipeline capable of a 13X speed increase.

NSGA-II/SDR-OLS: A Novel Large-Scale Many-Objective Optimization Method Using Opposition-Based Learning and Local Search

Recently, many-objective optimization problems (MaOPs) have become a hot issue of interest in academia and industry, and many more many-objective evolutionary algorithms (MaOEAs) have been proposed. NSGA-II/SDR (NSGA-II with a strengthened dominance relation) is an improved NSGA-II, created by replacing the traditional Pareto dominance relation with a new dominance relation, termed SDR, which is better than the original algorithm in solving small-scale MaOPs with few decision variables, but performs poorly in large-scale MaOPs. To address these problems, we added the following improvements to the NSGA-II/SDR to obtain NSGA-II/SDR-OLS, which enables it to better achieve a balance between population convergence and diversity when solving large-scale MaOPs: (1) The opposition-based learning (OBL) strategy is introduced in the initial population initialization stage, and the final initial population is formed by the initial population and the opposition-based population, which optimizes the quality and convergence of the population; (2) the local search (LS) strategy is introduced to expand the diversity of populations by finding neighborhood solutions, in order to avoid solutions falling into local optima too early. NSGA-II/SDR-OLS is compared with the original algorithm on nine benchmark problems to verify the effectiveness of its improvement. Then, we compare our algorithm with six existing algorithms, which are promising region-based multi-objective evolutionary algorithms (PREA), a scalable small subpopulation-based covariance matrix adaptation evolution strategy (S3-CMA-ES), a decomposition-based multi-objective evolutionary algorithm guided by growing neural gas (DEA-GNG), a reference vector-guided evolutionary algorithm (RVEA), NSGA-II with conflict-based partitioning strategy (NSGA-II-conflict), and a genetic algorithm using reference-point-based non-dominated sorting (NSGA-III).The proposed algorithm has achieved the best results in the vast majority of test cases, indicating that our algorithm has strong competitiveness.

Agent-based mathematical model of COVID-19 spread in Novosibirsk region: Identifiability, optimization and forecasting

Abstract The problem of identification of unknown epidemiological parameters (contagiosity, the initial number of infected individuals, probability of being tested) of an agent-based model of COVID-19 spread in Novosibirsk region is solved and analyzed. The first stage of modeling involves data analysis based on the machine learning approach that allows one to determine correlated datasets of performed PCR tests and number of daily diagnoses and detect some features (seasonality, stationarity, data correlation) to be used for COVID-19 spread modeling. At the second stage, the unknown model parameters that depend on the date of introducing of containment measures are calibrated with the usage of additional measurements such as the number of daily diagnosed and tested people using PCR, their daily mortality rate and other statistical information about the disease. The calibration is based on minimization of the misfit function for daily diagnosed data. The OPTUNA optimization framework with tree-structured Parzen estimator and covariance matrix adaptation evolution strategy is used to minimize the misfit function. Due to ill-posedness of identification problem, the identifiability analysis is carried out to construct the regularization algorithm. At the third stage, the identified parameters of COVID-19 for Novosibirsk region and different scenarios of COVID-19 spread are analyzed in relation to introduced quarantine measures. This kind of modeling can be used to select effective anti-pandemic programs.

State-of-charge estimation of lithium-ion batteries based on MCC-AEKF in non-Gaussian noise environment

The extended Kalman filter (EKF) has low estimation accuracy for lithium-ion battery SOC under non-Gaussian noise interference. To solve this problem, an adaptive EKF based on maximum correlation-entropy criterion (MCC-AEKF) was proposed. In this algorithm, the maximum correlation entropy criterion is used to obtain accurate estimation results by replacing the overall solution with the local optimal solution. Meanwhile, the adaptive covariance matrix is established to update the process noise variance to reduce the SOC estimation error. MCC-AEKF were used to estimate the SOC based on two kinds of battery data. The experiments show that, under non-Gaussian noise interference and at operating temperature of 25 °C, compared with EKF and MCC-EKF, the estimation accuracy of MCC-AEKF improved by 80.3% and 24.2% separately for battery 1. For battery 2, its estimation accuracy improved by 72.6% and 22.0% separately, and are also the highest at 10 °C and 40 °C. Given wrong initial SOC values, MCC-AEKF can converges to real value within 30s. All results present the good accuracy and robustness of the proposed method under different cases.

An improved hyperparameter optimization framework for AutoML systems using evolutionary algorithms

For any machine learning model, finding the optimal hyperparameter setting has a direct and significant impact on the model’s performance. In this paper, we discuss different types of hyperparameter optimization techniques. We compare the performance of some of the hyperparameter optimization techniques on image classification datasets with the help of AutoML models. In particular, the paper studies Bayesian optimization in depth and proposes the use of genetic algorithm, differential evolution and covariance matrix adaptation—evolutionary strategy for acquisition function optimization. Moreover, we compare these variants of Bayesian optimization with conventional Bayesian optimization and observe that the use of covariance matrix adaptation—evolutionary strategy and differential evolution improves the performance of standard Bayesian optimization. We also notice that Bayesian optimization tends to perform poorly when genetic algorithm is used for acquisition function optimization.

Output-only structural identification with random decrement technique

Most of existing time domain methods fail to identify the unknown structural parameter in the ambient condition, because the force location is unknown and the number of unknown forces is usually larger than the number of sensors. A novel output-only structural identification method based on the random decrement (RD) technique and covariance matrix adaptation evolutionary strategy (CMA-ES) is presented. The RD functions of a structure under ambient excitation are extracted and treated as free vibration responses for structural parameter identification. The unknown structural parameters are identified by minimizing the discrepancy between the measured and calculated RD functions with the CMA-ES. Unlike traditional output-only methods, which require the estimation of the input excitation in the process, the proposed method does not require the identification of the input forces, thereby significantly reducing the computational cost. Numerical studies on two- and three-dimensional trusses and experimental studies on a steel frame are presented to demonstrate the accuracy and efficiency of the proposed approach. The effects of measurement noise and measurement duration on the identified results are examined. Results show that the proposed method can obtain satisfactory results even with 20% measurement noise and structural damage less than 10% can be detected. Comparison studies with the genetic algorithm (GA) and particle swarm optimization (PSO) are presented to verify the computational efficiency of the proposed method. Results also show that the proposed CMA-ES has a faster convergence speed than the GA and PSO method. The results demonstrate that the proposed technique can be used to detect damage severity and location in structures under ambient conditions.

A novel TomoSAR imaging method with few observations based on nested array

AbstractSynthetic aperture radar tomography (TomoSAR) baseline optimization method is capable of reducing system complexity and improving the temporal coherence of data, which has become important research in the field of TomoSAR. A nested TomoSAR technique is proposed, which introduces the nested array into TomoSAR as the baseline configuration. This technique obtains uniform and continuous difference co‐array through the nested array to increase the degrees of freedom of the system and expands the virtual aperture along the elevation direction. In order to make full use of the difference co‐array, covariance matrix of the echo needs to be obtained. Therefore, a TomoSAR sparse reconstruction algorithm based on the nested array is proposed, which uses adaptive covariance matrix estimation to improve the estimation performance in complex scenes. The effectiveness of the proposed method through simulated and real data experiments is demonstrated. Compared with traditional TomoSAR and coprime TomoSAR, the imaging results of this proposed method have better anti‐noise performance and retain more image information.

Detailed composition of iron ions in interplanetary coronal mass ejections based on a multipopulation approach

Context. Coronal mass ejections (CMEs) are extremely dynamical, large-scale events in which plasma – but not only the coronal plasma – is ejected into interplanetary space. If a CME is detected in situ by a spacecraft located in the interplanetary medium, it is then called an interplanetary coronal mass ejection (ICME). This solar activity has been studied widely since coronagraphs were first flown into space in the early 1970s. Aims. Charge states of heavy ions reflect important information about the coronal temperature profile due to the freeze-in effect and it is estimated that iron ions freeze in at heights of ∼5 solar radii. However, the measured charge-state distribution of iron ions cannot be composed of only one single group of plasma. To identify the different populations of iron charge-state composition of ICMEs and determine their sources, we developed a model that independently uses two, three, and four populations of iron ions to fit the measured charge-state distribution in ICMEs detected by the Advanced Composition Explorer (ACE) at 1 AU. Methods. Three parameters are used to identify a certain population, namely freeze-in temperature, relative abundance, and kappa value (κ), which together describe the potential non-Maxwellian kappa distributions of coronal electrons. Our method chooses the reduced chi-squared to describe the goodness of fit of the model to the observations. The parameters of our model are optimized with the covariance-matrix-adaptation evolution strategy (CMA-ES). Results. Two major types of ICMEs are identified according to the existence of hot material, and both, that is, the cool type and the hot type, have two main subtypes. Different populations in those types have their own features related to freeze-in temperature and κ. The electron velocity distribution function usually contains a significant hot tail in typical coronal material and hot material, while the Maxwellian distribution appears more frequently in mid-temperature material. Our model is also suitable for all types of solar wind and the existence of hot populations as well as the change of temperatures of individual populations may indicate boundaries between ICMEs and individual solar wind streams.

Multi-Objective Optimal Power Flow Solutions Using Improved Multi-Objective Mayfly Algorithm (IMOMA)

This paper realizes the implementation of Improved Multi-objective Mayfly Algorithm (IMOMA) for getting optimal solutions related to optimal power flow problem with smooth and nonsmooth fuel cost coefficients. It is performed by considering Simulated Binary Crossover, polynomial mutation and dynamic crowding distance in the existing Multi-objective Mayfly Algorithm. The optimal power flow problem is formulated as a Multi-objective Optimization Problem that consists of different objective functions, viz. fuel cost with/ without valve point loading effect, active power losses, voltage deviation and voltage stability. The performance of Improved Multi-objective Mayfly Algorithm is interpreted in terms of the present Multi-objective Mayfly Algorithm and Nondominated Sorting Genetic Algorithm-II. The algorithms are applied under different operating scenarios of the IEEE 30-bus test system, 62-bus Indian utility system and IEEE 118-bus test system with different combinations of objective functions. The obtained Pareto fronts achieved through the implementation of Improved Multi-objective Mayfly Algorithm, Multi-objective Mayfly Algorithm and Nondominated Sorting Genetic Algorithm-II are compared with the reference Pareto front attained by using weighted sum method based on the Covariance Matrix-adapted Evolution Strategy method. The performances of these algorithms are individually analyzed and validated by considering the performance metrics such as convergence, divergence, generational distance, inverted generational distance, minimum spacing, spread and spacing. The best compromising solution is achieved by implementing the Technique for Order of Preference by Similarity to Ideal Solution method. The overall result has shown the effectiveness of Improved Multi-objective Mayfly Algorithm for solving multi-objective optimal power flow problem.

Learning Emotion Assessment Method Based on Belief Rule Base and Evidential Reasoning

Learning emotion assessment is a non-negligible step in analyzing learners’ cognitive processing. Data are the basis of the learning emotion assessment. However, the existing learning emotion assessment models cannot balance model accuracy and interpretability well due to the influence of uncertainty in the process of data collection and model parameter errors. Given the above problems, a new learning emotion assessment model based on evidence reasoning and a belief rule base (E-BRB) is proposed in this paper. First, the transformation matrix is introduced to transform multiple emotional indicators into the same standard framework and integrate them, which keeps the consistency of information transformation. Second, the relationship between emotional indicators and learning emotion states is modeled by E-BRB in conjunction with expert knowledge. In addition, we employ the projection covariance matrix adaptation evolution strategy (P-CMA-ES) to optimize the model parameters and improve the model’s accuracy. Finally, to demonstrate the effectiveness of the proposed model, it is applied to emotion assessment in science learning. The experimental results show that the model has better accuracy than data-driven models such as neural networks.

Angle-Based Parametrization with Evolutionary Optimization for OESCL-Band Y-Junction Splitters

The design of passive photonic devices based on geometry optimization can lead to energy-efficient, small-footprint, and fabrication-ready geometries. In this work, we propose an angle-based parametrization method to optimize Y-junction splitters based on multimode interferometers. The selected figure of merit was the transmittance in the SCL and OESCL optical fiber communication bands. The performances of three optimization methods were compared: (i) particle swarm optimization (PSO), (ii) genetic algorithm (GA), and (iii) the covariance matrix adaptation-evolution strategy (CMA-ES). The results show that CMA-ES parametrization produces similar transmittance results (≤1.5% of difference) to PSO in the first 40 generations. The CMA-ES results are identical in the SCL (1460–1625 nm) and OESCL (1260–1625 nm) bands, whereas the GA and PSO executions are slightly different in terms of the rate and similarity of the figure of merit.

Controlling Sequential Hybrid Evolutionary Algorithm by Q-Learning [Research Frontier] [Research Frontier

Many state-of-the-art evolutionary algorithms (EAs) can be categorized as sequential hybrid EAs, in which various EAs are sequentially executed. The timing to switch from one EA to another is critical to the performance of the hybrid EA because the switching time determines the allocation of computational resources and thereby it helps balance exploration and exploitation. In this article, a framework for adaptive parameter control for hybrid EAs is proposed, in which the switching time is controlled by a learned agent rather than a manually designed scheme. First the framework is applied to an adaptive differential evolution algorithm, LSHADE, to control when to use the scheme to reduce the population. Then the framework is applied to the algorithm that won the CEC 2018 competition, i.e., the hybrid sampling evolution strategy (HSES), to control when to switch from the univariate sampling phase to the Covariance Matrix Adaptation Evolution Strategy phase. The agents for parameter control in LSHADE and HSES are trained by using Q-learning and deep Q-learning to obtain the learned algorithms Q-LSHADE and DQ-HSES. The results of experiments on the CEC 2014 and 2018 test suites show that the learned algorithms significantly outperform their counterparts and some state-of-the-art EAs.