Parameter Adaptation Scheme Research Articles

Integrating reinforcement learning and supervisory control theory for optimal directed control of discrete-event systems

Directed control is crucial for implementing controllers on programmable logic controllers. A directed controller is one that actively selects at most one controllable event (control command) to execute at any instant. This study investigates the numerical optimization problem of directed control for discrete-event systems, such as traffic systems and robotic systems. By integrating supervisory control theory and reinforcement learning, an algorithmic procedure is designed to synthesize an optimal directed controller, which minimizes the total cost associated with event execution while ensuring the safety and liveness of the controlled system. Two improved Q-learning algorithms incorporating dynamic parameter adaptation strategies are developed to enhance the global search ability of basic Q-learning. To demonstrate applicability and effectiveness, we apply the proposed method to control a guideway system and a multi-train traffic system, respectively. The experimental results indicate the superiority of the proposed method over the comparison methods.

Voltage stability control strategy for DC microgrid based on adaptive virtual DC motor

The large-scale integration of distributed energy sources and power electronic devices results in the DC microgrid exhibiting significant low inertia and weak damping characteristics. This, in turn, leads to inevitable fluctuations in the DC bus voltage, which endanger the stable operation of the DC system. Energy storage devices can provide equivalent inertia. To enhance the inertia and response speed of the DC bus interface converter, this paper proposes a power allocation parameter adaptive virtual DC motor control strategy based on a hybrid energy storage unit. The strategy introduces power allocation control to regulate the energy storage converter on the basis of virtual inertia parameter adaptive control, thereby enabling the energy storage converter to simulate the inertia and damping characteristics of a DC generator. The small-signal stability of the system is analyzed by establishing a small-signal model of the photovoltaic energy storage system and utilizing the impedance ratio criterion. Finally, the proposed control strategy is validated through simulation. The results demonstrate that the strategy effectively mitigates the fluctuations in bus voltage under varying photovoltaic power and sudden load changes, ensures the power distribution in the hybrid energy storage system, and enhances the dynamic response of the system.

Hybrid golden jackal and golden sine optimizer for tuning PID controllers

In the domain of control engineering, effectively tuning the parameters of proportional-integral-derivative (PID) controllers has persistently posed a challenge. This study proposes a hybrid algorithm (HGJGSO) that combines golden jackal optimization (GJO) and golden sine algorithm (Gold-SA) for tuning PID controllers. To accelerate the convergence of GJO, a nonlinear parameter adaptation strategy is incorporated. The improved GJO is combined with Gold-SA, capitalizing on the expedited convergence speed offered by the improved GJO, coupled with the global optimization and precise search capabilities of Gold-SA. HGJGSO maximizes the strengths of two algorithms, facilitating a comprehensive and balanced exploration and exploitation. The effectiveness of HGJGSO is assessed through tuning the PID controllers for three typical systems. The results indicate that HGJGSO surpasses the comparison tuning methods. To evaluate the applicability of HGJGSO, it is used to tune the cascade PID controllers for trajectory tracking in a quadrotor UAV. The results demonstrate the superiority of HGJGSO in addressing practical challenges.

Microstimulation-based path tracking control of pigeon robots through parameter adaptive strategy

Research on animal robots utilizing neural electrical stimulation is a significant focus within the field of neuro-control, though precise behavior control remains challenging. This study proposes a parameter-adaptive strategy to achieve accurate path tracking. First, the mapping relationship between neural electrical stimulation parameters and corresponding behavioral responses is comprehensively quantified. Next, adjustment rules related to the parameter-adaptive control strategy are established to dynamically generate different stimulation patterns. A parameter-adaptive path tracking control strategy (PAPTCS), based on fuzzy control principles, is designed for the precise path tracking tasks of pigeon robots in open environments. The results indicate that altering stimulation parameter levels significantly affects turning angles, with higher UPN and PTN inducing changes in the pigeons' motion state. In experimental scenarios, the average control efficiency of this system was 82.165%. This study provides a reference method for the precise control of pigeon robot behavior, contributing to research on accurate target path tracking.

A nonlinear randomly reuse-based mutated whale optimization algorithm and its application for solving engineering problems

The whale optimization algorithm (WOA) is a meta-heuristic optimization algorithm inspired by the social behavior of whale hunting. However, the WOA has the defects of easily falling into local optimum and slow convergence speed. Therefore, to overcome the shortcomings of the original WOA, a novel variant of WOA called nonlinear randomly reuse-based mutated whale optimization algorithm (NRRMWOA) is proposed in this study. In detail, the proposed NRRMWOA includes three novel strategies as compared with original WOA. Firstly, a nonlinear adaptive parameter strategy is introduced to achieve nonlinear adjustment of search pattern along with the iteration time. The second improvement is the random reuse strategy, which can fully utilize the current optimal whale to improve the solution accuracy. The thirdly proposed late disturbance mutation strategy has the function of increasing population diversity in the later stage of iteration. The three proposed strategies complement each other and significantly improve the convergence efficiency and global exploration capability of WOA. To verify the superiority of the proposed NRRMWOA, a series of typical benchmark functions are utilized to conduct comprehensive numerical experiments. Compared with three sets of comparison algorithms (five variants of WOA, five advanced algorithms, and five well-known algorithms), the ARVs (average ranking values) obtained by NRRMWOA are 1.4667, 1.9167, and 2.1333, respectively. The rankings are all the best in the corresponding group. The detailed experimental and statistics results indicate that the presented NRRMWOA has higher convergence speed and better solution quality compared with the three sets of existing algorithms. In addition, 15 real-world optimization problems are employed to further test the performance of NRRMWOA for solving engineering optimization problems. For the results of 15 real-world optimization problems, the proposed NRRMWOA achieves the best results including 8 best “Mean”, 10 best “Min.”, and 7 best “Max.”. As a conclusion, the proposed NRRMWOA is a promising and excellent algorithm.

A triple population adaptive differential evolution

The Differential Evolution (DE) algorithm is one of the most efficient algorithms for complex numerical optimization. However, the nature of differential mutation and crossover hinders the individuals from a major change and always guides them toward their superior neighbors. There's a lack of useful directional information to help the population escape from early convergence. To solve the above problem, this paper proposes a novel Triple-population-based Adaptive Differential Evolution (TPADE) to enhance the evolutionary efficiency in solving various complex numerical optimization problems. First, a population division method with symmetrical linear reduction is designed to divide the parent population of each iteration into three sub-populations of different sizes, i.e., superior sub-population, medium sub-population, and inferior sub-population. Each sub-population adopts distinct differential mutation and crossover operators to maintain balanced search directions. Second, a superior-trial-preserved selection mechanism is proposed to screen useful directional information to guide the next iteration of evolution. Third, an effective parameter adaptation strategy is designed with the linear population size reduction strategy to avoid redundant search. Experiments are then conducted to show that the TPADE exhibits well performance compared with eleven state-of-the-art DE variants, CEC winners, and their variants on the CEC'2014, CEC'2017, and CEC'2022 benchmark suites. The C++ source code of TPADE can be downloaded from https://github.com/DoubleGong/TPADE.

APM: Adaptive parameter multiplexing for class incremental learning

In recent developments within the domain of image classification, deep neural networks (DNNs) have attracted considerable scholarly interest and have been extensively trained using data in closed environments. Such training methodologies contrast sharply with the inherently open, progressive, and adaptive processes of the natural visual system, leading to emergent challenges. Among these, catastrophic forgetting is notable, where the network acquisition of new class information precipitates the erosion of previously established knowledge. Additionally, the network encounters the stability-plasticity dilemma, necessitating a delicate equilibrium between assimilating novel classes and retaining existing ones. To address these issues, we propose a novel incremental learning model, termed Adaptive Parameter Multiplexing (APM), which incorporates a cross-class parameter adaptive incremental strategy. Central to our methodology is the conceptualization of parameter multiplexing or incremental as a learnable optimization problem, enabling the model to autonomously evaluate and decide on the necessity for parameter adjustment throughout its training lifecycle. This framework is designed to enhance the ability of the network to extract features for new class categories effectively through incremental parameters while simultaneously employing parameter multiplexing to augment storage optimization. Our model is underpinned by a dual strategy of coarse-grained and fine-grained parameter multiplexing, guided by a learnable score that dynamically assesses the appropriateness of parameter multiplexing versus incremental updates, facilitating an optimized balance for incremental model performance and storage. In addition, we have integrated a novel regularization loss mechanism for the learnable score to optimize storage efficiency. The effectiveness of APM is empirically validated through rigorous testing on benchmark datasets, including ImageNet100, CIFAR100, CIFAR10, and CUB200. The experimental outcomes indicate that, with a trace amount of parameter increase, our model achieves significant enhancements in classification performance across both new and previously established classes, thereby surpassing existing benchmarks set by state-of-the-art algorithms in the field.

Photovoltaic model parameters identification using diversity improvement-oriented differential evolution

Fast and accurate parameter identification of the photovoltaic (PV) model is crucial for calculating, controlling, and managing PV generation systems. Numerous meta-heuristic algorithms have been applied to identify unknown parameters due to the multimodal and nonlinear characteristics of the parameter identification problems. Although many of them can obtain satisfactory results, problems such as premature convergence and population stagnation still exist, influencing the optimization performance. A novel variant of Differential Evolution, namely, Diversity Improvement-Oriented Differential Evolution (DIODE), is proposed to mitigate these deficiencies and obtain reliable parameters for PV models. In DIODE, an adaptive perturbation strategy is employed to perturb current individuals to mitigate premature convergence by enhancing population diversity. Secondly, a diversity improvement mechanism is proposed, where information on the covariance matrix and fitness improvement of individuals is used as a diversity indicator to detect stagnant individuals, which are then updated by the intervention strategy. Lastly, a novel parameter adaptation strategy is employed to maintain a sound balance between exploration and exploitation. The proposed DIODE algorithm is applied to parameter identification problems of six PV models, including single, double, and triple diode and three PV module models. In addition, a large test bed containing 72 benchmark functions from CEC2014, CEC2017, and CEC2022 test suites is employed to verify DIODE’s overall performance in terms of optimization accuracy. Experiment results demonstrate that DIODE can secure accurate parameters of PV models and achieve highly competitive performance on benchmark functions.

Adaptive control strategy of VSG parameters based on improved fuzzy control

Abstract A virtual synchronous generator (VSG) enables the converter output to have similar external characteristics to the synchronous generator, but its virtual rotary inertia and damping factor need to be flexibly adapted to strengthen the dynamic stability of the system. Given the shortcomings of existing adaptive strategies, this paper presents a VSG parameter adaptive control strategy based on improved fuzzy control. GRO is used to optimize the quantization factor and scale factor of the fuzzy controller, to strengthen the adaptive ability of the fuzzy controller to the rotary inertia and damping factor. Simulink simulation is used to verify the superiority of the control strategy.

An Underwater Passive Electric Field Positioning Method Based on Scalar Potential

In order to fulfill the practical application demands of precisely localizing underwater vehicles using passive electric field localization technology, we propose a scalar-potential-based method for the passive electric field localization of underwater vehicles. This method is grounded on an intelligent differential evolution algorithm and is particularly suited for use in three-layer and stratified oceanic environments. Firstly, based on the potential distribution law of constant current elements in a three-layer parallel stratified ocean environment, the mathematical positioning model is established using the mirror method. Secondly, the differential evolution (DE) algorithm is enhanced with a parameter-adaptive strategy and a boundary mutation processing mechanism to optimize the key objective function in the positioning problem. Additionally, the simulation experiments of the current element in the layered model prove the effectiveness of the proposed positioning method and show that it has no special requirements for the sensor measurement array, but the large range and moderate number of sensors are beneficial to improve the positioning effect. Finally, the laboratory experiments on the positioning method proposed in this paper, involving underwater simulated current elements and underwater vehicle tracks, were carried out successfully. The results indicate that the positioning method proposed in this paper can achieve the performance requirements of independent initial value, strong anti-noise capabilities, rapid positioning speed, easy implementation, and suitability in shallow sea environments. These findings suggest a promising practical application potential for the proposed method.

Reinforcement learning-assisted evolutionary algorithm: A survey and research opportunities

Evolutionary algorithms (EA), a class of stochastic search methods based on the principles of natural evolution, have received widespread acclaim for their exceptional performance in various real-world optimization problems. While researchers worldwide have proposed a wide variety of EAs, certain limitations remain, such as slow convergence speed and poor generalization capabilities. Consequently, numerous scholars actively explore improvements to algorithmic structures, operators, search patterns, etc., to enhance their optimization performance. Reinforcement learning (RL) integrated as a component in the EA framework has demonstrated superior performance in recent years. This paper presents a comprehensive survey on integrating reinforcement learning into the evolutionary algorithm, referred to as reinforcement learning-assisted evolutionary algorithm (RL-EA). We begin with the conceptual outlines of reinforcement learning and the evolutionary algorithm. We then provide a taxonomy of RL-EA. Subsequently, we discuss the RL-EA integration method, the RL-assisted strategy adopted by RL-EA, and its applications according to the existing literature. The RL-assisted procedure is divided according to the implemented functions including solution generation, learnable objective function, algorithm/operator/sub-population selection, parameter adaptation, and other strategies. Additionally, different attribute settings of RL in RL-EA are discussed. In the applications of RL-EA section, we also demonstrate the excellent performance of RL-EA on several benchmarks and a range of public datasets to facilitate a quick comparative study. Finally, we analyze potential directions for future research. This survey serves as a rich resource for researchers interested in RL-EA as it overviews the current state-of-the-art and highlights the associated challenges. By leveraging this survey, readers can swiftly gain insights into RL-EA to develop efficient algorithms, thereby fostering further advancements in this emerging field.

A cooperative interaction strategy for vehicle platoons to obtain merging gaps in connected environments

Vehicle platoons can significantly improve traffic throughput and reduce fuel consumption and emissions. In the formation of platoons, it is crucial to generate safe merging gaps. This process requires appropriate cooperative management and control strategies. This study proposes a cooperative interaction strategy for vehicle platoons, including a communication management system and vehicle control strategies. These strategies can reduce velocity fluctuation in the process of gap generation and improve traffic performance. First, a communication management system within a platoon was developed, according to the standard communication protocol (SAE J2735), ensuring that external vehicles can join the platoon efficiently and orderly. Next, a cooperative adaptive cruise control (CACC) system was designed, which adopts feedforward and feedback control. Furthermore, the influence of increasing gaps on the stability of the platoon was considered. A cooperative control strategy for a virtual guiding vehicle (VGV) was introduced to switch the following target and linearly change the distance input of the controller. In this way, the downstream vehicles were guided to smoothly generate a safe merging gap, which can reduce speed fluctuation, and ensure the stability and safety of the platoon. Finally, the entire process of interaction in a vehicle platoon was tested in a simulation environment. The results showed that, compared with the parameter adaptive control strategy, the maximum velocity overshoot of the platoon vehicles was reduced by 56%, recovery stabilization time was reduced by 47%, and vehicle jitter was reduced by 43%. The driving security and platoon stability were both within the control boundaries set for evaluation.

Differential Evolution with perturbation mechanism and covariance matrix based stagnation indicator for numerical optimization

Differential Evolution (DE), as a promising population-based stochastic optimization algorithm, has drawn attention from researchers of various fields owing to its simple operation, strong robustness and few control parameters. However, classic DE suffers from drawbacks such as premature convergence and stagnation resulted from reduced population diversity when tackling complicated optimization problems with high dimensions. To mitigate these deficiencies, Differential Evolution with Perturbation mechanism and Covariance Matrix based stagnation indicator (PCM-DE) is proposed in this paper. There are three main modifications in the algorithm. First, a two-phase parameter adaptation strategy based on fitness value is proposed to guide the search towards promising direction according to different stages of evolution. Second, a perturbation mechanism for archived population is proposed, where a new weight coefficient is defined by exploiting information of fitness value and position in archived individuals. Third, a stagnation indicator based on covariance matrix is proposed to assess the population diversity and the information of variance is employed to perturb stagnant individuals. To validate the effectiveness of PCM-DE, it is compared with several state-of-the-art DE variants and non-DE based algorithms under a large test suite containing 100 benchmark functions. Besides, three truss optimization problems are also employed to verify the feasibility and scalability of the algorithm. The experiment results confirm its highly competitive performance in terms of solution accuracy and convergence speed.

A modified adaptive beluga whale optimization based on spiral search and elitist strategy for short-term hydrothermal scheduling

Short-term hydrothermal scheduling is a complex dynamic nonlinear optimization problem which considers various hydropower constraints. In this study, an adaptive beluga whale optimization method based on a spiral search and elitist strategy is proposed to solve this complex problem. The proposed method has three main advantages: First, an adaptive parameter strategy based on a cosine function achieves a better balance between exploration and exploitation. Second, the spiral search and elite strategy guides search agents to search for focused areas and effectively narrows the scope of the invalid search space. Third, instead of the penalty function method, a feasibility-based selection comparison technique and heuristic strategy are adopted to address complex constraints in STHS. In comparison to the results of benchmark functions from CEC 2015, ASEBWO generally demonstrates superior convergence accuracy when compared to the other seven algorithms. Additionally, the feasibility and effectiveness of ASEBWO are verified using two hydrothermal scheduling systems and compared with those of more than 10 algorithms. The comparison results demonstrate that ASEBWO achieves the best solution quality.

Automated rail-water intermodal transport container terminal handling equipment cooperative scheduling based on bidirectional hybrid flow-shop scheduling problem

The importance of rail-water intermodal transport is becoming more and more prominent in recent years. As a key node of rail-water intermodal transportation, the operation efficiency of automated container terminal directly affects the overall transportation efficiency. However, there are few studies on the collaborative scheduling of automated container terminal equipment in rail-water intermodal transportation. Therefore, this paper focuses on the ‘Automated Quay Crane (AQC)-Automated Straddle Carrier (ASC)-Automated Rail Mounted Gantry Crane (ARMG)’ operation system and addresses the operational efficiency requirements of the Automated Rail-Water Intermodal Transport Container Terminal (ARWITCT), Based on the bidirectional Hybrid Flow-shop Scheduling Problem (HFSP), a scheduling optimization model for the ARWITCT is established with the objective of minimizing the maximum completion time. To solve this complex scheduling problem, Particle Swarm Optimization based on Adaptive Tent Chaos Mapping (PSO-ATCM) is designed. In this algorithm, a parameter adaptive adjustment strategy and a Tent Chaos Mapping Optimization Strategy are introduced to help PSO jump out of local optimum. Problem-feature-based encoding and decoding methods are designed for the problem. An ‘insertion-translation’ mechanism is used to judge and resolve the conflicts between import and export containers during the decoding process. The effect of proposed scheduling model and the performance of the proposed algorithm are validated via comprehensive comparison experiments. To validate the effectiveness of proposed scheduling model, we compare the unidirectional HFSP model with the bidirectional HFSP model. The proposed PSO-ATCM is compared with standard Genetic Algorithm, Particle Swarm Optimization algorithm and Adaptive Genetic Algorithm. Computational results demonstrate that the proposed model is effective and PSO-ATCM performs better than other three compared algorithms.

A Game Model for Analyzing Wireless Sensor Networks of 5G Environment Based on Adaptive Equilibrium Optimizer Algorithm.

Wireless sensors networks (WSNs) play an important role in life. With the development of 5G, its security issues have also raised concerns. Therefore, it is an important topic to study the offense and defense confrontation in WSNs. A complete information static game model is established to analyze the offense and defense confrontation problem of WSNs in 5G. An adaptive equilibrium optimizer algorithm (AEO) based on parameter adaptive strategy is proposed, which can jump out of the local optimal solution better. Experiments show that the optimization ability of AEO outperforms other algorithms on at least 80% of the 23 classical test functions of CEC. The convergence speed of AEO is better in the early stage of population iteration. The optimal offensive and defensive strategy under different offense and defense resources through simulation experiments is analyzed. The conclusion shows that when the offensive resources are large, the offender takes an indiscriminate attack. When the defense resources are small, the defender should defend the most important elements, and when the defense resources are large, the defender should allocate the same resources to defend each element to obtain the maximum benefit. This paper provides new solution ideas for the security problems under the offense and defense game in WSNs.

COLREGs-compliant unmanned surface vehicles collision avoidance based on improved differential evolution algorithm

Unmanned surface vessel (USV) has a wide range of applications in oceanographic research, resource development, environment detection, and security rescue due to its advantages of maneuverability, flexibility, fast response, and intelligence. The ability of USVs to autonomously and effectively avoid obstacles in highly dynamic and uncertain marine environments is a prerequisite for the successful completion of their tasks. Therefore, in this article, a USV collision avoidance based on International Regulations for Preventing Collisions at Sea and the Collision Risk Model with the Improved Differential Evolution Algorithm (CRI-DE) has been considered. Based on the International Regulations for Preventing Collisions at Sea (COLREGs) and common practices of seafarers, an improved ship collision risk model is proposed. Specifically, the model is innovatively combined with the differential evolution algorithm (DE) as a constraint condition to further realize path planning in complex situations. Moreover, chaotic multi-population parallel optimization, parameter adaptive adjustment strategy, and the construction of fitness function based on individual path points are added to the DE. In this way, the ability to escape from local optima and enrich population diversity can be guaranteed. Finally, experiments based on the proposed CRI-DE are conducted and the results indicate the efficiency and effectiveness of the proposed method.

Parameter adaptive model predictive control strategy of NPC three-level virtual synchronous generator

The Virtual Synchronous Generator (VSG) emulates the characteristics of a synchronous generator to provide inertia and damping for renewable energy systems. In the case of using the NPC three-level converter structure, traditional control methods require complex dual-loop control and internal PI parameter tuning. Furthermore, although fixed-parameter VSG control can provide inertia and damping when a significant power load is switched in an islanded microgrid, it cannot guarantee frequency regulation performance. To address these issues, this paper proposes an NPC three-level VSG parameter adaptive finite control set model predictive control strategy. This method eliminates the need for dual-loop control and PI parameter tuning. By incorporating angular velocity deviation and its rate of change into adaptive adjustment, a Tracking-Differentiator (TD) is designed to calculate the rate of change of angular velocity. This approach avoids frequent fluctuation of adaptive parameters during load power switching and improves the frequency stability of the microgrid. The effectiveness of the proposed strategy is validated through simulation and experimental verification.

A novel adaptive parameter strategy differential evolution algorithm and its application in midcourse guidance maneuver decision-making

In this paper, a novel Adaptive Parameter Strategy Differential Evolution (APSDE) algorithm is proposed to overcome the parameters dependence and avoid local optima. The Parameter Update Mechanism (PUM), which has three different strategies, is used to reduce the dependence on parameters of DE. The Adaptive Proportion Adjustment Mechanism (APAM) is used to balance the proportion of PUM strategies in different development terms of exploitation and exploration, and the Random Restart Mechanism (RRM) is used to improve population diversity when exploitation is in stagnation. The proposed algorithm is verified in the CEC2018 test functions and the results show that APSDE has good abilities of exploitation, exploration, convergence, and stability. Secondly, Midcourse Guidance Maneuver Decision-making (MGMD) in Beyond Visual Range (BVR) air combat is studied and transformed into a single objective variational optimization problem, a MGMD system based on APSDE is established. Finally, the simulation of MGMD is carried out. The APSDE ranks first in the typical MGMD scenario experiment. In the adaptive Midcourse guidance confrontation, the winning rate of APSDE is 54%, and the statistical results show that the APSDE has an excellent MGMD ability.

Reinforcement learning-based hybrid differential evolution for global optimization of interplanetary trajectory design

The global optimization and design of interplanetary trajectories is one of the most important tasks in deep space exploration. Its search space is often characterized by multi-constraints, extreme non-linearity and sensitivity to initial conditions. To cope with these difficulties, an improved reinforcement learning-based hybrid differential evolution (DE) named RL_HDE is proposed in this paper. In RL_HDE, a novel multi-mutation strategy LSHADE based on an adaptive Q-Learning framework is proposed for global exploration. To further balance the global exploration and local exploitation of RL_HDE, a new parameter adaptive strategy based on Q-Learning is designed to control two trigger parameters. The performance of RL_HDE is verified by the well-known Global Trajectory Optimization Problems (GTOP), which are developed by Advanced Concepts Team of European Space Agency (ESA-ACT). A comparison of RL_HDE with three sets of algorithms, including ten state-of-the-art hybrid evolutionary algorithms, four interplanetary trajectory design algorithms, and seven reinforcement learning-based hybridizations. Experimental statistical results demonstrate that RL_HDE outperforms other competitors in terms of convergence efficiency and accuracy. RL_HDE has better performance for solving complex interplanetary trajectory design problems such as Cassini2 and Messenger-full.