Optimization In Dynamic Environments Research Articles

A survey of multi-population optimization algorithms for tracking the moving optimum in dynamic environments

AbstractThe solution spaces of many real-world optimization problems change over time. Such problems are called dynamic optimization problems (DOPs), which pose unique challenges that necessitate adaptive strategies from optimization algorithms to maintain optimal performance and responsiveness to environmental changes. Tracking the moving optimum (TMO) is an important class of DOPs where the goal is to identify and deploy the best-found solution in each environments Multi-population dynamic optimization algorithms are particularly effective at solving TMOs due to their flexible structures and potential for adaptability. These algorithms are usually complex methods that are built by assembling multiple components, each of which is responsible for addressing a specific challenge or improving the tracking performance in response to changes. This survey provides an in-depth review of multi-population dynamic optimization algorithms, focusing on describing these algorithms as a set of multiple cooperating components, the synergy between these components, and their collective effectiveness and/or efficiency in addressing the challenges of TMOs. Additionally, this survey reviews benchmarking practices within this domain and outlines promising directions for future research.

Online composite optimization with time-varying regularizers

This paper investigates online composite optimization in dynamic environments, where each objective or loss function contains a time-varying nondifferentiable regularizer. To resolve it, an online proximal gradient algorithm is studied for two distinct scenarios, including convex and strongly convex objectives without the smooth condition. In both scenarios, unlike most of works, an extended version of the conventional path variation is employed to bound the considered performance metric, i.e., dynamic regret. In the convex scenario, a bound O(T1−βDβ(T)+T) is obtained which is comparable to the best-known result, where Dβ(T) is the extended path variation with β∈[0,1) and T being the total number of rounds. In strongly convex case, a bound O(logT(1+T−βDβ(T))) on the dynamic regret is established. In the end, numerical examples are presented to support the theoretical findings.

Local feature matching and tracking optimization of machine VSLAM based on AG Sinkhorn

SummaryAt present, the visual Simultaneous localization and mapping method has a wide range of applications in mobile robots and unmanned driving. However, when the environment changes, the accuracy of the system drops sharply. Aiming at the low matching accuracy of local feature matching and tracking optimization in current visual simultaneous localization and mapping methods, the attention map neural network and Sinkhorn algorithm are used to extract and match the image features. Then, it improves the real‐time performance of the visual simultaneous localization and mapping system by local optimization of feature point matching, and proposes an optimization method for visual simultaneous localization and mapping in view of an end‐to‐end algorithm. The experiment illustrates that in the experimental comparison of HPatches View dataset, the Homography estimation was improved by 0.092, the matching accuracy was improved by 0.087, and the matching recall was improved by 0.038. In the experimental comparison of HPatches Illum dataset, the Homography estimation was improved by 0.015, the matching accuracy is improved by 0.076, and the matching recall was improved by 0.036. The local feature matching and tracking optimization method of the visual Simultaneous localization and mapping system through the attention map neural network and Sinkhorn algorithm can enhance the matching accuracy of local feature matching and tracking optimization in dynamic environment and under changing lighting conditions.

Topology-Driven Parallel Trajectory Optimization in Dynamic Environments

Topology-Driven Parallel Trajectory Optimization in Dynamic Environments

Byzantine-Resilient Distributed Bandit Online Optimization in Dynamic Environments

Byzantine-Resilient Distributed Bandit Online Optimization in Dynamic Environments

A Data Stream Ensemble Assisted Multifactorial Evolutionary Algorithm for Offline Data-Driven Dynamic Optimization.

Existing work on offline data-driven optimization mainly focuses on problems in static environments, and little attention has been paid to problems in dynamic environments. Offline data-driven optimization in dynamic environments is a challenging problem because the distribution of collected data varies over time, requiring surrogate models and optimal solutions tracking with time. This paper proposes a knowledge-transfer-based data-driven optimization algorithm to address these issues. First, an ensemble learning method is adopted to train surrogate models to leverage the knowledge of data in historical environments as well as adapt to new environments. Specifically, given data in a new environment, a model is constructed with the new data, and the preserved models of historical environments are further trained with the new data. Then, these models are considered to be base learners and combined as an ensemble surrogate model. After that, all base learners and the ensemble surrogate model are simultaneously optimized in a multitask environment for finding optimal solutions for real fitness functions. In this way, the optimization tasks in the previous environments can be used to accelerate the tracking of the optimum in the current environment. Since the ensemble model is the most accurate surrogate, we assign more individuals to the ensemble surrogate than its base learners. Empirical results on six dynamic optimization benchmark problems demonstrate the effectiveness of the proposed algorithm compared with four state-of-the-art offline data-driven optimization algorithms. Code is available at https://github.com/Peacefulyang/DSE_MFS.git.

Enhanced reinforcement learning in two-layer economic model predictive control for operation optimization in dynamic environment

Economic model predictive control (EMPC) has attracted an abundance of interest in both academic and industrial communities in recent years because it is able to increase the economic profits of dynamic systems. For greater computational efficiency, two-layer EMPC schemes are applied in some complex process control. However, the performance of two-layer EMPC is significantly influenced by the accuracy of the chosen process model. Reinforcement learning (RL) has been studied as a model-free strategy of model-based control approaches, but its safety and stability remain a concern. In order to estimate the model parameters of nonlinear dynamic systems in real time, this work introduces a unique scheme for merging two-layer EMPC and RL. In this scheme, the two-layer EMPC technique maintains closed-loop stability and recursive feasibility while operating the closed-loop dynamic system optimally. And the RL agent continually compares the observed states to the predictions made by the EMPC and modifies the time-varying parameters as necessary. The usability of the proposed scheme is shown on a chemical process of ethylene oxide production in dynamic environment. This work enables online and continuous control, optimization, and model correction, and makes process production optimization more feasible and profitable.

A modified Q-learning path planning approach using distortion concept and optimization in dynamic environment for autonomous mobile robot

Autonomous mobile robot path planning in unknown and dynamic environment is a crucial task for successful mobile robot navigation. This study proposes an improved Q-learning (IQL) algorithm to address the challenges of path planning in such environments. To this end, three different modes are introduced into the IQL algorithm, namely the normal mode, the distortion mode, and the optimization mode. The normal mode operates according to the standard Q-learning procedures. The distortion mode distorts the Q-values of states around dynamic obstacles to facilitate avoidance, while the optimization mode is employed to overcome the local minimum problem. The efficacy of the IQL algorithm is assessed through a series of comparative studies involving fourteen navigation environments, each with distinct obstacle layouts and types. Comparative analyses are performed based on several metrics, including computational time, travelled distance, collision rate, and success rate. The proposed IQL algorithm exhibits a lower collision rate and a higher success rate when compared to dynamic window approach, influence zone and inflated A*.

A Kriging model-based evolutionary algorithm with support vector machine for dynamic multimodal optimization

Dynamic multimodal optimization problems (DMMOPs) have to search multiple global optimal solutions with the objectives and constraints dynamically changing over time. In recent years, dynamic optimization problems and multimodal optimization problems have been extensively studied in the field of evolutionary computation. However, DMMOPs have not yet been paid significant attention and only a few studies have been designed for dynamic multimodal optimization. The key issue in optimizing DMMOPs is to address the challenges induced by both the multimodal nature and the dynamic nature. Existing works perform poorly in locating all global optima in static environments and tracking global optima with various change modes. Therefore, in this paper, a Kriging Model-based Evolutionary Algorithm with Support Vector Machine called KMEA-SVM is proposed for tackling DMMOPs. Two important operators are designed in this algorithm, including a Kriging-based preselection and a support vector machine (SVM)-based prediction. The aim of Kriging-based preselection is to search all global optimal solutions more efficiently by preselecting promising solutions with a trained Kriging model, while the purpose of SVM-based prediction is to predict more outstanding solutions as the initial population for new environment when the environment changes. The proposed KMEA-SVM is compared with several state-of-the-art evolutionary algorithms on twenty-four test DMMOPs and the experimental results validate the advantages of KMEA-SVM on seeking more multiple optima in dynamic environments.

Phase Transition Model between Swarm Behavior and Territorial Behavior

Abstract Swarm behavior is collective behavior of creatures moving with their neighbors, such as a flock of birds or a school of fishes. From the system optimization standpoint of view, there have been a number of “bioinspired” models proposed to apply the swarm behavior to organize complex systems and/or to incorporate robustness and flexibility. Territorial behavior is another group form of homogeneous individuals in which each one lives in a separate region (territory) from each other. For system optimization. The most notable application is coverage control. There are some creatures which exhibit either swarm behavior and territorial behavior according to their circumstances and conditions. However, as far as we know, there is no attempt toward any model or algorithm to combine both the swarm behavior and territorial behavior to make systems adapt to the change of circumstances and conditions. This paper proposes a phase transition model to combine swarm behavior and territorial behavior. The model is expected to apply to network optimization in dynamic environments, especially transition between cloud computing and edge computing. Through simulation-based experiments, we verify the convergence behavior of the model. Keywords: Swarm behavior, Territorial behavior, Dynamic transition

RL based hyper-parameters optimization algorithm (ROA) for convolutional neural network

AbstractMany real-world applications necessitate optimization in dynamic situations, where the difficulty is to locate and follow the optima of a time-dependent objective function. To solve dynamic optimization problems (DOPs), many evolutionary techniques have been created. However, more efficient solutions are still required. Recently, a new intriguing trend in dealing with optimization in dynamic environments has developed, with new reinforcement learning (RL) algorithms predicted to breathe fresh life into the DOPs community. In this paper, a new Q-learning RL-based optimization algorithm (ROA) for CNN hyperparameter optimization is proposed. Two datasets were used to test the proposed RL model (MNIST dataset, and CIFAR-10 dataset). Due to the use of RL for hyperparameter optimization, very competitive results and good performance were produced. From the experimental results, it is observed that the CNN optimized by ROA has higher accuracy than CNN without optimization. When using the MNIST dataset, it is shown that the accuracy of the CNN optimized by ROA when learning 5 epoch is 98.97%, which is greater than the 97.62% of the CNN without optimization. When using the CIFAR-10 dataset, it is shown that the accuracy of the CNN optimized by ROA when learning 10 epoch is 73.40 percent, which is greater than 71.73% of the CNN without optimization.

A Culture-Based Artificial Bee Colony Algorithm for Optimization in Dynamic Environments

A novel culture-based multiswarm artificial bee colony (CMABC) algorithm was proposed to address dynamic optimization problems. The historical experience of sub-swarms is preserved as cultural knowledge to guide the subsequent evolutionary process. Experiments were conducted on the moving peaks benchmark function. The results show that the CMABC algorithm was better than, or at least comparable to, the basic ABC algorithm, and other state-of-the-art algorithms.

Evolutionary-Mean shift algorithm for dynamic multimodal function optimization

Recently, many dynamic optimization algorithms based on metaheuristic methods have been proposed. Although these schemes are highly efficient in determining a single global optimum, they fail in locating multiple optimal solutions. The central goal of dynamic multimodal optimization is to detect multiple optimal solutions for an optimization problem where its objective function is modified over time. Locating many optimal solutions (global and local) in a dynamic multimodal optimization problem is particularly crucial for several applications since the best solution could not always be the best implementable alternative due to various practical limitations. In spite of its importance, the problem of dynamic multimodal optimization through evolutionary principles has been scarcely considered in the literature. On the other hand, mean shift is a non-parametric and iterative process for detecting local maxima in a density function represented by a set of samples. Mean shift maintains interesting adaptive characteristics that allow it to find local maxima under dynamic environments. In this paper, the mean shift scheme is proposed to detect global and local optima in dynamic optimization problems. In the proposed method, the search strategy of the mean shift is modified to consider not only the density but also the fitness value of the candidate solutions. A competitive memory, along with a dynamic strategy, has also been added to accelerate the convergence process by using important information from previous environments. As a result, the proposed approach can effectively identify most of the global and local optima in dynamic environments. To demonstrate the performance of the proposed algorithm, a set of comparisons with other well-known dynamic optimization methods has been conducted. The study considers the benchmark generator of the CEC competition for dynamic optimization. The experimental results suggest a very competitive performance of the proposed scheme in terms of accuracy and robustness.

Adaptive multi-swarm in dynamic environments

Abstract Multi-population is a promising approach to optimization in dynamic environments. To appropriately distribute multiple populations to distinct areas of the search space and refine the best solution found by each population, an adaptive multi-swarm framework for dynamic optimization problems is proposed, in which several adaptations of multi-population approaches are developed for a better exploration/exploitation tradeoff. As the first intention, a basic adaptation is the combination of a group of active swarms and a group of inactive swarms. The group of active swarms are devoted to exploring new areas of the search space, and the group of inactive swarms are devoted to preserving useful experiences. One kind of swarm can be transformed into another. An active swarm becomes inactive after it converges. An inactive swarm will become active and search for new optima again when an environmental change occurs. For the second intention, another basic adaptation is the application of a local search to the best individual of a stagnated swarm. The experimental results on various moving peaks benchmarks show that the proposed framework is competitive with other state-of-the-art methods and more effective for dynamic environments under many peaks, severe changes, and high dimensionalities.

Mobile Robot Path Planning Based on Time Taboo Ant Colony Optimization in Dynamic Environment.

This article aims to improve the problem of slow convergence speed, poor global search ability, and unknown time-varying dynamic obstacles in the path planning of ant colony optimization in dynamic environment. An improved ant colony optimization algorithm using time taboo strategy is proposed, namely, time taboo ant colony optimization (TTACO), which uses adaptive initial pheromone distribution, rollback strategy, and pheromone preferential limited update to improve the algorithm's convergence speed and global search ability. For the poor global search ability of the algorithm and the unknown time-varying problem of dynamic obstacles in a dynamic environment, a time taboo strategy is first proposed, based on which a three-step arbitration method is put forward to improve its weakness in global search. For the unknown time-varying dynamic obstacles, an occupancy grid prediction model is proposed based on the time taboo strategy to solve the problem of dynamic obstacle avoidance. In order to improve the algorithm's calculation speed when avoiding obstacles, an ant colony information inheritance mechanism is established. Finally, the algorithm is used to conduct dynamic simulation experiments in a simulated factory environment and is compared with other similar algorithms. The experimental results show that the TTACO can obtain a better path and accelerate the convergence speed of the algorithm in a static environment and can successfully avoid dynamic obstacles in a dynamic environment.

A Species Conservation-Based Particle Swarm Optimization with Local Search for Dynamic Optimization Problems.

In the optimization of problems in dynamic environments, algorithms need to not only find the global optimal solutions in a specific environment but also to continuously track the moving optimal solutions over dynamic environments. To address this requirement, a species conservation-based particle swarm optimization (PSO), combined with a spatial neighbourhood best searching technique, is proposed. This algorithm employs a species conservation technique to save the found optima distributed in the search space, and these saved optima either transferred into the new population or replaced by the better individual within a certain distance in the subsequent evolution. The particles in the population are attracted by its history best and the optimal solution nearby based on the Euclidean distance other than the index-based. An experimental study is conducted based on the moving peaks benchmark to verify the performance of the proposed algorithm in comparison with several state-of-the-art algorithms widely used in dynamic optimization problems. The experimental results show the effectiveness and efficiency of the proposed algorithm for tracking the moving optima in dynamic environments.

A multi-population differential evolution algorithm based on cellular learning automata and evolutionary context information for optimization in dynamic environments

This paper presents a multi-population differential evolution algorithm to address dynamic optimization problems. In the proposed approach, a cellular learning automaton adjusts the behavior of each subpopulation by adaptively controlling its updating schemes. As the environment changes over time, an evolving population may go through a quite number of state transitions. Each state demands specific characteristics from an optimizer; hence, an adapted evolutionary scheme for one state may be unsuitable for the upcoming ones. Additionally, a learning approach may have limited time to adapt to a newly encountered state due to the frequentness of the environmental changes. Hence, it is infeasible for a dynamic optimizer to unlearn its existing beliefs to accommodate the practices required to embrace newly encountered states. In order to address this issue, we introduce a context dependent learning approach, which can adapt the behavior of each subpopulation according to the contexts of its different states. The performance of the proposed approach is compared with several state-of-the-art dynamic optimizers over the GDBG benchmark set. Comparison results indicate that the proposed method can achieve statistically superior performances on a wide range of tested instances.

Adaptive online distributed optimization in dynamic environments

ABSTRACT This paper presents a dynamic regret analysis on the decentralized online convex optimization problems computed over a network of agents. The goal is to distributively optimize a global function which can be decomposed into the summation of local cost functions associated with each agent. By exploiting the convexity of cost functions, previous studies have shown that the dynamic regret of a decentralized gradient-based algorithm can be upper bounded by the path-length of the comparator sequence and the connectivity in the underlying network. In this paper, we illustrate that the dynamic regret can be further improved by allowing the learner to use a class of adaptive search directions and step-sizes from estimates of first and second moments of the gradients. Specifically, we develop a new class of decentralized and stochastic algorithms based on the adaptive gradient method in the dynamic environment and show that their regret bounds for certain realistic classes of loss functions are considerably better than existing bounds. Numerical simulations verify the theoretical results and demonstrate the efficiency of the new proposed method in practice.

Time Series Forecasting Using Neural Networks: Are Recurrent Connections Necessary?

Artificial neural networks (NNs) are widely used in modeling and forecasting time series. Since most practical time series are non-stationary, NN forecasters are often implemented using recurrent/delayed connections to handle the temporal component of the time varying sequence. These recurrent/delayed connections increase the number of weights required to be optimized during training of the NN. Particle swarm optimization (PSO) is now an established method for training NNs, and was shown in several studies to outperform the classical backpropagation training algorithm. The original PSO was, however, designed for static environments. In dealing with non-stationary data, modified versions of PSOs for optimization in dynamic environments are used. These dynamic PSOs have been successfully used to train NNs on classification problems under non-stationary environments. This paper formulates training of a NN forecaster as dynamic optimization problem to investigate if recurrent/delayed connections are necessary in a NN time series forecaster when a dynamic PSO is used as the training algorithm. Experiments were carried out on eight forecasting problems. For each problem, a feedforward NN (FNN) is trained with a dynamic PSO algorithm and the performance is compared to that obtained from four different types of recurrent NNs (RNN) each trained using gradient descent, a standard PSO for static environments and the dynamic PSO algorithm. The RNNs employed were an Elman NN, a Jordan NN, a multirecurrent NN and a time delay NN. The performance of these forecasting models were evaluated under three different dynamic environmental scenarios. The results show that the FNNs trained with the dynamic PSO significantly outperformed all the RNNs trained using any of the other algorithms considered. These findings highlight that recurrent/delayed connections are not necessary in NNs used for time series forecasting (for the time series considered in this study) as long as a dynamic PSO algorithm is used as the training method.

Estimation of distribution algorithms based on increment clustering for multiple optima in dynamic environments

Aiming to locate and track multiple optima in dynamic multimodal environments, an estimation of distribution algorithms based on increment clustering is proposed. The main idea of the proposed algorithm is to construct several probability models based on an increment clustering which improved performance for locating multiple local optima and contributed to find the global optimal solution quickly for dynamic multimodal problems. Meanwhile, a policy of diffusion search is introduced to enhance the diversity of the population in a guided fashion when the environment is changed. The policy uses both the current population information and the part history information of the optimal solutions available. Experimental studies on the moving peaks benchmark are carried out to evaluate the performance of the proposed algorithm in comparison with several state-of-the-art algorithms from the literature. The results show that the proposed algorithm is effective for the function with moving optimum and can adapt to the dynamic environments rapidly.