Continuous Optimization Problems Research Articles

Surrogate-assisted evolutionary algorithms for expensive combinatorial optimization: a survey

AbstractAs potent approaches for addressing computationally expensive optimization problems, surrogate-assisted evolutionary algorithms (SAEAs) have garnered increasing attention. Prevailing endeavors in evolutionary computation predominantly concentrate on expensive continuous optimization problems, with a notable scarcity of investigations directed toward expensive combinatorial optimization problems (ECOPs). Nevertheless, numerous ECOPs persist in practical applications. The widespread prevalence of such problems starkly contrasts the limited development of relevant research. Motivated by this disparity, this paper conducts a comprehensive survey on SAEAs tailored to address ECOPs. This survey comprises two primary segments. The first segment synthesizes prevalent global, local, hybrid, and learning search strategies, elucidating their respective strengths and weaknesses. Subsequently, the second segment furnishes an overview of surrogate-based evaluation technologies, delving into three pivotal facets: model selection, construction, and management. The paper also discusses several potential future directions for SAEAs with a focus towards expensive combinatorial optimization.

A Reentry Trajectory Planning Algorithm via Pseudo-Spectral Convexification and Method of Multipliers

The reentry trajectory planning problem of hypersonic vehicles is generally a continuous and nonconvex optimization problem, and it constitutes a critical challenge within the field of aerospace engineering. In this paper, an improved sequential convexification algorithm is proposed to solve it and achieve online trajectory planning. In the proposed algorithm, the Chebyshev pseudo-spectral method with high-accuracy approximation performance is first employed to discretize the continuous dynamic equations. Subsequently, based on the multipliers and linearization methods, the original nonconvex trajectory planning problem is transformed into a series of relaxed convex subproblems in the form of an augmented Lagrange function. Then, the interior point method is utilized to iteratively solve the relaxed convex subproblem until the expected convergence precision is achieved. The convex-optimization-based and multipliers methods guarantee the promotion of fast convergence precision, making it suitable for online trajectory planning applications. Finally, numerical simulations are conducted to verify the performance of the proposed algorithm. The simulation results show that the algorithm possesses better convergence performance, and the solution time can reach the level of seconds, which is more than 97% less than nonlinear programming algorithms, such as the sequential quadratic programming algorithm.

Reentry trajectory planning for hypersonic vehicles via an improved sequential convex programming method

The reentry trajectory planning problem of hypersonic vehicles is generally a continuous and nonconvex optimization problem. The solution efficiency and convergence properties of the planning algorithm become an essential point when considering the practical application scenarios. In this paper, an augmented-Lagrange-based improved sequential convex programming algorithm is proposed to solve it and achieve superior solutions with higher accuracy and efficiency. Firstly, the mapped Chebyshev pseudo-spectral method is utilized to convert the continuous optimal control problem into a highly approximately discretized one. In this framework, the discretization technique is employed to discretize the states and control variables at the time interval, which leads to the improvement of ill-conditioned differential matrix and better approximation properties of the variables. Moreover, by introducing slack variables, automatically updated penalty parameters and Lagrange multipliers during the convexification processing, the original nonconvex problem is transformed into an augmented-Lagrange-function-formed relaxed convex one. Then, the arrived convex subproblem can be solved iteratively by the matured interior-point method until the predetermined precision is satisfied. Based on this transformation, it can effectively alleviate the numerical difficulty and improve the convergence property of the proposed improved sequential convex programming algorithm. Finally, Rigorous theoretical analysis is conducted to prove that the solution sequence generated by the proposed algorithm could converge to an approximate global solution of the original problem. Numerical simulations are also presented, and it is demonstrated that the solution time of the proposed algorithm is more than 95% less than that of the traditional nonlinear programming methods without losing the merits of high precision.

Enhancing forest optimization algorithm with gravitational search for nonlinear continuous optimization

Meta-heuristic algorithms have great role in solving problems related to optimization. Meta-heuristic method cannot solve problems related to optimization due to No Free Lunch theory. Hence different optimization methods are proposed by various researchers each year in order to solve optimization problems. Forest Optimization Algorithm (FOA) is an evolutionary optimization algorithm that is appropriate for continuous nonlinear optimization problems. The algorithm drawbacks include entrapment in local optimum and failure in achieving global optimum. The paper proposes hybrid algorithm called FOAGSA, in which the Gravitational Search Algorithm (GSA) is employed to improve the FOA performance in order to solve nonlinear continuous problems. The FOAGSA was evaluated through 39 benchmark optimization functions and two engineering problems. The experimental results proved that the FOAGSA exhibited acceptable results compared to state-of-art and well-known Meta-heuristic algorithms. Friedman ranking algorithm was utilized to compare FOAGSA with existing methods. The FOAGSA was ranked first on that basis.

Oscillatory Neural Network-Based Ising Machine Using 2D Memristors.

Neural networks are increasingly used to solve optimization problems in various fields, including operations research, design automation, and gene sequencing. However, these networks face challenges due to the nondeterministic polynomial time (NP)-hard issue, which results in exponentially increasing computational complexity as the problem size grows. Conventional digital hardware struggles with the von Neumann bottleneck, the slowdown of Moore's law, and the complexity arising from heterogeneous system design. Two-dimensional (2D) memristors offer a potential solution to these hardware challenges, with their in-memory computing, decent scalability, and rich dynamic behaviors. In this study, we explore the use of nonvolatile 2D memristors to emulate synapses in a discrete-time Hopfield neural network, enabling the network to solve continuous optimization problems, like finding the minimum value of a quadratic polynomial, and tackle combinatorial optimization problems like Max-Cut. Additionally, we coupled volatile memristor-based oscillators with nonvolatile memristor synapses to create an oscillatory neural network-based Ising machine, a continuous-time analog dynamic system capable of solving combinatorial optimization problems including Max-Cut and map coloring through phase synchronization. Our findings demonstrate that 2D memristors have the potential to significantly enhance the efficiency, compactness, and homogeneity of integrated Ising machines, which is useful for future advances in neural networks for optimization problems.

Geometric batch optimization for packing equal circles in a circle on large scale

The problem of packing equal circles in a circular container is a classic and famous packing problem, which is well-studied in academia and has a variety of applications in industry. The problem is computationally challenging, and researchers mainly focus on small to moderate-scale instances with the number of circular items n less than 320 in the literature. In this work, we aim to solve this problem on large scale, and it is the bottleneck for most global optimization problems. Specifically, we propose a novel geometric batch optimization method that makes batch gradient descent based on the geometric locations of packing items. This method not only can speed up the convergence process of continuous optimization significantly but also reduce the memory requirement during the program’s runtime. Then, we propose a heuristic search method, called solution-space exploring and descent, that can discover a feasible solution efficiently on large scale. Besides, we propose an adaptive neighbor object maintenance method to maintain the neighbor structure applied in the continuous optimization process. In this way, we can find high-quality solutions on large-scale instances within reasonable computational times. Extensive experiments on the benchmark instances sampled from n= 300 to 1000 show that our proposed algorithm outperforms the state-of-the-art algorithms and performs excellently on large-scale instances. In particular, our algorithm found 10 improved solutions among the 21 well-studied moderate-scale instances and 95 improved solutions among the 101 sampled large-scale instances. Furthermore, our geometric batch optimization, heuristic search, and adaptive maintenance methods are general and can be adapted to other packing and continuous optimization problems.

On the adaptation of the Lagrange formalism to continuous time stochastic optimal control: A Lagrange-Chow redux

We show how the classical Lagrangian approach to solving constrained optimization problems from standard calculus can be extended to solve continuous time stochastic optimal control problems. Connections to mainstream approaches such as the Hamilton-Jacobi-Bellman equation and the stochastic maximum principle are drawn. Our approach is linked to the stochastic maximum principle, but more direct and tied to the classical Lagrangian principle, avoiding the use of backward stochastic differential equations in its formulation. Using infinite dimensional functional analysis, we formalize and extend the approach first outlined in Chow (1992) within a rigorous mathematical setting using infinite dimensional functional analysis. We provide examples that demonstrate the usefulness and effectiveness of our approach in practice. Further, we demonstrate the potential for numerical applications facilitating some of our key equations in combination with Monte Carlo backward simulation and linear regression, therefore illustrating a completely different and new avenue for the numerical application of Chow's methods.

OBABC: A one-dimensional binary artificial bee colony algorithm for binary optimization

Artificial bee colony (ABC) algorithm is a widely utilized swarm intelligence (SI) algorithm for addressing continuous optimization problems. However, most binary variants of ABC (BABC) algorithms may suffer from issues such as invalid searches and high complexity when applied to binary problems. To address these challenges, we first establish a set of criteria for developing a BABC algorithm. Following these criteria, we propose a novel BABC algorithm, denoted as oBABC, which not only adheres to the defined criteria but also successfully inherits the advantages of original ABC algorithm. To evaluate the performance of oBABC and verify its effectiveness, experiments are conducted on two typical binary problems: uncapacitated facility location problem (UFLP) and maximum cut problem (Max-Cut). The experimental results reveal the following findings: 1) The validity of the criteria and the accuracy of the theoretical analysis are confirmed. oBABC exhibits high search efficiency with an invalid learning rate (ILR) of 0 %, while the ILRs of other BABC algorithms almost exceeds 20 %. 2) In terms of search efficiency and capability, oBABC exhibits a significant improvement in search efficiency and consistently ranks at the top in terms of optimization capability. These results suggest that oBABC may be a highly efficient and effective tool for solving binary problems.

Quaternary Continuous Classical Boundary Optimal Control Problem Dominating by Parabolic System

In this paper, our purpose is to study the quaternary continuous classical boundary optimal control vector problem dominated by a quaternary linear parabolic boundary value problem. Under suitable assumptions and with a given quaternary continuous classical boundary control vector, the existence theorem for a unique quaternary state vector solution to the weak form is stated and demonstrated via the method of Galerkin. Furthermore, the continuity of the Lipschitz operator between the state vector solution to the weak form for the dominating equations and the corresponding are proved. The existence of a quaternary continuous classical boundary optimal control vector is stated and demonstrated under suitable Assumptions. The mathematical formulation for the quaternary adjoint boundary value problem associated with each considered boundary value problem is obtained and the Frѐchet derivative for the objective function is derived. Finally, the necessary conditions for the optimality theorem of the problem are stated and demonstrated.

Learning Ultrametric Trees for Optimal Transport Regression

Optimal transport provides a metric which quantifies the dissimilarity between probability measures. For measures supported in discrete metric spaces, finding the optimal transport distance has cubic time complexity in the size of the space. However, measures supported on trees admit a closed-form optimal transport that can be computed in linear time. In this paper, we aim to find an optimal tree structure for a given discrete metric space so that the tree-Wasserstein distance approximates the optimal transport distance in the original space. One of our key ideas is to cast the problem in ultrametric spaces. This helps us optimize over the space of ultrametric trees --- a mixed-discrete and continuous optimization problem --- via projected gradient decent over the space of ultrametric matrices. During optimization, we project the parameters to the ultrametric space via a hierarchical minimum spanning tree algorithm, equivalent to the closest projection to ultrametrics under the supremum norm. Experimental results on real datasets show that our approach outperforms previous approaches (e.g. Flowtree, Quadtree) in approximating optimal transport distances. Finally, experiments on synthetic data generated on ground truth trees show that our algorithm can accurately uncover the underlying trees.

The multidepot drone general routing problem with duration and capacity constraints

AbstractThis paper studies the multidepot drone general routing problem with duration and capacity constraints (MDdGRP), an extension of the classical general routing problem with several depots. A fleet of drones with limited flight time and payload, each one located in a different depot, must jointly perform the service. In this continuous optimization problem, a set of lines of a network has to be serviced and a set of nodes has to be visited to deliver a given commodity. The aim is to find a set of drone routes performing the complete service with the shortest total duration. Aerial drones can enter a line at any point, service a portion of that line, and exit through another point, without following the lines of the network. While this flexibility allows for potentially better solutions, it also adds complexity to the problem. To deal with this, we digitize the MDdGRP instances by approximating each line with a polygonal chain containing a finite number of points that can be used to enter and exit each line. Thus, an instance of a discrete multidepot general routing problem (MDGRP) is obtained. This paper proposes an integer formulation for the MDGRP, some families of valid inequalities, and a branch‐and‐cut algorithm based on the strengthened formulation. Two matheuristic algorithms are presented, focused on sequentially selecting (and adding) some promising intermediate points to enter and exit a line. Instances involving up to six depots, 22 delivery points, and 88 original lines have been generated to test the performance of the methods proposed.

Optimization for cost-effective design of water distribution networks: a comprehensive learning approach

AbstractThe Comprehensive Learning Gravitational Search Algorithm (CLGSA) has demonstrated its effectiveness in solving continuous optimization problems. In this research, we extended the CLGSA to tackle NP-hard combinatorial problems and introduced the Discrete Comprehensive Learning Gravitational Search Algorithm (D-CLGSA). The D-CLGSA framework incorporated a refined position and velocity update scheme tailored for discrete problems. To evaluate the algorithm's efficiency, we conducted two sets of experiments. Firstly, we assessed its performance on a diverse range of 24 benchmarks encompassing unimodal, multimodal, composite, and special discrete functions. Secondly, we applied the D-CLGSA to a practical optimization problem involving water distribution network planning and management. The D-CLGSA model was coupled with the hydraulic simulation solver EPANET to identify the optimal design for the water distribution network, aiming for cost-effectiveness. We evaluated the model's performance on six distribution networks, namely Two-loop network, Hanoi network, New-York City network, GoYang network, BakRyun network, and Balerma network. The results of our study were promising, surpassing previous studies in the field. Consequently, the D-CLGSA model holds great potential as an optimizer for economically and reliably planning and managing water networks.

BinDMO: a new Binary Dwarf Mongoose Optimization algorithm on based Z-shaped, U-shaped, and taper-shaped transfer functions for CEC-2017 benchmarks

Intelligent swarm optimization algorithms have become increasingly common due to their success in solving real-world problems. Dwarf Mongoose Optimization (DMO) algorithm is a newly proposed intelligent swarm optimization algorithm in recent years. It was developed for continuous optimization problem solutions in its original paper. But real-world problems are not always problems that take continuously variable values. Real-world problems are often problems with discrete variables. Therefore, heuristic algorithms proposed for continuous optimization problems need to be updated to solve discrete optimization problems. In this study, DMO has been updated for binary optimization problems and the Binary DMO (BinDMO) algorithm has been proposed. In binary optimization, the search space consists of binary variable values. Transfer functions are often used in the conversion of continuous variable values to binary variable values. In this study, twelve different transfer functions were used (four Z-shaped, four U-shaped, and four Taper-shaped). Thus, twelve different BinDMO variations were obtained (BinDMO1, BinDMO2, …, BinDMO12). The achievements of BinDMO variations were tested on thirteen different unimodal and multimodal classical benchmark functions. The effectiveness of population sizes on the effectiveness of BinDMO was also investigated. When the results were examined, it was determined that the most successful BinDMO variation was BinDMO1 (with Z1-shaped transfer function). The most successful BinDMO variation was compared with three different binary heuristic algorithms selected from the literature (SO, PDO, and AFT) on CEC-2017 benchmark functions. According to the average results, BinDMO was the most successful binary heuristic algorithm. This has proven that BinDMO can be chosen as an alternative algorithm for binary optimization problems.

The Emergency Braking Game: a game theoretic approach for maneuvering in a dense crowd of pedestrians

We introduce an algorithm that maneuvers a vehicle through an area with randomly moving pedestrians. In non-critical situations, our strategy is to avoid pedestrians by steering, whereas dangerously moving pedestrians are avoided by braking, possibly coming to a complete stop. The distinction between non-critical and dangerous situations, as well as proof of safety, is based on a continuous optimization problem that we define. In this abstract problem, called Emergency Braking Game, one pedestrian is actively trying to collide with a continuously decelerating car. We show how to determine the outcome of the game based on the initial states of the car and the pedestrian. Using this information, our algorithm can initiate deceleration in the real scenario in time to avoid collision. The method’s safety is proven theoretically, and its efficiency is shown in simulations with randomly moving pedestrians.

Pflacco: Feature-Based Landscape Analysis of Continuous and Constrained Optimization Problems in Python.

The herein proposed Python package pflacco provides a set of numerical features to characterize single-objective continuous and constrained optimization problems. Thereby, pflacco addresses two major challenges in the area optimization. Firstly, it provides the means to develop an understanding of a given problem instance, which is crucial for designing, selecting, or configuring optimization algorithms in general. Secondly, these numerical features can be utilized in the research streams of automated algorithm selection and configuration. While the majority of these landscape features is already available in the R package flacco, our Python implementation offers these tools to an even wider audience and thereby promotes research interests and novel avenues in the area of optimization.

A novel state transition algorithm with adaptive fuzzy penalty for multi-constraint UAV path planning

Unmanned aerial vehicles (UAVs) require pre-planned flight paths that are energy-efficient, safe and smooth across their wide range of application scenarios. In this study, a novel UAV path planning method is proposed. Firstly, the UAV path planning under numerous obstacles is modeled as a continuous constrained optimization problem. The cost function is formulated as a linear combination of length, height variation, and smoothness, while the constraints include obstacle avoidance, height limitation, and the maneuverability of UAV. Subsequently, a novel constrained state transition algorithm with adaptive fuzzy penalty (AFSTA) is proposed to solve the optimization problem. In AFSTA, a novel adaptive fuzzy penalty function is designed to leverage expert knowledge to establish a reasonable mapping relationship from the fitness value and the degree of constraint violation to the penalty factor for a candidate solution. Meanwhile, the state transition algorithm (STA) is used as the search engine for both global and local search. Experimental results illustrate that the proposed method can find energy-efficient, safe, and maneuverable flight paths successfully with the superiority over other state-of-the-art metaheuristic algorithms.

COMBSS: best subset selection via continuous optimization

The problem of best subset selection in linear regression is considered with the aim to find a fixed size subset of features that best fits the response. This is particularly challenging when the total available number of features is very large compared to the number of data samples. Existing optimal methods for solving this problem tend to be slow while fast methods tend to have low accuracy. Ideally, new methods perform best subset selection faster than existing optimal methods but with comparable accuracy, or, being more accurate than methods of comparable computational speed. Here, we propose a novel continuous optimization method that identifies a subset solution path, a small set of models of varying size, that consists of candidates for the single best subset of features, that is optimal in a specific sense in linear regression. Our method turns out to be fast, making the best subset selection possible when the number of features is well in excess of thousands. Because of the outstanding overall performance, framing the best subset selection challenge as a continuous optimization problem opens new research directions for feature extraction for a large variety of regression models.

Nested Markov chain hyper-heuristic (NMHH): a hybrid hyper-heuristic framework for single-objective continuous problems.

This article introduces a new hybrid hyper-heuristic framework that deals with single-objective continuous optimization problems. This approach employs a nested Markov chain on the base level in the search for the best-performing operators and their sequences and simulated annealing on the hyperlevel, which evolves the chain and the operator parameters. The novelty of the approach consists of the upper level of the Markov chain expressing the hybridization of global and local search operators and the lower level automatically selecting the best-performing operator sequences for the problem. Numerical experiments conducted on well-known benchmark functions and the comparison with another hyper-heuristic framework and six state-of-the-art metaheuristics show the effectiveness of the proposed approach.

A Comparative Study of Optimal PV Allocation in a Distribution Network Using Evolutionary Algorithms

The growing distributed energy resource (DER) penetration into distribution networks, such as through residential and commercial photovoltaics (PV), has emerged through a transition from passive to active networks, which takes the complexity of planning and operations to the next level. Optimal PV allocation (sizing and location) is challenging because it involves mixed-integer non-linear programming with three-phase non-linear unbalanced power flow equations. Meta-heuristic algorithms have proven their effectiveness in many complex engineering problems. Thus, in this study, we propose to achieve optimal PV allocation by using several basic evolutionary algorithms (EAs), particle swarm optimization (PSO), artificial bee colony (ABC), differential evolution (DE), and their variants, all of which are applied for a study of their performance levels. Two modified unbalanced IEEE test feeders (13 and 37 bus) are developed to evaluate these performance levels, with two objectives: one is to maximize PV penetration, and the other is to minimize the voltage deviation from 1.0 p.u. To handle the computational burden of the sequential power flow and unbalanced network, we adopt an efficient iterative load flow algorithm instead of the commonly used and yet highly simplified forward–backward sweep method. A comparative study of these basic EAs shows their general success in finding a near-optimal solution, except in the case of the DE, which is known for solving continuous optimization problems efficiently. From experiments run 30 times, it is observed that PSO-related algorithms are more efficient and robust in the maximum PV penetration case, while ABC-related algorithms are more efficient and robust in the minimum voltage deviation case.

Characterization of Constrained Continuous Multiobjective Optimization Problems: A Performance Space Perspective

Characterization of Constrained Continuous Multiobjective Optimization Problems: A Performance Space Perspective