Large-scale Optimization Problems Research Articles

An improved problem transformation algorithm for large-scale multi-objective optimization

Solving Large-Scale Multi-Objective Optimization Problems (LSMOPs) is the major challenge in evolution computation. Due to the large number of decision variables involved, it is not easy for existing multi-objective evolutionary algorithms to search the entire decision variable space with limited computation resources. To address the above problem, a large-scale multi-objective evolutionary algorithm based on problem transformation, called LSMOEA/PT, is presented in this paper. LSMOEA/PT uses an improved problem transformation strategy to transform LSMOPs into small-scale multi-objective problems to accelerate convergence speed and reduce computational resources. Specifically, The transformation strategy selects reference solutions to initialize a weight population by applying the opposite individual method. After optimizing the weight population, a dual line strategy is implemented to combine the weight vectors with the decision variables from the original population, thereby generating a new population. The new population is combined with the original population for environment selection. In LSMOEA/PT, a double learning swarm optimizer is introduced to speed up the convergence of the population. A uniform distribution strategy is conducted to update transformed solutions after the problem transformation process, increasing the diversity of the population. Experimental results show that LSMOEA/PT is competitive with eleven state-of-the-art algorithms on large-scale multi-objective optimization test problems with up to 2000 decision variables.

Improving two-layer encoding of evolutionary algorithms for sparse large-scale multiobjective optimization problems

Improving two-layer encoding of evolutionary algorithms for sparse large-scale multiobjective optimization problems

Robust Radiotherapy Planning with Spatially Based Uncertainty Sets

Radiotherapy treatment planning is a challenging large-scale optimization problem plagued by uncertainty. Following the robust optimization methodology, we propose a novel, spatially based uncertainty set for robust modeling of radiotherapy planning, producing solutions that are immune to unexpected changes in biological conditions. Our proposed uncertainty set realistically captures biological radiosensitivity patterns that are observed using recent advances in imaging, while its parameters can be personalized for individual patients. We exploit the structure of this set to devise a compact reformulation of the robust model. We develop a row-generation scheme to solve real, large-scale instances of the robust model. This method is then extended to a relaxation-based scheme for enforcing challenging, yet clinically important, dose-volume cardinality constraints. The computational performance of our algorithms, as well as the quality and robustness of the computed treatment plans, are demonstrated on simulated and real imaging data. Based on accepted performance measures, such as minimal target dose and homogeneity, these examples demonstrate that the spatially robust model achieves almost the same performance as the nominal model in the nominal scenario, and otherwise, the spatial model outperforms both the nominal and the box-uncertainty models.

A Dual-Competition-Based Particle Swarm Optimizer for Large-Scale Optimization

Large-scale particle swarm optimization (PSO) has long been a hot topic due to the following reasons: Swarm diversity preservation is still challenging for current PSO variants for large-scale optimization problems, resulting in difficulties for PSO in balancing its exploration and exploitation. Furthermore, current PSO variants for large-scale optimization problems often introduce additional operators to improve their ability in diversity preservation, leading to increased algorithm complexity. To address these issues, this paper proposes a dual-competition-based particle update strategy (DCS), which selects the particles to be updated and corresponding exemplars with two rounds of random pairing competitions, which can straightforwardly benefit swarm diversity preservation. Furthermore, DCS confirms the primary and secondary exemplars based on the fitness sorting operation for exploitation and exploration, respectively, leading to a dual-competition-based swarm optimizer. Thanks to the proposed DCS, on the one hand, the proposed algorithm is able to protect more than half of the particles from being updated to benefit diversity preservation at the swarm level. On the other hand, DCS provides an efficient exploration and exploitation exemplar selection mechanism, which is beneficial for balancing exploration and exploitation at the particle update level. Additionally, this paper analyzes the stability conditions and computational complexity of the proposed algorithm. In the experimental section, based on seven state-of-the-art algorithms and a recently proposed large-scale benchmark suite, this paper verifies the competitiveness of the proposed algorithm in large-scale optimization problems.

Large-scale and cooperative graybox parallel optimization on the supercomputer Fugaku

We design, develop and analyze parallel variants of a state-of-the-art graybox optimization algorithm, namely Drils (Deterministic Recombination and Iterated Local Search), for attacking large-scale pseudo-boolean optimization problems on top of the large-scale computing facilities offered by the supercomputer Fugaku. We first adopt a Master/Worker design coupled with a fully distributed Island-based model, ending up with a number of hybrid OpenMP/MPI implementations of high-level parallel Drils versions. We show that such a design, although effective, can be substantially improved by enabling a more focused iteration-level cooperation mechanism between the core graybox components of the original serial Drils algorithm. Extensive experiments are conducted in order to provide a systematic analysis of the impact of the designed parallel algorithms on search behavior, and their ability to compute high-quality solutions using increasing number of CPU-cores. Results using up to 1024×12-cores NUMA nodes, and NK-landscapes with up to 10,000 binary variables are reported, providing evidence on the relative strength of the designed hybrid cooperative graybox parallel search.

A two-stage direction-guided evolutionary algorithm for large-scale multiobjective optimization

Large-scale multiobjective optimization problems (LSMOPs) have exponential growth in the search space as the decision variables increase, and the vast search space poses a challenge to the performance of multiobjective evolutionary algorithms (MOEAs). Many current large-scale MOEAs need to consume a large amount of computational resources to get good performance. This paper proposes a two-stage direction-guided evolutionary algorithm for large-scale multiobjective optimization (LMOEA-S2D) to balance the performance and computational resource overhead. The algorithm exploits the Pareto-optimality property of domination and the diversity-preserving property of decomposition to optimize the performance in the two stages, respectively, and designs a corresponding direction-guided mechanism to improve search efficiency. LMOEA-S2D designs global direction search and local direction search in the domination-based stage for efficient exploitation to accelerate population convergence. To promote greater population diversity, a hybrid direction search was devised to aid diversity exploration in the decomposition-based stage, and this facilitates even distribution of candidate solutions across the Pareto optimal frontier. LMOEA-S2D is compared with five state-of-the-art large-scale MOEAs on some large-scale multiobjective test suites with 100 to 5,000 decision variables. The experimental results show that LMOEA-S2D significantly outperformed all compared algorithms under limited computational resources.

Cooperative coevolutionary differential evolution with adjacent intensity matrix with linkage identification for large-scale optimization problems in noisy environments

Cooperative coevolutionary differential evolution with adjacent intensity matrix with linkage identification for large-scale optimization problems in noisy environments

A mid-range approximation method assisted by trust region strategy for aerodynamic shape optimization

This paper presents an efficient solution for high-fidelity large-scale aerodynamic shape optimization problems based on several developments in the mid-range approximation method within a trust region optimization framework. The mid-range approximation method is an iterative optimization technique that utilizes mid-range approximations to replace the physical experiments or simulations during the optimization based on the selected trust region strategy. It could transform the original optimization problem into a sequence of approximate sub-optimization problems. In this work, an improved trust region strategy is proposed to contain more optimization states with a flexible and controllable performance to suit different types of problems. A metamodel assembly technique and its gradient-enhanced version are developed to further relax the requirements of computational costs in the mid-range approximation method. Its performance is discussed through a detailed comparison of metamodel performance using a mathematical benchmark case named Vanderplaats scalable beam. The wing only of the Common Research Model is offered to the proposed method for aerodynamic shape optimization. The optimization has 1 design objective, 232 design variables, and 135 design constraints. With all constraints satisfied, the optimized configuration has a 4.85 % improvement in wing drag performance. The shock region is greatly reduced and the wing pressure distribution is smooth and nearly parallel. These results show that the proposed method could achieve the design goal successfully within a reasonable computational cost for large-scale problems.

Distributed gradient-free and projection-free algorithm for stochastic constrained optimization

Distributed stochastic zeroth-order optimization (DSZO), in which the objective function is allocated over multiple agents and the derivative of cost functions is unavailable, arises frequently in large-scale machine learning and reinforcement learning. This paper introduces a distributed stochastic algorithm for DSZO in a projection-free and gradient-free manner via the Frank-Wolfe framework and the stochastic zeroth-order oracle (SZO). Such a scheme is particularly useful in large-scale constrained optimization problems where calculating gradients or projection operators is impractical, costly, or when the objective function is not differentiable everywhere. Specifically, the proposed algorithm, enhanced by recursive momentum and gradient tracking techniques, guarantees convergence with just a single batch per iteration. This significant improvement over existing algorithms substantially lowers the computational complexity. Under mild conditions, we prove that the complexity bounds on SZO of the proposed algorithm are O(n/ϵ2)\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$\\mathcal{O}(n/\\epsilon ^{2})$\\end{document} and O(n(21ϵ))\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$\\mathcal{O}(n(2^{\\frac{1}{\\epsilon}}))$\\end{document} for convex and nonconvex cases, respectively. The efficacy of the algorithm is verified on black-box binary classification problems against several competing alternatives.

An inexact variable metric variant of incremental aggregated Forward-Backward method for the large-scale composite optimization problem

ABSTRACT This paper focuses on the large-scale composite optimization problem, which is the sum of a finite number of smooth (possibly nonconvex) functions and a convex (possibly nonsmooth) function. A common method to solve this problem is proximal incremental aggregated gradient (PIAG) method (N. D. Vanli, M. Gurbuzbalaban and A. Ozdaglar, SIAM Journal on Optimization, 2018, 28(2): 1282–1300), which exploits the gradient information from previous iterations to approximate the gradient at the current point. However, as many first-order algorithms, it may suffer from slow convergence. To address this issue, this paper proposes an inexact variable metric variant of incremental aggregated Forward-Backward (iVMFB-IAG) method. This methods incorporates a variable metric strategy to accelerate the PIAG method and introduces an inexact criterion to enhance its practicality. Under the Kurdyka-Łojasiewicz (KL) property and the uniformly bounded positive definiteness of the scaling matrix, the sequence generated by the iVMFB-IAG method converges globally to the critical point of the problem. Additionally, the convergence rate is established under the KL framework. Moreover, under certain restrictive conditions, the method exhibits local superlinear convergence. Through the numerical experiments on image reconstruction applications, we demonstrate the effectiveness of the iVMFB-IAG method by restoring two distinct images.

Feature selection by Universum embedding

Feature selection in classification is an important task in machine learning. Inspired by the success of Universum support vector machine proposed by Weston et al. on improving the classification ability of classical support vector machine, this paper considers a special type of Universum and further lets it play its role in both useful feature identification and separating hyperplane construction, aiming to improve both the feature selection ability and classification performance of Universum support vector machine. By introducing this special Universum, a redundant feature can be identified by observing whether some Universum sample is useful. In fact, we prove that by observing the dual solution of the optimization problem, useful features can be selected from a set satisfying some properties. Due to the introduction of these extra Universum samples, it needs to cope with a large-scale optimization problem. To improve the training efficiency, we modify the sequential minimal optimization algorithm and further combine it with the coordinate descent technique to solve the proposed model. Experimental results on artificial datasets, benchmark datasets, and text classification datasets demonstrate that the proposed method improves the classification performance of support vector machine and Universum support vector machine, and also has good feature selection ability.

Optimization of CO2 capture plants with surrogate model uncertainties

CO2 capture plants can help reduce the cost of deploying capture systems across the globe. However, the CO2 variability and model uncertainty represent operational challenges to capture CO2 from different sources. This work proposes a framework for analyzing the optimal plant design considering different flue gas sources. We show a methodology to generate large data sets from optimization runs using rigorous models in Aspen Plus®. The efficiency of the approach allows its application to large-scale optimization problems, with an average CPU time per run of 176 s.We additionally build surrogate models (SMs) for the capital and operating costs of the capture plants, employing an iterative procedure to generate SMs using ALAMO. We systematically reject SMs with high uncertainty in the estimated parameters. This approach results in SMs with favorable bias-variance tradeoffs, enabling their effective application to optimization problems under uncertainty, as demonstrated with a pooling problem of CO2 streams.

MCHIAO: a modified coronavirus herd immunity-Aquila optimization algorithm based on chaotic behavior for solving engineering problems

AbstractThis paper proposes a hybrid Modified Coronavirus Herd Immunity Aquila Optimization Algorithm (MCHIAO) that compiles the Enhanced Coronavirus Herd Immunity Optimizer (ECHIO) algorithm and Aquila Optimizer (AO). As one of the competitive human-based optimization algorithms, the Coronavirus Herd Immunity Optimizer (CHIO) exceeds some other biological-inspired algorithms. Compared to other optimization algorithms, CHIO showed good results. However, CHIO gets confined to local optima, and the accuracy of large-scale global optimization problems is decreased. On the other hand, although AO has significant local exploitation capabilities, its global exploration capabilities are insufficient. Subsequently, a novel metaheuristic optimizer, Modified Coronavirus Herd Immunity Aquila Optimizer (MCHIAO), is presented to overcome these restrictions and adapt it to solve feature selection challenges. In this paper, MCHIAO is proposed with three main enhancements to overcome these issues and reach higher optimal results which are cases categorizing, enhancing the new genes’ value equation using the chaotic system as inspired by the chaotic behavior of the coronavirus and generating a new formula to switch between expanded and narrowed exploitation. MCHIAO demonstrates it’s worth contra ten well-known state-of-the-art optimization algorithms (GOA, MFO, MPA, GWO, HHO, SSA, WOA, IAO, NOA, NGO) in addition to AO and CHIO. Friedman average rank and Wilcoxon statistical analysis (p-value) are conducted on all state-of-the-art algorithms testing 23 benchmark functions. Wilcoxon test and Friedman are conducted as well on the 29 CEC2017 functions. Moreover, some statistical tests are conducted on the 10 CEC2019 benchmark functions. Six real-world problems are used to validate the proposed MCHIAO against the same twelve state-of-the-art algorithms. On classical functions, including 24 unimodal and 44 multimodal functions, respectively, the exploitative and explorative behavior of the hybrid algorithm MCHIAO is evaluated. The statistical significance of the proposed technique for all functions is demonstrated by the p-values calculated using the Wilcoxon rank-sum test, as these p-values are found to be less than 0.05.

Efficient knowledge model for whale optimization algorithm to solve large-scale problems

In the process of digital transformation and development in various industries, there are more and more large-scale optimization problems. Currently, swarm intelligence optimization algorithms are the best method to solve such problems. However, previous experimental research has found that there is still room for improvement in the performance of using existing swarm intelligence optimization algorithms to solve such problems. To obtain the high-precision optimal value of whale optimization algorithm (WOA) for solving large-scale optimization problems, the optimization problem knowledge model is studied to guide the iterative process of WOA algorithm, and a novel whale optimization algorithm based on knowledge model guidance (KMGWOA) is proposed. First, a population update strategy based on multiple elite individuals is proposed to reduce the impact of the local optimal values, and the knowledge model to guide population update is constructed by combining the proposed population update strategy with the population update strategy based on global optimal individual. Second, a collaborative reverse learning knowledge model with multiple elite and poor individuals in the solution space is proposed to prevent long-term non-ideal region search. The above two knowledge models guide the iterative process of WOA algorithm in solving large-scale optimization problems. The performance of the KMGWOA algorithm guided by the proposed knowledge models is tested through the well-known classical test functions. The results demonstrate that the proposed KMGWOA algorithm not only has good search ability for the theoretical optimal value, but also achieves higher accuracy in obtaining the optimal value when it is difficult to obtain the theoretical optimal value. Moreover, KMGWOA algorithm has fast convergence speed and high effective iteration percentage.

A stabilised Benders decomposition with adaptive oracles for large-scale stochastic programming with short-term and long-term uncertainty

Benders decomposition with adaptive oracles was proposed to solve large-scale optimisation problems with a column-bounded block-diagonal structure, where subproblems differ only in the right-hand side and cost coefficients. Adaptive Benders reduces computational effort significantly by iteratively building inexact cutting planes and valid upper and lower bounds. However, Adaptive Benders and standard Benders may suffer severe oscillation when solving degenerate models. Therefore, we propose stabilising Adaptive Benders with the level method and adaptively selecting which subproblems to solve each iteration for more accurate information. In addition, we propose a dynamic level method to improve the robustness of stabilised Adaptive Benders by adjusting the level set each iteration. We compare stabilised Adaptive Benders with the unstabilised versions of Adaptive Benders with one subproblem solved per iteration and standard Benders on a multi-region long-term power system investment planning problem with short-term and long-term uncertainty. The problem is formulated as multi-horizon stochastic programming. Four algorithms were implemented to solve linear programming with up to 1 billion variables and 4.5 billion constraints. The computational results show that: (a) for a 1.00% convergence tolerance, the proposed stabilised method is up to 113.7 times faster than standard Benders and 2.1 times faster than unstabilised Adaptive Benders; (b) for a 0.10% convergence tolerance, the proposed stabilised method is up to 45.5 times faster than standard Benders and unstabilised Adaptive Benders cannot solve the largest instance to convergence tolerance due to severe oscillation and (c) dynamic level method makes stabilisation more robust.

Boosting scalability for large-scale multiobjective optimization via transfer weights

Large-scale multiobjective optimization problems (LSMOPs), which optimize multiple conflicting objectives with hundreds or even thousands of decision variables, demand increasing computational resources to assure satisfactory performance as the decision variables increase. Multiobjective evolutionary algorithms are naturally scalable, but as the dimension increases, the conflict between analyzing enough solutions and the limited number of function evaluations has hindered further improvements in scalability. In this paper, we first define the scalability of multiobjective evolutionary algorithms and design an indicator to quantitatively measure the scalability. Second, to boost scalability when solving LSMOPs, we propose a scalable multiobjective optimization algorithm by transferring weights between solutions to reduce dependency on ever-increasing computational resources as the problem dimension increases. The proposed framework entails constructing a latent decision space to determine evolutionary weights for chosen representative solutions. These computed weights are then transferred to the remaining solutions, enabling evolutionary optimization to proceed without requiring additional evaluations, even as the dimensionality increases. By utilizing the knowledge gained from the source solutions, each solution is customized with an evolutionary weight scheme that not only preserves computational resources but also enhances optimization performance, thereby boosting scalability. We have conducted experiments on LSMOPs to verify the effectiveness and scalability of the proposed algorithm. The proposed method outperforms selected state-of-the-art algorithms and gains scalability boosts in situations where the dimensions increase while the function evaluation remains constant.

Multi-task collaborative method based on manifold optimization for automated test case generation based on path coverage

Automated test case generation based on path coverage is not only a large-scale black-box optimization problem, but also a key scientific problem in software automatic testing technology. Evolutionary algorithms and other search-based algorithms are representative methods for this problem. However, existing research mainly focuses on the multi-function case, where generating test cases for multiple functions is difficult due to the combinatorial explosion of the path number. In this paper, we propose a multi-task collaborative method based on manifold optimization by considering the topological manifold relationship between the test case space (decision space) and the program path space (target space). This method achieves the goal of collaborative optimization for different function coverage tasks by coordinating the allocation of computing resources and knowledge transfer mechanisms among them. To verify the effectiveness of the proposed method, we compare it with general solution methods for single-function automated test case generation based on path coverage, such as the manifold-inspired search-based algorithm. The experimental results show that the proposed method outperforms the compared single-function optimization algorithms especially on programs with strong coding similarities. This study verifies the feasibility of collaborative optimization methods for solving large-scale black-box optimization problems, such as the automated test case generation based on path coverage, and expands their application scenarios.

Stabilized BB projection algorithm for large-scale convex constrained nonlinear monotone equations to signal and image processing problems

The Barzilai–Borwein (BB) method is a popular and efficient gradient method for solving large-scale unconstrained optimization problems. In general, it converges much faster than the steepest descent (Cauchy) method. However, it may not converge, even when the objective function is strongly convex. To overcome this, by virtue of bounding the distance between sequential iterates, [Burdakov et al. (2019)] proposed a stabilized BB method. In this paper, by combining the stabilized BB method with a projection approach, we develop a stabilized BB projection (SBBP) method to solve nonlinear monotone equations with convex constraints. SBBP does not involve computing matrices proving the usefulness for solving large-scale problems. Its global convergence and Q-linear convergence rate are also established. We report numerical experiments on recovering sparse signals and restoring blurred images arising from compressive sensing, demonstrating the usefulness of SBBP.

Taking the human out of decomposition-based optimization via artificial intelligence, Part II: Learning to initialize

The repeated solution of large-scale optimization problems arises frequently in process systems engineering tasks. Decomposition-based solution methods have been widely used to reduce the corresponding computational time, yet their implementation has multiple steps that are difficult to configure. We propose a machine learning approach to learn the optimal initialization of such algorithms which minimizes the computational time. Active and supervised learning is used to learn a surrogate model that predicts the computational performance for a given initialization. We apply this approach to the initialization of Generalized Benders Decomposition for the solution of mixed-integer model predictive control problems. The surrogate models are used to find the optimal initialization, which corresponds to the number of initial cuts that should be added in the master problem. The results show that the proposed approach can lead to a significant reduction in solution time, and active learning can reduce the data required for learning.

Learning-assisted optimization for transmission switching

AbstractThe design of new strategies that exploit methods from machine learning to facilitate the resolution of challenging and large-scale mathematical optimization problems has recently become an avenue of prolific and promising research. In this paper, we propose a novel learning procedure to assist in the solution of a well-known computationally difficult optimization problem in power systems: The Direct Current Optimal Transmission Switching (DC-OTS) problem. The DC-OTS problem consists in finding the configuration of the power network that results in the cheapest dispatch of the power generating units. With the increasing variability in the operating conditions of power grids, the DC-OTS problem has lately sparked renewed interest, because operational strategies that include topological network changes have proved to be effective and efficient in helping maintain the balance between generation and demand. The DC-OTS problem includes a set of binaries that determine the on/off status of the switchable transmission lines. Therefore, it takes the form of a mixed-integer program, which is NP-hard in general. In this paper, we propose an approach to tackle the DC-OTS problem that leverages known solutions to past instances of the problem to speed up the mixed-integer optimization of a new unseen model. Although our approach does not offer optimality guarantees, a series of numerical experiments run on a real-life power system dataset show that it features a very high success rate in identifying the optimal grid topology (especially when compared to alternative competing heuristics), while rendering remarkable speed-up factors.