Large-scale Optimization Problems Research Articles

A fluid–particle decomposition approach to matching market design for crowdsourced delivery systems

This paper considers a crowdsourced delivery (CSD) system that effectively utilizes the existing trips to fulfill parcel delivery as a matching problem between CSD drivers and delivery tasks. This matching problem has two major challenges. First, it is a large-scale combinatorial optimization problem that is hard to solve in a reasonable computational time. Second, the evaluation of the objective function for socially optimal matching contains the utility of drivers for performing the tasks, which is generally unobservable private information. To address these challenges, this paper proposes a hierarchical distribution mechanism of CSD tasks that decomposes the matching problem into the task partition (master problem) and individual task-driver matching within smaller groups of drivers (sub-problems). We incorporate an auction mechanism with truth-telling and efficiency into the sub-problems so that the drivers’ perceived utilities are revealed through their bids. Furthermore, we formulate the master problem as a fluid model based on continuously approximated decision variables. By exploiting the random utility framework, we analytically represent the objective function of the problem using continuous variables, without explicitly knowing the drivers’ utilities. The numerical experiment shows that the proposed approach solved large-scale matching problems at least 100 times faster than a naive LP solver and approximated the original objective value with errors of less than 1%. The code of this work is available at https://github.com/yuki-oyama/fluid-particle-csd.

A new diagonalization based method for parallel-in-time solution of linear-quadratic optimal control problems

A new diagonalization technique for the parallel-in-time solution of linear-quadratic optimal control problems with time-invariant system matrices is introduced. The target problems are often derived from a semi-discretization of a Partial Differential Equation (PDE)-constrained optimization problem. The solution of large-scale time dependent optimal control problems is computationally challenging as the states, controls, and adjoints are coupled to each other throughout the whole time domain. This computational difficulty motivates the use of parallel-in-time methods. For time-periodic problems our diagonalization efficiently transforms the discretized optimality system into K (=number of time steps) decoupled complex valued 2n y by 2n y systems, where n y is the dimension of the state space. These systems resemble optimality systems corresponding to a steady-state version of the optimal control problem and they can be solved in parallel across the time steps, but are complex valued. For optimal control problems with initial value state equations a direct solution via diagonalization is not possible, but an efficient preconditioner can be constructed from the corresponding time periodic optimal control problem. The preconditioner can be efficiently applied parallel-in-time using the diagonalization technique. The observed number of preconditioned GMRES iterations is small and insensitive to the size of the problem discretization.

Many-Objective Whale Optimization Algorithm for Engineering Design and Large-Scale Many-Objective Optimization Problems

In this paper, a novel Many-Objective Whale Optimization Algorithm (MaOWOA) is proposed to overcome the challenges of large-scale many-objective optimization problems (LSMOPs) encountered in diverse fields such as engineering. Existing algorithms suffer from curse of dimensionality i.e., they are unable to balance convergence with diversity in extensive decision-making scenarios. MaOWOA introduces strategies to accelerate convergence, balance convergence and diversity in solutions and enhance diversity in high-dimensional spaces. The prime contributions of this paper are—development of MaOWOA, incorporation an Information Feedback Mechanism (IFM) for rapid convergence, a Reference Point-based Selection (RPS) to balance convergence and diversity and a Niche Preservation Strategy (NPS) to improve diversity and prevent overcrowding. A comprehensive evaluation demonstrates MaOWOA superior performance over existing algorithms (MaOPSO, MOEA/DD, MaOABC, NSGA-III) across LSMOP1-LSMOP9 benchmarks and RWMaOP1-RWMaOP5 problems. Results validated using Wilcoxon rank sum tests, highlight MaOWOA excellence in key metrics such as generational distance, spread, spacing, runtime, inverse generational distance and hypervolume, outperforming in 71.8% of tested scenarios. Thus, MaOWOA represents a significant advancement in many-objective optimization, offering new avenues for addressing LSMOPs and RWMaOPs’ inherent challenges. This paper details MaOWOA development, theoretical basis and effectiveness, marking a promising direction for future research in optimization strategies amidst growing problem complexity.

Research on optimization algorithms for localization and capacity determination of chargers considering the spatiotemporal distribution of electric vehicles

The optimized layout of electric vehicle (EV) chargers is not only crucial for users' convenience but also a key element in urban sustainable development, energy transition, and the promotion of new energy vehicles. In order to provide a basis for the problem of localization and capacity determination of chargers and compare the merits of several mainstream algorithms, this paper first establishes an optimization model with the objective of minimizing the total investment cost of all the chargers and the constraint of meeting the charging demands of all electric vehicles. Optimizations were performed using genetic algorithm (GA), surrogate optimization algorithm (SOA), and mixed integer linear programming (MILP) algorithm, respectively. In the case of using MILP, the original nonlinear optimization problem was transformed into a linear problem. In the planning of city-level EV chargers, MILP took 14182.57 s to calculate the minimum cost of 34.62 million yuan. After retaining only 10% of the original data amount, SOA took 87651.34 s to calculate the minimum cost of 3.01 million yuan. The results indicate that GA is prone to falling into local optima and is not suitable for large-scale optimization problems. SOA, on the other hand, requires significant memory consumption, so the issue of memory usage needs to be carefully considered when using it directly. Although MILP is only applicable to linear programming problems, it has the advantages of lower memory usage and higher reliability if the problem can be transformed into a linear one.

Distributed sparsity constrained optimization over the Stiefel manifold

Distributed optimization aims to effectively complete specified tasks through cooperation among multi-agent systems, which has achieved great success in large-scale optimization problems. However, it remains a challenging task to develop an effective distributed algorithm with theoretical guarantees, especially when dealing with nonconvex constraints. More importantly, high-dimensional data often exhibits inherent structures such as sparsity, which if exploited accurately, can significantly enhance the capture of its intrinsic characteristics. In this paper, we introduce a novel distributed sparsity constrained optimization framework over the Stiefel manifold, abbreviated as DREAM. DREAM innovatively integrates the ℓ2,0-norm constraint and Stiefel manifold constraint within a distributed optimization setting, which has not been investigated in existing literature. Unlike the existing distributed methods, the proposed DREAM not only can extract the similarity information among samples, but also more flexibly determine the number of features to be extracted. Then, we develop an efficient Newton augmented Lagrangian-based algorithm. In theory, we delve into the relationship between the minimizer, the Karush-Kuhn–Tucker point, and the stationary point, and rigorously demonstrate that the sequence generated by our algorithm converges to a stationary point. Extensive numerical experiments verify its superiority over state-of-the-art distributed methods.

Multi-period optimal capacity expansion planning scheme of regional integrated energy systems considering multi-time scale uncertainty and generation low-carbon retrofit

With increasingly challenging climate crisis, carbon neutrality has become a necessary path for the whole world. However, there still lacks a comprehensive investigation on the optimal capacity expansion planning for a low-carbon transformation while considering multiple uncertainties. In this paper, a multi-period optimal capacity expansion planning scheme for regional integrated energy systems considering multi-time scale uncertainties and generation low-carbon retrofit is proposed. First, to better describe the short-time scale uncertainty caused by renewable energy and load fluctuations, a multi-dimensional tempo-spatial correlated scenario generation method is proposed to more accurately capture stochastic features with the least amounts of representative scenarios. Second, a multi-period optimal planning model considering generation low-carbon retrofit and short-/long-time scale uncertainties is developed. The proposed model combines information gap decision theory and chance constraints to stress both uncertainties simultaneously, which is further linearized to facilitate the computation. Third, an improved bilinear Benders decomposition (IBBD) method is utilized to efficiently solve the proposed large-scale optimal planning problem. Finally, numerical experiments of a real world test system of WF city show: 1) the maximum errors of PDF and CDF between generated scenarios and historical data are 0.108 and 0.016, validating the effectiveness of the proposed scenario generation method; 2) the low-carbon retrofit reduces system carbon emissions by 14.12 × 107 kg, while the total system cost is $956,411,387 with additional energy coupling units and renewables installed, indicating the reliability of the multi-period planning model; 3) the IBBD algorithm demonstrates applicability and efficiency in solving the proposed multi-period optimal expansion planning problem, reducing computation time by over 80.39 % compared to commercial solvers, such as GUROBI.

Robust radiotherapy planning with spatially-based uncertainty sets

Radiotherapy treatment planning is a challenging large-scale optimization problem plagued by uncertainty. Following the robust optimization methodology, we propose a novel, spatially based uncertainty set for robust modeling of radiotherapy planning, producing solutions that are immune to unexpected changes in biological conditions. Our proposed uncertainty set realistically captures biological radiosensitivity patterns that are observed using recent advances in imaging, while its parameters can be personalized for individual patients. We exploit the structure of this set to devise a compact reformulation of the robust model. We develop a row-generation scheme to solve real, large-scale instances of the robust model. This method is then extended to a relaxation-based scheme for enforcing challenging, yet clinically important, dose–volume cardinality constraints. The computational performance of our algorithms, as well as the quality and robustness of the computed treatment plans, are demonstrated on simulated and real imaging data. Based on accepted performance measures, such as minimal target dose and homogeneity, these examples demonstrate that the spatially robust model achieves almost the same performance as the nominal model in the nominal scenario, and otherwise, the spatial model outperforms both the nominal and the box-uncertainty models.

An improved problem transformation algorithm for large-scale multi-objective optimization

Solving Large-Scale Multi-Objective Optimization Problems (LSMOPs) is the major challenge in evolution computation. Due to the large number of decision variables involved, it is not easy for existing multi-objective evolutionary algorithms to search the entire decision variable space with limited computation resources. To address the above problem, a large-scale multi-objective evolutionary algorithm based on problem transformation, called LSMOEA/PT, is presented in this paper. LSMOEA/PT uses an improved problem transformation strategy to transform LSMOPs into small-scale multi-objective problems to accelerate convergence speed and reduce computational resources. Specifically, The transformation strategy selects reference solutions to initialize a weight population by applying the opposite individual method. After optimizing the weight population, a dual line strategy is implemented to combine the weight vectors with the decision variables from the original population, thereby generating a new population. The new population is combined with the original population for environment selection. In LSMOEA/PT, a double learning swarm optimizer is introduced to speed up the convergence of the population. A uniform distribution strategy is conducted to update transformed solutions after the problem transformation process, increasing the diversity of the population. Experimental results show that LSMOEA/PT is competitive with eleven state-of-the-art algorithms on large-scale multi-objective optimization test problems with up to 2000 decision variables.

Improving two-layer encoding of evolutionary algorithms for sparse large-scale multiobjective optimization problems

Improving two-layer encoding of evolutionary algorithms for sparse large-scale multiobjective optimization problems

A Dual-Competition-Based Particle Swarm Optimizer for Large-Scale Optimization

Large-scale particle swarm optimization (PSO) has long been a hot topic due to the following reasons: Swarm diversity preservation is still challenging for current PSO variants for large-scale optimization problems, resulting in difficulties for PSO in balancing its exploration and exploitation. Furthermore, current PSO variants for large-scale optimization problems often introduce additional operators to improve their ability in diversity preservation, leading to increased algorithm complexity. To address these issues, this paper proposes a dual-competition-based particle update strategy (DCS), which selects the particles to be updated and corresponding exemplars with two rounds of random pairing competitions, which can straightforwardly benefit swarm diversity preservation. Furthermore, DCS confirms the primary and secondary exemplars based on the fitness sorting operation for exploitation and exploration, respectively, leading to a dual-competition-based swarm optimizer. Thanks to the proposed DCS, on the one hand, the proposed algorithm is able to protect more than half of the particles from being updated to benefit diversity preservation at the swarm level. On the other hand, DCS provides an efficient exploration and exploitation exemplar selection mechanism, which is beneficial for balancing exploration and exploitation at the particle update level. Additionally, this paper analyzes the stability conditions and computational complexity of the proposed algorithm. In the experimental section, based on seven state-of-the-art algorithms and a recently proposed large-scale benchmark suite, this paper verifies the competitiveness of the proposed algorithm in large-scale optimization problems.

A rule-based online energy management strategy for long-endurance stratospheric airships

Stratospheric airships are typically equipped with photovoltaic energy systems to ensure consistent and controlled long-duration flights. To maximize the utilization of energy generated by the airship's solar array for mission requirements while maintaining energy balance in a computation-efficient manner, we analyzed the characteristics of global optimal energy profiles obtained through the pseudospectral method and distilled them into a rule-based energy management strategy. The rule-based strategy simplifies solving large-scale nonlinear optimization problems posed by pseudospectral method into solving problems for optimal rule logic thresholds, which greatly enhances the speed of strategy generation. Leveraging rapid strategy generation, we developed an online energy management scheme that can achieve closed-loop control by repeatedly updating the rule logic thresholds based on the feedback states. Simulation results demonstrate that the proposed energy management strategy is more effective in utilizing energy to meet mission requirements compared to the two-phase energy management strategy from a previous study. Notably, the online energy management scheme can adapt to uncertainties such as modeling errors, rather than merely providing an open-loop plan.

Large-scale and cooperative graybox parallel optimization on the supercomputer Fugaku

We design, develop and analyze parallel variants of a state-of-the-art graybox optimization algorithm, namely Drils (Deterministic Recombination and Iterated Local Search), for attacking large-scale pseudo-boolean optimization problems on top of the large-scale computing facilities offered by the supercomputer Fugaku. We first adopt a Master/Worker design coupled with a fully distributed Island-based model, ending up with a number of hybrid OpenMP/MPI implementations of high-level parallel Drils versions. We show that such a design, although effective, can be substantially improved by enabling a more focused iteration-level cooperation mechanism between the core graybox components of the original serial Drils algorithm. Extensive experiments are conducted in order to provide a systematic analysis of the impact of the designed parallel algorithms on search behavior, and their ability to compute high-quality solutions using increasing number of CPU-cores. Results using up to 1024×12-cores NUMA nodes, and NK-landscapes with up to 10,000 binary variables are reported, providing evidence on the relative strength of the designed hybrid cooperative graybox parallel search.

A two-stage direction-guided evolutionary algorithm for large-scale multiobjective optimization

Large-scale multiobjective optimization problems (LSMOPs) have exponential growth in the search space as the decision variables increase, and the vast search space poses a challenge to the performance of multiobjective evolutionary algorithms (MOEAs). Many current large-scale MOEAs need to consume a large amount of computational resources to get good performance. This paper proposes a two-stage direction-guided evolutionary algorithm for large-scale multiobjective optimization (LMOEA-S2D) to balance the performance and computational resource overhead. The algorithm exploits the Pareto-optimality property of domination and the diversity-preserving property of decomposition to optimize the performance in the two stages, respectively, and designs a corresponding direction-guided mechanism to improve search efficiency. LMOEA-S2D designs global direction search and local direction search in the domination-based stage for efficient exploitation to accelerate population convergence. To promote greater population diversity, a hybrid direction search was devised to aid diversity exploration in the decomposition-based stage, and this facilitates even distribution of candidate solutions across the Pareto optimal frontier. LMOEA-S2D is compared with five state-of-the-art large-scale MOEAs on some large-scale multiobjective test suites with 100 to 5,000 decision variables. The experimental results show that LMOEA-S2D significantly outperformed all compared algorithms under limited computational resources.

Cooperative coevolutionary differential evolution with adjacent intensity matrix with linkage identification for large-scale optimization problems in noisy environments

Cooperative coevolutionary differential evolution with adjacent intensity matrix with linkage identification for large-scale optimization problems in noisy environments

A mid-range approximation method assisted by trust region strategy for aerodynamic shape optimization

This paper presents an efficient solution for high-fidelity large-scale aerodynamic shape optimization problems based on several developments in the mid-range approximation method within a trust region optimization framework. The mid-range approximation method is an iterative optimization technique that utilizes mid-range approximations to replace the physical experiments or simulations during the optimization based on the selected trust region strategy. It could transform the original optimization problem into a sequence of approximate sub-optimization problems. In this work, an improved trust region strategy is proposed to contain more optimization states with a flexible and controllable performance to suit different types of problems. A metamodel assembly technique and its gradient-enhanced version are developed to further relax the requirements of computational costs in the mid-range approximation method. Its performance is discussed through a detailed comparison of metamodel performance using a mathematical benchmark case named Vanderplaats scalable beam. The wing only of the Common Research Model is offered to the proposed method for aerodynamic shape optimization. The optimization has 1 design objective, 232 design variables, and 135 design constraints. With all constraints satisfied, the optimized configuration has a 4.85 % improvement in wing drag performance. The shock region is greatly reduced and the wing pressure distribution is smooth and nearly parallel. These results show that the proposed method could achieve the design goal successfully within a reasonable computational cost for large-scale problems.

Distributed gradient-free and projection-free algorithm for stochastic constrained optimization

Distributed stochastic zeroth-order optimization (DSZO), in which the objective function is allocated over multiple agents and the derivative of cost functions is unavailable, arises frequently in large-scale machine learning and reinforcement learning. This paper introduces a distributed stochastic algorithm for DSZO in a projection-free and gradient-free manner via the Frank-Wolfe framework and the stochastic zeroth-order oracle (SZO). Such a scheme is particularly useful in large-scale constrained optimization problems where calculating gradients or projection operators is impractical, costly, or when the objective function is not differentiable everywhere. Specifically, the proposed algorithm, enhanced by recursive momentum and gradient tracking techniques, guarantees convergence with just a single batch per iteration. This significant improvement over existing algorithms substantially lowers the computational complexity. Under mild conditions, we prove that the complexity bounds on SZO of the proposed algorithm are O(n/ϵ2)\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$\\mathcal{O}(n/\\epsilon ^{2})$\\end{document} and O(n(21ϵ))\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$\\mathcal{O}(n(2^{\\frac{1}{\\epsilon}}))$\\end{document} for convex and nonconvex cases, respectively. The efficacy of the algorithm is verified on black-box binary classification problems against several competing alternatives.

An inexact variable metric variant of incremental aggregated Forward-Backward method for the large-scale composite optimization problem

ABSTRACT This paper focuses on the large-scale composite optimization problem, which is the sum of a finite number of smooth (possibly nonconvex) functions and a convex (possibly nonsmooth) function. A common method to solve this problem is proximal incremental aggregated gradient (PIAG) method (N. D. Vanli, M. Gurbuzbalaban and A. Ozdaglar, SIAM Journal on Optimization, 2018, 28(2): 1282–1300), which exploits the gradient information from previous iterations to approximate the gradient at the current point. However, as many first-order algorithms, it may suffer from slow convergence. To address this issue, this paper proposes an inexact variable metric variant of incremental aggregated Forward-Backward (iVMFB-IAG) method. This methods incorporates a variable metric strategy to accelerate the PIAG method and introduces an inexact criterion to enhance its practicality. Under the Kurdyka-Łojasiewicz (KL) property and the uniformly bounded positive definiteness of the scaling matrix, the sequence generated by the iVMFB-IAG method converges globally to the critical point of the problem. Additionally, the convergence rate is established under the KL framework. Moreover, under certain restrictive conditions, the method exhibits local superlinear convergence. Through the numerical experiments on image reconstruction applications, we demonstrate the effectiveness of the iVMFB-IAG method by restoring two distinct images.

Feature selection by Universum embedding

Feature selection in classification is an important task in machine learning. Inspired by the success of Universum support vector machine proposed by Weston et al. on improving the classification ability of classical support vector machine, this paper considers a special type of Universum and further lets it play its role in both useful feature identification and separating hyperplane construction, aiming to improve both the feature selection ability and classification performance of Universum support vector machine. By introducing this special Universum, a redundant feature can be identified by observing whether some Universum sample is useful. In fact, we prove that by observing the dual solution of the optimization problem, useful features can be selected from a set satisfying some properties. Due to the introduction of these extra Universum samples, it needs to cope with a large-scale optimization problem. To improve the training efficiency, we modify the sequential minimal optimization algorithm and further combine it with the coordinate descent technique to solve the proposed model. Experimental results on artificial datasets, benchmark datasets, and text classification datasets demonstrate that the proposed method improves the classification performance of support vector machine and Universum support vector machine, and also has good feature selection ability.

Optimization of CO2 capture plants with surrogate model uncertainties

CO2 capture plants can help reduce the cost of deploying capture systems across the globe. However, the CO2 variability and model uncertainty represent operational challenges to capture CO2 from different sources. This work proposes a framework for analyzing the optimal plant design considering different flue gas sources. We show a methodology to generate large data sets from optimization runs using rigorous models in Aspen Plus®. The efficiency of the approach allows its application to large-scale optimization problems, with an average CPU time per run of 176 s.We additionally build surrogate models (SMs) for the capital and operating costs of the capture plants, employing an iterative procedure to generate SMs using ALAMO. We systematically reject SMs with high uncertainty in the estimated parameters. This approach results in SMs with favorable bias-variance tradeoffs, enabling their effective application to optimization problems under uncertainty, as demonstrated with a pooling problem of CO2 streams.

MCHIAO: a modified coronavirus herd immunity-Aquila optimization algorithm based on chaotic behavior for solving engineering problems

AbstractThis paper proposes a hybrid Modified Coronavirus Herd Immunity Aquila Optimization Algorithm (MCHIAO) that compiles the Enhanced Coronavirus Herd Immunity Optimizer (ECHIO) algorithm and Aquila Optimizer (AO). As one of the competitive human-based optimization algorithms, the Coronavirus Herd Immunity Optimizer (CHIO) exceeds some other biological-inspired algorithms. Compared to other optimization algorithms, CHIO showed good results. However, CHIO gets confined to local optima, and the accuracy of large-scale global optimization problems is decreased. On the other hand, although AO has significant local exploitation capabilities, its global exploration capabilities are insufficient. Subsequently, a novel metaheuristic optimizer, Modified Coronavirus Herd Immunity Aquila Optimizer (MCHIAO), is presented to overcome these restrictions and adapt it to solve feature selection challenges. In this paper, MCHIAO is proposed with three main enhancements to overcome these issues and reach higher optimal results which are cases categorizing, enhancing the new genes’ value equation using the chaotic system as inspired by the chaotic behavior of the coronavirus and generating a new formula to switch between expanded and narrowed exploitation. MCHIAO demonstrates it’s worth contra ten well-known state-of-the-art optimization algorithms (GOA, MFO, MPA, GWO, HHO, SSA, WOA, IAO, NOA, NGO) in addition to AO and CHIO. Friedman average rank and Wilcoxon statistical analysis (p-value) are conducted on all state-of-the-art algorithms testing 23 benchmark functions. Wilcoxon test and Friedman are conducted as well on the 29 CEC2017 functions. Moreover, some statistical tests are conducted on the 10 CEC2019 benchmark functions. Six real-world problems are used to validate the proposed MCHIAO against the same twelve state-of-the-art algorithms. On classical functions, including 24 unimodal and 44 multimodal functions, respectively, the exploitative and explorative behavior of the hybrid algorithm MCHIAO is evaluated. The statistical significance of the proposed technique for all functions is demonstrated by the p-values calculated using the Wilcoxon rank-sum test, as these p-values are found to be less than 0.05.