Large-scale Optimization Research Articles

Benders decomposition for the energy aware task scheduling of constellations of nanosatellites

This paper describes a novel method for optimizing nanosatellite constellation scheduling based on Benders decomposition. The optimization of nanosatellite constellations is challenging due to the large number of variables and constraints involved, which increases the computational work required to get optimal solutions as the number of participating objects increases. Our strategy overcomes this obstacle by decomposing the complex problem into smaller, more manageable subproblems, rendering the original formulation scalable. The outcomes demonstrate that the Benders decomposition achieves the same optimal solutions obtained with the Gurobi solver applied to the baseline formulation while being significantly faster and more efficient, especially for larger constellation sizes. This method provides a valuable tool for decision-makers in the nanosatellite job scheduling process, highlighting the potential benefits of using decomposition methods such as the Benders decomposition for large-scale or challenging optimization problems in Offline Nanosatellite Task Scheduling (ONTS).

Managing network congestion with link-based incentives: A surrogate-based optimization approach

Incentive-based travel demand management (IBTDM) programs endow monetary incentives to encourage travel demand redistribution across space and time. They are more appealing than alternatives such as congestion charging because commuters do not need to pay out of pocket. However, such congestion-alleviation solutions are usually managed by small private companies with constrained incentive budgets. Thus, the incentive should be wisely determined so that a limited incentive budget can be effectively used to fulfill maximum social welfare while maintaining the financial health of the IBTDM program. It is essential to know whether IBTDM is financially sound—that is, whether financial investment in IBTDM will lead to more than the equivalent value in total system travel time reduction. However, optimizing the link-based endowment scheme in a large-scale network is challenging because 1) the objective function and the budget constraint are both characterized by expensive-to-evaluate functions without closed form, and 2) it is a large-scale optimization problem that contains massive amount of decision variables. In this study, a computationally efficient surrogate-based optimization framework that is suitable for high-dimensional problems is proposed. A simulation-based dynamic traffic assignment model is used to evaluate the performance of transportation systems, and a Kriging model with partial least squares acts as the surrogate to approximate the simulation model. The results show that the optimal network-wide link-based incentive scheme improves the performance of the system. The higher the incentive budget, the more effective the incentive and the lower the marginal utility of the incentive. Furthermore, in a well-designed incentive scheme, a $1M investment in IBTDM would lead to much more than the equivalent of $1M in total system travel time reduction, which proves the economic viability of IBTDM and provides support for its promotion. IBTDM implemented within smaller regions and tighter incentive budgets produces higher utility ratios.

Distributed generation aggregators considering low-carbon credits optimize dispatch strategies

Abstract Against the backdrop of China’s implementation of the “dual carbon” target and carbon emissions trading policies, renewable energy generation technologies have matured and received support from related policies. Distributed power sources have played a crucial role in the power system, and aggregators have integrated a large number of distributed power sources with diverse characteristics, shielding the complex characteristics of the underlying distributed power sources from grid scheduling. This article introduces the revenue optimization of low-carbon integration to optimize the aggregator’s scheduling model, designs distributed renewable energy generation units, and studies the solution strategy based on quantum genetic algorithms for large-scale optimization scheduling problems. The aggregator’s optimization variables divide the entire optimization problem and consider low-carbon integration to achieve distributed management of green energy parks, providing a feasible theoretical framework for the further development of distributed power sources. It has important practical significance in energy conservation, emissions reduction, and ecological environmental protection.

A co-evolutionary algorithm based on sparsity clustering for sparse large-scale multi-objective optimization

Sparse large-scale multi-objective optimization problems (LSMOPs), which are characterized by high dimensional search space and sparse Pareto optimal solutions, have a widespread existence in academic research and practical applications. While the high dimensional decision space poses challenges to multi-objective evolutionary algorithms (MOEAs), the difficulty of solving sparse LSMOPs can be alleviated by utilizing the prior knowledge that the optimal solutions are sparse. In this paper, a co-evolutionary algorithm based on sparsity clustering, namely SCEA, is proposed, where the prior knowledge of sparse optimal solutions is utilized explicitly. At each generation, SCEA first calculates the current optimal sparsity by sparsity clustering. Then, SCEA divides the population into a winner subpopulation and two loser subpopulations. While the winner subpopulation reproduces offspring solutions by conventional genetic operators, the loser subpopulations generate offspring solutions along two competitive directions under the guidance of current optimal sparsity and variable importance. In the experiments, four state-of-the-art MOEAs are selected as the comparative algorithms. Experimental results show that the proposed algorithm is superior to the four competitors on both benchmark problems and practical applications, which include the sparse signal reconstruction problem, the community detection problem, and the instance selection problem.

GPU-accelerated Bloch simulations and MR-STAT reconstructions using the Julia programming language.

MR-STAT is a relatively new multiparametric quantitative MRI technique in which quantitative paramater maps are obtained by solving a large-scale nonlinear optimization problem. Managing reconstruction times is one of the main challenges of MR-STAT. In this work we leverage GPU hardware to reduce MR-STAT reconstruction times. A highly optimized, GPU-compatible Bloch simulation toolbox is developed as part of this work that can be utilized for other quantitative MRI techniques as well. The Julia programming language was used to develop a flexible yet highly performant and GPU-compatible Bloch simulation toolbox called BlochSimulators.jl. The runtime performance of the toolbox is benchmarked against other Bloch simulation toolboxes. Furthermore, a (partially matrix-free) modification of a previously presented (matrix-free) MR-STAT reconstruction algorithm is proposed and implemented using the Julia language on GPU hardware. The proposed algorithm is combined with BlochSimulators.jl and the resulting MR-STAT reconstruction times on GPU hardware are compared to previously presented MR-STAT reconstruction times. The BlochSimulators.jl package demonstrates superior runtime performance on both CPU and GPU hardware when compared to other existing Bloch simulation toolboxes. The GPU-accelerated partially matrix-free MR-STAT reconstruction algorithm, which relies on BlochSimulators.jl, allows for reconstructions of 68 seconds per two-dimensional (2D slice). By combining the proposed Bloch simulation toolbox and the partially matrix-free reconstruction algorithm, 2D MR-STAT reconstructions can be performed in the order of one minute on a modern GPU card. The Bloch simulation toolbox can be utilized for other quantitative MRI techniques as well, for example for online dictionary generation for MR Fingerprinting.

Large-scale sandwich structures optimization using Bayesian method

Benefiting from advanced features like high stiffness-to-weight ratios, sandwich structures are widely used in aerospace for primary and secondary structures. As tasks grow more complex and structures increase in scale, high-dimensional design spaces inevitably arise. Optimizing large-scale sandwich structures efficiently and intelligently presents certain challenges. Additionally, functional requirements and constraints, such as thermal deformation, should be fully considered in the design of practical structures like solar arrays, which involve a large expensive analysis and make the problem more complicated. This paper proposes to use efficient Bayesian optimization with the active subspace (AC) method to address this type of problem. The active subspace method, combined with the adaptive kriging and global sensitivity analysis (GSA), is employed for dimension reduction and reconstructing the design space. Then the structure is optimized within the reconstructed design space using the efficient Bayesian method. The proposed optimization strategy is applied to a case study of solar array with sandwich panels, demonstrating that the developed framework is feasible and effective for structural optimization of large-scale sandwich structures.

Improving Performance Insensitivity of Large-Scale Multiobjective Optimization via Monte Carlo Tree Search.

The large-scale multiobjective optimization problem (LSMOP) is characterized by simultaneously optimizing multiple conflicting objectives and involving hundreds of decision variables. Many real-world applications in engineering can be modeled as LSMOPs; simultaneously, engineering applications require insensitivity in performance. This requirement typically means that the algorithm should not only produce good results in terms of performance for every run but also the performance of multiple runs should not fluctuate too much. However, existing large-scale multiobjective optimization algorithms often focus on improving algorithm performance, but pay little attention to improving the insensitivity characteristic of algorithms. This directly leads to substantial limitations when solving practical problems. In this work, we propose an evolutionary algorithm called large-scale multiobjective optimization algorithm via Monte Carlo tree search, which is based on the Monte Carlo tree search and aims to improve the performance and insensitivity of solving LSMOPs. The proposed method samples decision variables to construct new nodes on the Monte Carlo tree for optimization and evaluation, and it selects nodes with good evaluations for further searches in order to reduce the performance sensitivity caused by large-scale decision variables. We propose two metrics to measure the sensitivity of the algorithm and compare the proposed algorithm with several state-of-the-art designs on different benchmark functions and metrics. The experimental results confirm the effectiveness and performance insensitivity of the proposed design for solving LSMOPs.

Debiasing In-Sample Policy Performance for Small-Data, Large-Scale Optimization

In many modern large-scale decision-making problems, data can be scarce. As a result, traditional methods such as cross-validation perform poorly in evaluating the performance of decision-making policies. In “Debiasing In-Sample Policy Performance for Small-Data, Large-Scale Optimization,” Gupta, Huang, and Rusmevichientong propose a novel estimator of the out-of-sample performance for a policy in data-driven optimization. Unlike cross-validation, their approach avoids sacrificing training data for evaluation. As a result, they theoretically show the estimator is asymptotically unbiased as the problem size grows. Furthermore, they show that the estimator is asymptotically optimal when applied to more specialized “weakly coupled” optimization problems. Finally, using a case study on dispatching emergency medical response services, they demonstrate their proposed method provides more accurate estimates of out-of-sample performance and selects better policies.

Large-scale quantum approximate optimization on nonplanar graphs with machine learning noise mitigation

Quantum computers are increasing in size and quality but are still very noisy. Error mitigation extends the size of the quantum circuits that noisy devices can meaningfully execute. However, state-of-the-art error mitigation methods are hard to implement and the limited qubit connectivity in superconducting qubit devices restricts most applications to the hardware's native topology. Here we show a quantum approximate optimization algorithm (QAOA) on nonplanar random regular graphs with up to 40 nodes enabled by a machine learning-based error mitigation. We use a swap network with careful decision-variable-to-qubit mapping and a feed-forward neural network to optimize a depth-two QAOA on up to 40 qubits. We observe a meaningful parameter optimization for the largest graph which requires running quantum circuits with 958 two-qubit gates. Our paper emphasizes the need to mitigate samples, and not only expectation values, in quantum approximate optimization. These results are a step towards executing quantum approximate optimization at a scale that is not classically simulable. Reaching such system sizes is key to properly understanding the true potential of heuristic algorithms like QAOA. Published by the American Physical Society 2024

Large-Scale Binary Matrix Optimization for Multimicrogrids Network Structure Design

Large-Scale Binary Matrix Optimization for Multimicrogrids Network Structure Design

Distributed Stochastic Consensus Optimization Using Communication-Censoring Strategy

In this paper, a novel communication-efficient distributed stochastic algorithm (referred as CO-DSA) is proposed for solving large-scale consensus optimization problems. As compared to the existing relevant work where only a sublinear convergence rate is obtained for strongly convex and smooth objective functions, CO-DSA achieves a linear convergence rate even in the presence of an event-triggered based communication-censoring strategy. Moreover, by properly setting the threshold function of the event-triggered communication scheme, CO-DSA maintains the same convergence rate as the algorithm without event-triggered communication. This means CO-DSA theoretically yields communication efficiency for free. Numerical experiments verify the theoretical findings and also show the excellent communication saving effect of CO-DSA in large distributed networks.

A Large-Scale Multiobjective Particle Swarm Optimizer With Enhanced Balance of Convergence and Diversity.

Large-scale multiobjective optimization problems (LSMOPs) continue to be challenging for existing multiobjective evolutionary algorithms (MOEAs). The main difficulties are that: 1) the diversity preservation in both the objective space and the decision space needs to be taken into account when solving LSMOPs and 2) the existing learning structures in current MOEAs usually make the learning operators only coincidentally serve convergence and diversity, leading to difficulties in balancing these two factors. Therefore, balancing convergence and diversity in current MOEAs is difficult. To address these issues, this article proposes a multiobjective particle swarm optimizer with enhanced balance of convergence and diversity (MPSO-EBCD). In MPSO-EBCD, a novel velocity update structure for multiobjective particle swarm optimization is put forward, dividing the convergence, and diversity preservation operations into independent components. Following the proposed update structure, a weighted convergence factor is introduced to serve the convergence strategy, whilst a diversity preservation strategy is built to uniformly distribute the particles in the searched space based on a proposed multidimensional local sparseness degree indicator. By this means, MPSO-EBCD is able to balance convergence and diversity with specific parameters in independent operators. Experimental results on LSMOP benchmarks and a voltage transformer optimization problem demonstrate the competitiveness of the proposed algorithm compared to several state-of-the-art MOEAs.

Model-Free Approximate Bayesian Learning for Large-Scale Conversion Funnel Optimization

The flexibility of choosing the ad action as a function of the consumer state is critical for modern-day marketing campaigns. We study the problem of identifying the optimal sequential personalized interventions that maximize the adoption probability for a new product. We model consumer behavior by a conversion funnel that captures the state of each consumer (e.g., interaction history with the firm) and allows the consumer behavior to vary as a function of both her state and firm’s sequential interventions. We show our model captures consumer behavior with very high accuracy (out-of-sample area under the curve of over 0.95) in a real-world email marketing dataset. However, it results in a very large-scale learning problem, where the firm must learn the state-specific effects of various interventions from consumer interactions. We propose a novel attribution-based decision-making algorithm for this problem that we call model-free approximate Bayesian learning. Our algorithm inherits the interpretability and scalability of Thompson sampling for bandits and maintains an approximate belief over the value of each state-specific intervention. The belief is updated as the algorithm interacts with the consumers. Despite being an approximation to the Bayes update, we prove the asymptotic optimality of our algorithm and analyze its convergence rate. We show that our algorithm significantly outperforms traditional approaches on extensive simulations calibrated to a real-world email marketing dataset.

A randomized operator splitting scheme inspired by stochastic optimization methods

In this paper, we combine the operator splitting methodology for abstract evolution equations with that of stochastic methods for large-scale optimization problems. The combination results in a randomized splitting scheme, which in a given time step does not necessarily use all the parts of the split operator. This is in contrast to deterministic splitting schemes which always use every part at least once, and often several times. As a result, the computational cost can be significantly decreased in comparison to such methods. We rigorously define a randomized operator splitting scheme in an abstract setting and provide an error analysis where we prove that the temporal convergence order of the scheme is at least 1/2. We illustrate the theory by numerical experiments on both linear and quasilinear diffusion problems, using a randomized domain decomposition approach. We conclude that choosing the randomization in certain ways may improve the order to 1. This is as accurate as applying e.g. backward (implicit) Euler to the full problem, without splitting.

Benchmark problems for large-scale constrained multi-objective optimization with baseline results

The interests in evolutionary constrained multiobjective optimization are rapidly increasing during the past two decades. However, most related studies are limited to small-scale problems, despite the fact that many practical problems contain large-scale decision variables. Although several large-scale constrained multi-objective evolutionary algorithms (CMOEAs) have been developed, they are still tested on benchmarks that are designed for small-scale problems without the features of large-scale problems. To promote the research on large-scale constrained multi-objective optimization (LSCMO), this paper proposes a new LSCMO benchmark based on the design principles of large-scale multi-objective optimization and constrained multi-objective optimization. In this benchmark, more realistic features are considered, such as mixed linkages between constraint variables and unconstrained variables, imbalanced contributions of variables to the objectives, varying number constraint functions. Besides, to better solve the proposed benchmark, a bidirectional sampling strategy is proposed, where a convergence direction sampling and a diversity direction sampling are used to accelerate the convergence and maintain diversity respectively. Furthermore, the proposed bidirectional sampling strategy is embedded into an existing CMOEA to improve the search ability of algorithm in the large-scale search space with constraints. In experiments, the proposed algorithm is compared with several latest peer algorithms, and the results verify that the designed benchmark functions can effectively test the performance of algorithms and the proposed algorithm can better tackle the new benchmark. Finally, the proposed algorithm is used to solve the network structure control-based personalized drug target recognition problems with more than 2000 decision variables, and results show its superiority.

A duality-preserving adjoint method for segregated Navier–Stokes solvers

Adjoint methods efficiently compute gradients for systems with many inputs and have been widely used for large-scale gradient-based optimization in fluid mechanics. To ensure optimization's numerical robustness, we need to develop an adjoint solution algorithm that has a similar, if not the same, convergence rate as the primal flow solver at each optimization iteration. This consistent primal-adjoint convergence behavior is also called duality-preserving (DP). Existing DP adjoint methods are mostly for fully coupled Navier–Stokes (NS) solvers with compressible flows. However, few DP adjoints have been proposed for segregated NS solvers, e.g., the semi-implicit method for pressure-linked equations (SIMPLE) algorithm solvers widely used for incompressible flow simulations. This study will fill this gap by deriving a fixed-point segregated adjoint formulation that fully preserves the convergence behavior of primal solvers. We first rewrite the steady-state segregated NS primal solution process into a fully coupled left-preconditioned Richardson format. Then, we transpose the preconditioner matrix to obtain a DP iterative adjoint formulation. In addition to algebraic derivation, we create a new graph representation that significantly streamlines the DP adjoint development for new segregated solvers. We prove that the proposed graph representation is equivalent to the above preconditioner transpose approach and guarantees duality. We evaluate our proposed algorithm using three cases with increasing complexity: a miniature-sized problem, an airfoil, and a wing. To quantify duality, we compare the eigenvalues between the primal and adjoint solvers and observe excellent agreements. We also evaluate to what extent various numerical settings (e.g., inner iteration tolerances and non-dual preconditioner) impact the eigenvalue distributions. Our proposed adjoint achieves machine-precision accurate gradient computation with competitive speed. Finally, we incorporate our adjoint solver into a large-scale gradient-based optimization framework and demonstrate its capability for wing aerodynamic shape optimization. The proposed fixed-point adjoint approach has the potential to make the adjoint solution more robust and efficient for any segregated NS solvers.

Determining the Right Levels of Health Coaching and Heart Rate Variability Biofeedback in a Workplace Behavior Change Intervention: Multiphase Optimization Strategy Preparation Study.

Work-related stress is associated with poor job performance and negative health outcomes. Changing health behaviors through corporate wellness programs can improve physical and mental health and help employees manage stress. This project sought to pilot the potential addition of brief coaching and biofeedback to an 8-week web-based self-help program to improve employee stress using the multiphase optimization strategy. This study aims to determine which candidate components will be tested in a later optimization phase and at what dose they will be tested, examine the feasibility and acceptability of delivering the different components, investigate whether the outcomes can be feasibly measured, and review evidence to build a conceptual model before the optimization phase. The study was positioned within the preparation phase of the multiphase optimization strategy. It is a 2×2×2×2 design with 4 components: 2 types of health coaching and 2 types of biofeedback. All components were tested by turning them on or off. A total of 16 adult office workers (mean age 40, SD 14.3 years; n=15 women) completed an 8-week self-paced web-based stress management and health behavior change program and were randomly assigned to 1 of the 16 conditions, created from a combination of the 4 candidate components. Assessments included web analytics, surveys, and interviews regarding program recommendations, likes, and dislikes. Findings from the interviews provided suggestions to improve the intervention (eg, separating wellness from stress content) and trial conduct (eg, streamlining the onboarding process). On average, participants logged into the wellness program 83 times (range 36-291), with 75% (12/16) participant retention and 67% (8/12) survey completion. There were no reported problems with coaching or obtaining data from interviews or apps. The interview findings suggested potential mediators to include and assess in a future conceptual model. The results provided areas to improve the intervention content and trial methods. Instead of progressing to the next scheduled large-scale optimization phase, our plan to iterate through a second preparation phase after making changes to the protocol, apps, and corporate coaching partner.

A modified competitive swarm optimizer guided by space sampling for large-scale multi-objective optimization

Multi-objective evolutionary algorithms have demonstrated promising performance in solving multi/many-objective problems. However, their performance decreases sharply when dealing with multi-objective optimization problems with hundreds or thousands of decision variables, which prevent them from quickly converging to the Pareto front. To this end, this article proposes a modified competitive swarm optimizer guided by space sampling for large-scale multi-objective optimization (LSMCSO-SS). In the initialization phase of the algorithm, we propose a space sampling method, which samples a set of individuals according to the upper and lower bounds of the decision space. Then, they are added to the initial population to guide the evolution of the algorithm. In the iteration process of the algorithm, we propose a modified competitive swarm optimizer (CSO). Different from the original CSO algorithm, we add a new velocity component to the losers to further improve the convergence speed. In the experiments, we compare our algorithm with seven state-of-the-art large-scale multi-objective evolutionary algorithms based on the inverted generational distance plus (IGD+) indicator upon nine large-scale multi-objective optimization benchmark problems with up to 5000 decision variables, and the numerous experimental results manifest that the proposed method performs the best on most test instances, which further demonstrates that it outperforms all the seven comparison algorithms.

Solving Large-Scale Unconstrained Optimization Problems with an Efficient Conjugate Gradient Class

The main goal of this paper is to introduce an appropriate conjugate gradient class to solve unconstrained optimization problems. The presented class enjoys the benefits of having three free parameters, its directions are descent, and it can fulfill the Dai–Liao conjugacy condition. Global convergence property of the new class is proved under the weak-Wolfe–Powell line search technique. Numerical efficiency of the proposed class is confirmed in three sets of experiments including 210 test problems and 11 disparate conjugate gradient methods.

A multi-swarm optimizer with a reinforcement learning mechanism for large-scale optimization

Large-scale global optimization (LSGO) problems involve numerous decision variables, are similar to real-world problems, and have generated research interest. To solve LSGO, a particle swarm optimizer (PSO) has been used. However, the many local optima and huge search space severely limit the effectiveness of the classic PSO. Dealing with the complexity of LSGO while avoiding the local optima is the main challenge of large-scale optimization algorithms. A multiswarm strategy has also been introduced to improve swarm diversity; however, it reduces the convergence speed. Previous studies have shown that reinforcement learning (RL) can improve the convergence ability of EAs owing to its increasing learning ability. In this study, we develop a multiswarm optimizer with an RL mechanism (MSORL) for LSGO. The MSORL includes a tri-particle group structure for subswarms to save the computational cost and balance the diversity and convergence. An RL-guided updating strategy is designed to enhance the convergence speed, and an adaptive tolerance-based search mechanism is employed to improve the diversity and avoid the local optima. The experimental results prove that the MSORL outperforms other state-of-the-art algorithms in terms of the convergence accuracy and speed.