Large-scale Optimization Research Articles

MOCPSO: A multi-objective cooperative particle swarm optimization algorithm with dual search strategies

Particle swarm optimization (PSO) is a widely embraced meta-heuristic approach to tackling the complexities of multi-objective optimization problems (MOPs), renowned for its simplicity and swift convergence. However, when faced with large-scale multi-objective optimization problems (LSMOPs), most PSOs suffer from limited local search capabilities and insufficient randomness. This can result in suboptimal results, particularly in high-dimensional spaces. To address these issues, this paper introduces MOCPSO, a Multi-Objective Cooperative Particle Swarm Optimization Algorithm with Dual Search Strategies. MOCPSO incorporates a diversity search strategy (DSS) to augment perturbation and enhance the local search scope of particles, alongside a more convergent search strategy (CSS) to expedite particle convergence. Moreover, MOCPSO utilizes a three-category framework to effectively leverage the benefits of both DSS and CSS. Experimental results on benchmark LSMOPs with 500, 1000, and 2000 decision variables demonstrate that MOCPSO outperforms existing state-of-the-art large-scale multi-objective evolutionary algorithms on most test instances.

Sequential Geometric Programming Method for Parameter Estimation of a Nonlinear System in Microbial Continuous Fermentation

This paper addresses the problem of parameter estimation for the microbial continuous fermentation of glycerol to 1,3-propanediol. A nonlinear dynamical system is first presented to describe the microbial continuous fermentation. Some mathematical properties of the dynamical system in the microbial continuous fermentation are also presented. A parameter estimation model is proposed to estimate the parameters of the dynamical system. The proposed estimation model is a large-scale, nonlinear, and nonconvex optimization problem if the number of experimental groups is large. A sequential geometric programming (SGP) method is proposed to efficiently solve the parameter estimation problem. The results indicated that our proposed SGP method can yield smaller errors between the experimental and calculated steady-state concentrations than the existing seven methods. For the five error indices considered, that is, the concentration errors of biomass, glycerol, 1,3-propanediol, acetic acid, and ethanol, the results obtained using the proposed SGP method are better than those obtained using the methods in the literature (Xiu et al., Gao et al., Sun et al., Sun et al., Li and Qu, Wang et al., and Zhang and Xu), with improvements of approximately 71.86–95.03%, 52.08–94.87%, 99.70–99.98%, 5.39–90.29%, and 12.67–80.83%, respectively. This concludes that the established dynamical system can better describe the microbial continuous fermentation. We also present that our established dynamical system has multiple positive steady states in some fermentation conditions. We observe that there are two regions of multiple positive steady states at relatively high values of substrate glycerol concentration in feed medium.

Powered stochastic optimization with hypergradient descent for large-scale learning systems

Stochastic optimization (SO) algorithms based on the Powerball function, namely powered stochastic optimization (PoweredSO) algorithms, have been confirmed, effectively, and demonstrated great potential in the context of large-scale optimization and machine learning tasks. Nevertheless, the issue of how to determine the learning rate for PoweredSO is a challenge and still unsolved problem. In this paper, we propose a class of adaptive PoweredSO approaches that are efficient, scalable and robust. It takes advantage of the hypergradient descent (HD) technique to automatically acquire an online learning rate for PoweredSO-like methods. In the first part, we study the behavior of the canonical PoweredSO algorithm, the Powerball stochastic gradient descent (pbSGD) method, with HD. The existing PoweredSO algorithms also suffer from the high variance because they take the similar algorithmic framework to SO algorithms, arising from sampling tactics. Therefore, the second portion develops an adaptive powered variance-reduced optimization method via utilizing both variance-reduced technique and HD. Moreover, we present the convergence analysis of the proposed algorithms and explore their iteration complexity on non-convex cases. Numerical experiments are conducted on machine learning tasks, verifying the superior performance over modern SO algorithms.

Stochastic Localization Methods for Convex Discrete Optimization via Simulation

Solving Convex Discrete Optimization via Simulation via Stochastic Localization Algorithms Many decision-making problems in operations research and management science require the optimization of large-scale complex stochastic systems. For a number of applications, the objective function exhibits convexity in the discrete decision variables or the problem can be transformed into a convex one. In “Stochastic Localization Methods for Convex Discrete Optimization via Simulation,” Zhang, Zheng, and Lavaei propose provably efficient simulation-optimization algorithms for general large-scale convex discrete optimization via simulation problems. By utilizing the convex structure and the idea of localization and cutting-plane methods, the developed stochastic localization algorithms demonstrate a polynomial dependence on the dimension and scale of the decision space. In addition, the simulation cost is upper bounded by a value that is independent of the objective function. The stochastic localization methods also exhibit a superior numerical performance compared with existing algorithms.

A Lyapunov Theory for Finite-Sample Guarantees of Markovian Stochastic Approximation

The stochastic approximation (SA) method stands as the foundational mathematical tool for modern large-scale optimization and machine learning. Therefore, gaining a theoretical understanding of SA algorithms is of fundamental interest. In their paper titled “A Lyapunov Theory for Finite-Sample Guarantees of Markovian Stochastic Approximation,” Chen et al. present a unified Lyapunov framework for the finite-sample analysis of a Markovian SA algorithm under a contractive operator with respect to an arbitrary norm. The key novelty lies in the construction of a smooth Lyapunov function called the generalized Moreau envelope. The authors demonstrate the effectiveness of their SA results in the context of reinforcement learning (RL), specifically through popular algorithms such as variants of temporal difference (TD) learning and Q-learning. As byproducts, the results provide theoretical insights into the efficiency of bootstrapping in TD learning with eligibility traces and the bias-variance tradeoff in off-policy learning.

Some improved Dai–Yuan conjugate gradient methods for large-scale unconstrained optimization problems

Some improved Dai–Yuan conjugate gradient methods for large-scale unconstrained optimization problems

An Advanced Cloud Data Streaming Framework for Optimized Container Resource Allocation, Job Scheduling, And Security Enhancement

In the realm of cloud data streaming, the central concerns are Container resource allocation and job scheduling. Cloud infrastructure relies on container virtualization to facilitate construction and migration processes. Previous models have employed migration techniques to manage cloud container allocation and resource scheduling, but these come at the cost of increased response time and network traffic. To address these challenges, a novel approach is introduced, the Reduced Optimal Migration model (ROM). This model selectively triggers migration processes based on recommendations, optimizing resource allocation through a Machine Learning (ML) Algorithm. Job scheduling is enhanced through a dedicated Task Scheduling Algorithm. For robust data security during migration, a security-based technique is implemented in the 'Security-based Container Scheduling Model,' which ensures data integrity and safeguards against attacks. This system operates seamlessly online and offline, utilizing Edge Computing. During offline periods, defensive containers maintain data security until the system owner restores online connectivity. This holistic framework proves highly effective in resolving complex issues associated with large-scale optimization of resource allocation, migration, and security. Empirical results confirm its efficiency and security enhancements. The proposed work introduces an advanced cloud data streaming framework that optimizes container resource allocation and job scheduling while enhancing security during migration. It proves effective in addressing the challenges inherent in large-scale cloud data streaming processes.

Three-dimensional topology optimization of natural convection using double multiple-relaxation-time lattice Boltzmann method

A topology optimization method based on the double multiple-relaxation-time (MRT) lattice Boltzmann method and level-set method is developed for 3D natural convection heat transfer with the adjoint double MRT model rigorously derived. The double MRT model with better numerical stability enables the optimization at higher Grashof number (Gr) and larger thermal conductivity ratio. Besides, the model is well-suited for large-scale 3D optimization problems since the ideal linear scaling can be nearly achieved for the whole optimization solution. The forward and optimization models are validated individually by typical problems with relative errors as 2 % and 5 % respectively. Physically reasonable heat sink designs in the shape of “thermal tree” or “thermal flower” are obtained, which can outperform the convectional pin fin and rectangular straight fin with at least 17.8 % lower temperature with Gr ranging from 5.9 × 103 to 1.6 × 106, mainly owing to better organization of fluid flow. Parametric studies find that increasing Gr or decreasing the thermal conductivity ratio results in more contracted heat sinks, while decreasing the penalty factor or increasing the spatial resolution leads to smoother design surface. Anisotropic regularization provides more flexibility in shape control, and adding more solid will enrich the structural details.

Difficulty and Contribution-Based Cooperative Coevolution for Large-Scale Optimization

Cooperative coevolution (CC) is a paradigm equipped with the divide-and-conquer strategy for solving large-scale optimization problems. Currently, the computational resource allocation schemes of most CC could be divided into two categories, namely equal allocation to all subproblems and preference allocation to the subproblems with large contribution. However, the difficult subproblems are not carefully considered by the existing computational resource allocation schemes. For these subproblems, the investment of computational resources cannot quickly improve the fitness value, which leads to their small early contribution and being neglected. In this paper, we comprehensively analyze the imbalanced nature of the subproblems from their difficulty and contribution in large-scale optimization problems. First, we propose a method to quantify the optimization difficulty of the problems during the evolution process, which considers both the difficulty of the fitness landscape and the behaviors of the optimization algorithm. Then, we propose a novel both difficulty and contribution based CC framework, called DCCC, which encourages the allocation of the computational resources to more contributing and more difficult subproblems. DCCC is tested on the CEC’2010 and CEC’2013 large-scale optimization benchmarks, and is compared with several typical CC frameworks and state-of-the-art large-scale optimization algorithms. The experimental results demonstrate that DCCC is very competitive.

Low-Dimensional Space Modeling-Based Differential Evolution for Large-Scale Global Optimization Problems

Large-Scale Global Optimization (LSGO) has been an active research field. Part of this interest is supported by its application to cutting-edge research such as Deep Learning, Big Data, and complex real-world problems such as image encryption, real-time traffic management, and more. However, the high dimensionality makes solving LSGO a significant challenge. Some recent research deal with the high dimensionality by mapping the optimization process to a reduced alternative space. Nonetheless, these works suffer from the changes in the search space topology and the loss of information caused by the dimensionality reduction. This paper proposes a hybrid metaheuristic, so-called LSMDE (Low-dimensional Space Modeling-based Differential Evolution), that uses the Singular Value Decomposition to build a low-dimensional search space from the features of candidate solutions generated by a new SHADE-based algorithm (GM-SHADE). GM-SHADE combines a Gaussian Mixture Model (GMM) and two specialized local algorithms: MTS-LS1 and L-BFGS-B, to promote a better exploration of the reduced search space. GMM mitigates the loss of information in mapping high-dimensional individuals to low-dimensional individuals. Furthermore, the proposal does not require prior knowledge of the search space topology, which makes it more flexible and adaptable to different LSGO problems. The results indicate that LSMDE is the most efficient method to deal with partially separable functions compared to other state-of-the-art algorithms and has the best overall performance in two of the three proposed experiments. Experimental results also show that the new approach achieves competitive results for non-separable and overlapping functions on the most recent test suite for LSGO problems.

Cooperative tri-population based evolutionary algorithm for large-scale multi-objective optimization

Cooperative tri-population based evolutionary algorithm for large-scale multi-objective optimization

Distributed Optimization for Autonomous Restoration in DER-Rich Distribution System

Autonomous service restoration (ASR) of active distribution network (ADN) reduces service restoration time with the help of measurement devices, smart switches and distributed energy resources (DERs) considering the system's operational and radiality constraints. Restoration scheme can be deployed in both centralized and distributed manner. However, with increasing data points, the requirement for measurement availability at the control center for the decision support makes centralized optimization challenging. The decomposition and coordination scheme of distributed algorithms enhances its ability to solve large scale optimization problems. Moreover, distributed optimization enables robustness, scalability and resiliency compared to centralized optimization. In this work, a) the distributed ASR problem is solved using a novel penalty-driven distributed alternating direction method of multipliers algorithm (PD-ADMM), b) a switch level decomposition and coordination technique is proposed, c) load restoration, DER utilization are considered directly and radiality of ADN is enforced using commodity flow model. The applicability of the algorithm is validated using modified IEEE 33-bus, IEEE 123-bus and 1069-bus test systems for contingency cases, communication failure, scalability, accuracy and sensitivity to number of partitions, starting points, change in DER / solar generation.

Redefined decision variable analysis method for large-scale optimization and its application to feature selection

Redefined decision variable analysis method for large-scale optimization and its application to feature selection

An Enhanced Backtracking Search Algorithm for the Flight Planning of a Multi-Drones-Assisted Commercial Parcel Delivery System

Using drones to carry out commercial parcel delivery can significantly promote the transformation and upgrading of the logistics industry thanks to the saving of human labor source, which is becoming a new component of intelligent transportation systems. However, the flight distance of drones is often constrained due to the limited battery capacity. To address this challenge, this paper designs a multi-drones-assisted commercial parcel delivery system, which supports long-distance delivery by a generalized service network (GSN). Each node of the GSN is equipped with charging piles to provide a charging service for drones. Given the limited number of charging piles at each node and the limited battery capacity of a drone, to ensure the efficient operation of the system, the flight planning problem of drones is converted into a large-scale optimization problem by a priority-based encoding mechanism. To solve this problem, an enhanced backtracking search algorithm (EBSA) is reported, which is inspired by the characteristics of the considered flight planning problem and the weak ability of the backtracking search algorithm to escape from a local optimum. The core components of EBSA are the designed comprehensive learning mechanism and local escape operator. Experimental results prove the validity of the improved strategies and the excellent performance of EBSA on the considered flight planning problem.

A Large-Scale Sensor Layout Optimization Algorithm for Improving the Accuracy of Inverse Finite Element Method.

The inverse finite element method (iFEM) based on fiber grating sensors has been demonstrated as a shape sensing method for health monitoring of large and complex engineering structures. However, the existing optimization algorithms cause the local optima and low computational efficiency for high-dimensional strain sensor layout optimization problems of complex antenna truss models. This paper proposes the improved adaptive large-scale cooperative coevolution (IALSCC) algorithm to obtain the strain sensors deployment on iFEM, and the method includes the initialization strategy, adaptive region partitioning strategy, and gbest selection and particle updating strategies, enhancing the reconstruction accuracy of iFEM for antenna truss structure and algorithm efficiency. The strain sensors optimization deployment on the antenna truss model for different postures is achieved, and the numerical results show that the optimization algorithm IALSCC proposed in this paper can well handle the high-dimensional sensor layout optimization problem.

A two-space-decomposition-based evolutionary algorithm for large-scale multiobjective optimization

Large-scale multiobjective optimization problems (LSMOPs) pose a great challenge to maintaining the diversity of solutions. However, existing large-scale multiobjective optimization algorithms (MOEAs) prefer to directly use environmental selection methods designed for small-scale optimization problems. These methods are not effective in solving complex LSMOPs. To address this issue, this paper proposes a two-space (decision space and objective space) decomposition (TSD)-based diversity maintenance mechanism. Its main idea is to explicitly decompose the decision space and objective space into a number of subspaces, each of which may contain some Pareto-optimal solutions. Searching for Pareto-optimal solutions in these subspaces may help maintain the diversity of solutions. To this end, a diversity design subspace (DDS) is constructed to decompose the decision space. Then, a large-scale MOEA (MOEA/TSD) is designed by using the proposed TSD-based diversity maintenance mechanism. Experimental studies validate the effectiveness of the proposed TSD mechanism. Compared with nine state-of-the-art large-scale MOEAs on 112 benchmark LSMOPs, our proposed algorithm offers considerable advantages in overall optimization performance. The source code of MOEA/TSD is available at https://github.com/yizhizhede/MOEATSD.

Effective implementation of text{L}{0}-regularised compressed sensing with chaotic-amplitude-controlled coherent Ising machines

Coherent Ising machine (CIM) is a network of optical parametric oscillators that can solve large-scale combinatorial optimisation problems by finding the ground state of an Ising Hamiltonian. As a practical application of CIM, Aonishi et al., proposed a quantum-classical hybrid system to solve optimisation problems of l_0-regularisation-based compressed sensing. In the hybrid system, the CIM was an open-loop system without an amplitude control feedback loop. In this case, the hybrid system is enhanced by using a closed-loop CIM to achieve chaotic behaviour around the target amplitude, which would enable escaping from local minima in the energy landscape. Both artificial and magnetic resonance image data were used for the testing of our proposed closed-loop system. Compared with the open-loop system, the results of this study demonstrate an improved degree of accuracy and a wider range of effectiveness.

A Three-term Conjugate Gradient Method with a Random Parameter for Large-scale Unconstrained Optimization and its Application in Regression Model

A Three-term Conjugate Gradient Method with a Random Parameter for Large-scale Unconstrained Optimization and its Application in Regression Model

A learning and potential area-mining evolutionary algorithm for large-scale multi-objective optimization

For large-scale multi-objective optimization problems, it is not easy for existing multi-objective evolutionary algorithms to search the entire decision variable space with limited computation resources. To alleviate this problem, we propose a learning and potential area-mining evolutionary algorithm to explore and exploit key regions for accelerating optimization. First, we mine promising areas by clustering the population-gathered regions. Then, potential directions are determined by a multi-guiding point scheme in these promising areas. Subsequently, we design a local search and global search scheme to enhance population convergence while ensuring diversity. Finally, a mutation strategy is used to improve diversity. We execute numerical experiments on two widely-used LSMOP benchmarks and compare the proposed algorithm with three state-of-the-art algorithms. The statistical results indicate that the proposed algorithm has significant performance.

A multi-step relay implementation of the successive iteration of analysis and design method for large-scale natural frequency-related topology optimization

A multi-step relay implementation of the successive iteration of analysis and design method for large-scale natural frequency-related topology optimization