Large Optimization Problems Research Articles

Cat swarm optimization with normal mutation for fast convergence of multimodal functions

A normal mutation strategy based cat swarm optimization (NMCSO) that features effective global search capabilities with accelerating convergence speed is presented. The classical CSO suffers from the premature convergence and gets easily trapped in the local optima because of the random mutation process. This frailty has restricted wider range of applications of the classical CSO. To overcome the drawbacks, the normal mutation is adopted in the mutation process of this paper. It enables the cats to seek the positions in better directions by avoiding the problem of premature convergence and local optima. Experiments are conducted on several benchmark unimodal, rotated, unrotated and shifted multimodal problems to demonstrate the effectiveness of the proposed method. Furthermore, NMCSO is also applied to solve the large parameter optimization problems. The experimental results illustrate that the proposed method is quite superior to classical CSO, particle swarm optimization (PSO) and some of the state of the art evolutionary algorithms in terms of convergence speed, global optimality, solution accuracy and algorithm reliability.

A global-best guided phase based optimization algorithm for scalable optimization problems and its application

Large scale optimization problems are more representative of real-world problems and remain one of the most challenging tasks for the design of new type of evolutionary algorithms. Very recently, a new meta-heuristic algorithm named Phase Based Optimization (PBO) inspired by the different motional features of individuals under three different phases (gas phase, liquid phase and solid phase) was proposed. In order to improve PBO for solving large scale optimization problems, an effective search strategy combining complete stochastic search (the diffusion operator) and global-best guided search (the improved perturbation operator) is utilized. The proposed strategy can provide well-balanced compromise between the population diversity (diversification) and convergence speed (intensification) especially in solving large scale optimization problems. We term the improved algorithm as Global-best guided PBO (GPBO) to avoid ambiguity. Seven well-known scalable benchmark functions and a real-world large scale transmission pricing problem are used to validate the performance of GPBO compared with some state-of-the-art algorithms. The experimental results demonstrate that GPBO can provide better solution accuracy and convergence ability in both large scale benchmark functions and real-world optimization problem.

PDPSO: THE FUSION OF PRIMAL-DUAL INTERIOR POINT METHOD AND PARTICLE SWARM OPTIMIZATION ALGORITHM

Particle Swarm Optimization (PSO) is a meta-heuristic algorithm that has been used to solve a variety of complex optimization problems. In spite of the acceptance of the algorithm in various fields, PSO still suffers from common issues such as premature convergence and local minima. This provides a platform for generating a variety of PSO variants. Although these variants are successful in addressing issues specific to a directed domain, they are still unable to resolve the issues effectively. The Interior-Point Methods (IPMs) are efficient tools for solving nonlinear optimization problems. On the one hand, the method is depicted as the most robust algorithm for solving large scale nonlinear optimization problems. On the other, similar to PSO, the methods are still plagued with several issues. We propose Primal-Dual Interior Point Particle Swarm Optimization (pdPSO) to resolve the shortcomings of a standard PSO without the limitations of the IPM methods. We applied the Primal Dual procedure to each particle in a finite number of iterations, and fed the PSO with the its output. We compared the performance of our new algorithm (pdPSO) with IPM and PSO using nine different dynamic benchmark functions. Our results revealed that pdPSO performed better than both the independent PSO algorithm and the IPM method. The proposed algorithm is not susceptible to premature convergence, and can better avoid local minima than conventional PSO, hence hypothetically it has the potential to perform better than many variants of PSO.

Tree-Seed Algorithm for Large-Scale Binary Optimization

Population-based swarm or evolutionary computation algorithms in optimization are attracted the interest of the researchers due their simple structure, optimization performance, easy-adaptation. Binary optimization problems can be also solved by using these algorithms. This paper focuses on solving large scale binary optimization problems by using Tree-Seed Algorithm (TSA) proposed for solving continuous optimization problems by imitating relationship between the trees and their seeds in nature. The basic TSA is modified by using xor logic gate for solving binary optimization problems in this study. In order to investigate the performance of the proposed algorithm, the numeric benchmark problems with the different dimensions are considered and obtained results show that the proposed algorithm produces effective and comparable solutions in terms of solution quality.Keywords: binary optimization, tree-seed algorithm, xor-gate, large-scale optimization

A Distributed Algorithm for Scenario-based Model Predictive Control using Primal Decomposition

In this paper, we consider the decomposition of scenario-based model predictive control problem. Scenario MPC explicitly considers the concept of recourse by representing the evolution of uncertainty by a discrete scenario tree, which can result in large optimization problems. Due to the inherent nature of the scenario tree, the problem can be decomposed into each scenario. The different subproblems are only coupled via the non-anticipativity constraints which ensures that the first control input is the same for all the scenarios. This constraint is relaxed in the dual decomposition approaches, which may lead to infeasibility of the non-anticipativity constraints if the master problem does not converge within the required time. In this paper, we present an alternative approach using primal decomposition which ensures feasibility of the non-anticipativity constraints throughout the iterations. The proposed method is demonstrated using gas-lift optimization as case study.

Multi-GPU configuration of 4D intensity modulated radiation therapy inverse planning using global optimization

We report on the design, implementation and characterization of a multi-graphic processing unit (GPU) computational platform for higher-order optimization in radiotherapy treatment planning. In collaboration with a commercial vendor (Varian Medical Systems, Palo Alto, CA), a research prototype GPU-enabled Eclipse (V13.6) workstation was configured. The hardware consisted of dual 8-core Xeon processors, 256 GB RAM and four NVIDIA Tesla K80 general purpose GPUs. We demonstrate the utility of this platform for large radiotherapy optimization problems through the development and characterization of a parallelized particle swarm optimization (PSO) four dimensional (4D) intensity modulated radiation therapy (IMRT) technique. The PSO engine was coupled to the Eclipse treatment planning system via a vendor-provided scripting interface. Specific challenges addressed in this implementation were (i) data management and (ii) non-uniform memory access (NUMA). For the former, we alternated between parameters over which the computation process was parallelized. For the latter, we reduced the amount of data required to be transferred over the NUMA bridge. The datasets examined in this study were approximately 300 GB in size, including 4D computed tomography images, anatomical structure contours and dose deposition matrices. For evaluation, we created a 4D-IMRT treatment plan for one lung cancer patient and analyzed computation speed while varying several parameters (number of respiratory phases, GPUs, PSO particles, and data matrix sizes). The optimized 4D-IMRT plan enhanced sparing of organs at risk by an average reduction of in maximum dose, compared to the clinical optimized IMRT plan, where the internal target volume was used. We validated our computation time analyses in two additional cases. The computation speed in our implementation did not monotonically increase with the number of GPUs. The optimal number of GPUs (five, in our study) is directly related to the hardware specifications. The optimization process took 35 min using 50 PSO particles, 25 iterations and 5 GPUs.

Robust Optimization of Forecast Combinations

A methodology is developed for constructing robust forecast combinations which improve upon a given benchmark specification for all symmetric and convex loss functions. The optimal forecast combination asymptotically almost surely dominates the benchmark and, in addition, minimizes the expected loss function, under standard regularity conditions. The optimum in a given sample can be found by solving a large Convex Optimization problem. An application to forecasting of changes of the S&P 500 Volatility Index shows that robust optimized combinations improve significantly upon the out-of-sample forecasting accuracy of simple averaging and unrestricted optimization.

Hierarchical parallel processing for design optimization - a case study

The optimization methods can be applied to design large scale mechanical systems. One popular method for increasing the efficiency of large scale design optimization problem is a hierarchical decomposition of the problem into a number of sub problems each with its own objective function, constraints, and design variables. Then each and every sub problem is optimized by using a powerful tool known as genetic algorithm, to get required optimum solutions. The solutions obtained from sub problems are used to get the required global optimum solution of a large and complex design problem.A well known speed reducer is considered as an example to illustrate the application of the above parallel processing method. Speed reducer includes gear, pinion, two shafts, four bearings enclosed in housing. The design objective is to minimize the overall volume, where 7 variables and 25 constraints are considered.

Sustainable energy hub design under uncertainty using Benders decomposition method

A sustainable framework for optimal design of an energy hub (EH), which aims to determine the optimal size of the candidate distributed energy resources (DERs), is proposed in this paper by providing a modeling framework to consider environmental (Env) and social (Soc) impacts of the EH. In order to determine the Env and Soc impacts, the life cycles of different components are analyzed. The external cost method is employed to model the Env and Soc impacts. Also, a two-stage stochastic programming method is utilized to address different uncertainties such as wind speed and solar radiations as well as electricity and heat demands. The problem is formulated as a mixed integer linear programming (MILP). In the optimal EH design problem, the optimal dispatch of DERs should be determined, which leads to a large optimization problem. Benders decomposition (BD) algorithm is used to decompose the original problem in order to address heavy computational burden of the problem, especially when a large number of scenarios is used to properly represent uncertainties. The simulation is performed on a case study which shows the effectiveness of using the BD. In addition, the impacts of taking external cost into consideration on the optimal configuration are investigated.

Enhanced method of fast re-routing with load balancing in software-defined networks

Abstract A two-level method of fast re-routing with load balancing in a software-defined network (SDN) is proposed. The novelty of the method consists, firstly, in the introduction of a two-level hierarchy of calculating the routing variables responsible for the formation of the primary and backup paths, and secondly, in ensuring a balanced load of the communication links of the network, which meets the requirements of the traffic engineering concept. The method provides implementation of link, node, path, and bandwidth protection schemes for fast re-routing in SDN. The separation in accordance with the interaction prediction principle along two hierarchical levels of the calculation functions of the primary (lower level) and backup (upper level) routes allowed to abandon the initial sufficiently large and nonlinear optimization problem by transiting to the iterative solution of linear optimization problems of half the dimension. The analysis of the proposed method confirmed its efficiency and effectiveness in terms of obtaining optimal solutions for ensuring balanced load of communication links and implementing the required network element protection schemes for fast re-routing in SDN.

Parallel cyclic reduction decomposition for dynamic optimization problems

Direct transcription of dynamic optimization problems, with differential-algebraic equations discretized and written as algebraic constraints, can create very large nonlinear optimization problems. When this discretized optimization problem is solved with an NLP solver, such as IPOPT, the dominant computational cost often lies in solving the linear system that generates Newton steps for the KKT system. Computational cost and memory constraints for this linear system solution raise many challenges as the system size increases. On the other hand, the linear KKT system for our dynamic optimization problem is sparse and structured, and can be permuted to form a block tridiagonal matrix. This study explores a parallel decomposition strategy for block tridiagonal systems that is based on cyclic reduction (CR) factorization of the KKT matrix. The classical CR method has good observed performance, but its numerical stability properties need further study for our KKT system. Finally, we discuss modifications to the CR decomposition that improve performance, and we apply the approach to four industrially relevant case studies. On the largest problem, a parallel speedup of a factor of four is observed when using eight processors.

Lasserre Hierarchy for Large Scale Polynomial Optimization in Real and Complex Variables

We propose general notions to deal with large scale polynomial optimization problems and demonstrate their efficiency on a key industrial problem of the 21st century, namely the optimal power flow problem. These notions enable us to find global minimizers on instances with up to 4,500 variables and 14,500 constraints. First, we generalize the Lasserre hierarchy from real to complex numbers in order to enhance its tractability when dealing with complex polynomial optimization. Complex numbers are typically used to represent oscillatory phenomena, which are omnipresent in physical systems. Using the notion of hyponormality in operator theory, we provide a finite convergence criterion which generalizes the Curto--Fialkow conditions of the real Lasserre hierarchy. Second, we introduce the multi-ordered Lasserre hierarchy in order to exploit sparsity in polynomial optimization problems (in real or complex variables) while preserving global convergence. It is based on two ideas: (1) to use a different relaxatio...

A modified scaled memoryless BFGS preconditioned conjugate gradient algorithm for nonsmooth convex optimization

This paper presents a nonmonotone scaled memoryless BFGS preconditioned conjugate gradient algorithm for solving nonsmooth convex optimization problems, which combines the idea of scaled memoryless BFGS preconditioned conjugate gradient method with the nonmonotone technique and the Moreau-Yosida regularization. The proposed method makes use of approximate function and gradient values of the Moreau-Yosida regularization instead of the corresponding exact values. Under mild conditions, the global convergence of the proposed method is established. Preliminary numerical results and related comparisons show that the proposed method can be applied to solve large scale nonsmooth convex optimization problems.

A parallel metaheuristic for large mixed-integer dynamic optimization problems, with applications in computational biology.

BackgroundWe consider a general class of global optimization problems dealing with nonlinear dynamic models. Although this class is relevant to many areas of science and engineering, here we are interested in applying this framework to the reverse engineering problem in computational systems biology, which yields very large mixed-integer dynamic optimization (MIDO) problems. In particular, we consider the framework of logic-based ordinary differential equations (ODEs).MethodsWe present saCeSS2, a parallel method for the solution of this class of problems. This method is based on an parallel cooperative scatter search metaheuristic, with new mechanisms of self-adaptation and specific extensions to handle large mixed-integer problems. We have paid special attention to the avoidance of convergence stagnation using adaptive cooperation strategies tailored to this class of problems.ResultsWe illustrate its performance with a set of three very challenging case studies from the domain of dynamic modelling of cell signaling. The simpler case study considers a synthetic signaling pathway and has 84 continuous and 34 binary decision variables. A second case study considers the dynamic modeling of signaling in liver cancer using high-throughput data, and has 135 continuous and 109 binaries decision variables. The third case study is an extremely difficult problem related with breast cancer, involving 690 continuous and 138 binary decision variables. We report computational results obtained in different infrastructures, including a local cluster, a large supercomputer and a public cloud platform. Interestingly, the results show how the cooperation of individual parallel searches modifies the systemic properties of the sequential algorithm, achieving superlinear speedups compared to an individual search (e.g. speedups of 15 with 10 cores), and significantly improving (above a 60%) the performance with respect to a non-cooperative parallel scheme. The scalability of the method is also good (tests were performed using up to 300 cores).ConclusionsThese results demonstrate that saCeSS2 can be used to successfully reverse engineer large dynamic models of complex biological pathways. Further, these results open up new possibilities for other MIDO-based large-scale applications in the life sciences such as metabolic engineering, synthetic biology, drug scheduling.

Deterministic global optimization of process flowsheets in a reduced space using McCormick relaxations

Deterministic global methods for flowsheet optimization have almost exclusively relied on an equation-oriented formulation where all model variables are controlled by the optimizer and all model equations are considered as equality constraints, which results in very large optimization problems. A possible alternative is a reduced-space formulation similar to the sequential modular infeasible path method employed in local flowsheet optimization. This approach exploits the structure of the model equations to achieve a reduction in problem size. The optimizer only operates on a small subset of the model variables and handles only few equality constraints, while the majority is hidden in externally defined functions from which function values and relaxations for the objective function and constraints can be queried. Tight relaxations and their subgradients for these external functions can be provided through the automatic propagation of McCormick relaxations. Three steam power cycles of increasing complexity are used as case studies to evaluate the different formulations. Unlike in local optimization or in previous sequential approaches relying on interval methods, the solution of the reduced-space formulation using McCormick relaxations enables dramatic reductions in computational time compared to the conventional equation-oriented formulation. Despite the simplicity of the implemented branch-and-bound solver that does not fully exploit the tight relaxations returned by the external functions but relies on further affine relaxation at a single point using the subgradients, in some cases it can solve the reduced-space formulation significantly faster without any range reduction than the state-of-the-art solver BARON can solve the equation-oriented formulation.

Operational deployment of compressive sensing systems for seismic data acquisition

Compressive sensing (CS) provides a new basis for sampling that can increase sampling efficiency for seismic data acquisition by an order of magnitude. A major challenge for this new technology is to show that theoretical increases in sampling efficiency can be translated to real efficiency gains in the field. Along with efficiency gains, data quality must be preserved in order to gain acceptance of a new acquisition technology. CS designs require solution of large optimization problems that are consistent with compressive sampling theory. We refer to our optimization framework for CS-based acquisition design and processing as compressive seismic imaging (CSI). We illustrate our CSI framework on example projects for ocean-bottom node, narrow-azimuth marine streamer, and land vibroseis acquisition. The ocean-bottom-node project was conducted in the UK North Sea during the difficult winter season. A CSI dual-source design was used to significantly reduce shooting time for this project. The project was completed on time, under budget, and with data quality that exceeded the quality of an overlapping uniformly sampled survey. The narrow-azimuth marine CSI survey project was acquired in offshore Australia for field development purposes. Nonuniform CSI sampling was used to increase sampling efficiency for both sources and cables, resulting in significant improvements in data quality and lateral resolution. The land vibroseis project was conducted on the North Slope of Alaska. In this case, the goal was to acquire a development survey of sufficient size within a short time window. Nonuniform CSI sampling was used to support the use of 10 or more vibrators shooting simultaneously, along with improving sampling efficiency for both sources and receivers. Compared to conventional designs, the CSI survey achieved an order of magnitude improvement in field acquisition efficiency and step-function improvements in data quality. These examples show that theoretical improvements in sampling efficiency from CS can make real and significant impacts on seismic data acquisition and processing.

Constrained bundle methods with inexact minimization applied to the energy regulation provision problem

We consider a class of large scale robust optimization problems. While the robust optimization literature often relies on structural assumptions to reformulate the problem in a tractable form using duality, this method is not always applicable and can result in problems which are very large. We propose an alternative way of solving such problems by applying a constrained bundle method. The originality of the method lies in the fact that the minimization steps in the bundle method are solved approximately using the alternating direction method of multipliers. Numerical results from a power grid regulation problem are presented and support the relevance of the approach.

Mobile Data Offloading in FiWi Enhanced LTE-A Heterogeneous Networks

This paper proposes to empower capacity- and coverage-centric fiber-wireless (FiWi) enhanced 4G Long-Term Evolution Advanced (LTE-A) heterogeneous networks (HetNets) with computation- and storage-centric mobile cloud computing to cope with the unprecedented growth of mobile data traffic. Minimizing energy consumption and maximizing revenue while offloading mobile data in such highly converged and hierarchical networks is not trivial, where multiple players (i.e., cloud service providers, macrocell and small cells, users) with multiple objectives coexist. This paper proposes a mobile data offloading framework using a noncooperative multi-level game-theoretic approach from an end-to-end perspective in the envisioned network. More specifically, we design three-level Stackelberg games, in which a single-leader multi-follower game, a multi-leader multi-follower game, and a single-leader multi-follower game are modeled in the introduced network such that individual players selfishly optimize their local payoff functions and collectively solve the large complex network-wide optimization problem. Further, we develop distributed mobile data offloading algorithms to reduce the complexity of the hierarchical games and to achieve a unique Nash equilibrium condition in each subgame. Simulation results show that by reaching the Nash equilibrium condition, the proposed solution helps minimize energy consumption, interference price, and processing cost, while maximizing revenues of the players in the envisioned network. In addition, the efficiency of the equilibria in terms of price of anarchy and price of stability is quantified for the best/worst case of the Nash equilibrium.

On mixed electron-photon radiation therapy optimization using the column generation approach.

Despite considerable increase in the number of degrees of freedom handled by recent radiotherapy optimisation algorithms, treatments are still typically delivered using a single modality. Column generation is an iterative method for solving large optimisation problems. It is well suited for mixed-modality (e.g., photon-electron) optimisation as the aperture shaping and modality selection problem can be solved rapidly, and the performance of the algorithm scales favourably with increasing degrees of freedom. We demonstrate that the column generation method applied to mixed photon-electron planning can efficiently generate treatment plans and investigate its behaviour under different aperture addition schemes. Column generation was applied to the problem of mixed-modality treatment planning for a chest wall case and a leg sarcoma case. 6MV beamlets (100cm SAD) were generated for the photon components along with 5 energies for electron beamlets (6, 9, 12, 16 and 20MeV), simulated as shortened-SAD (80cm) beams collimated with a photon MLC. For the chest wall case, IMRT-only, modulated electron radiation therapy (MERT)-only, and mixed electron-photon (MBRT) treatment plans were created using the same planning criteria. For the sarcoma case, MBRT and MERT plans were created to study the behaviour of the algorithm under two different sets of planning criteria designed to favour specific modalities. Finally, the efficiency and plan quality of four different aperture addition schemes was analysed by creating chest wall MBRT treatment plans which incorporate more than a single aperture per iteration of the column generation loop based on a heuristic aperture ranking scheme. MBRT plans produced superior target coverage and homogeneity relative to IMRT and MERT plans created using the same optimisation criteria, all the while preserving the normal tissue-sparing advantages of electron therapy. Adjusting the planning criteria to favour a specific modality in the sarcoma case resulted in the algorithm correctly emphasizing the appropriate modality. As expected, adding a single aperture per iteration yielded the lowest (best) cost function value per aperture included in the treatment plan. However, a greedier scheme was able to converge to approximately the same cost function after 125 apertures in one third of the running time. Electron apertures were on average 50-100% larger than photon apertures for all aperture addition schemes. The distribution of intensities among the available modalities followed a similar trend for all schemes, with the dominant modalities being 6MV photons along with 6, 9 and 20MeV electrons. The column generation method applied to mixed modality treatment planning was able to produce clinically realistic treatment plans and combined the advantages of photon and electron radiotherapy. The running time of the algorithm depended heavily on the choice of mixing scheme. Adding the highest ranked aperture for each modality provided the best trade-off between running time and plan quality for a fixed number of apertures. This work contributes an efficient methodology for the planning of mixed electron-photon treatments.

Biased randomization of heuristics using skewed probability distributions: A survey and some applications

Randomized heuristics are widely used to solve large scale combinatorial optimization problems. Among the plethora of randomized heuristics, this paper reviews those that contain biased-randomized procedures (BRPs). A BRP is a procedure to select the next constructive ‘movement’ from a list of candidates in which their elements have different probabilities based on some criteria (e.g., ranking, priority rule, heuristic value, etc.). The main idea behind biased randomization is the introduction of a slight modification in the greedy constructive behavior that provides a certain degree of randomness while maintaining the logic behind the heuristic. BRPs can be categorized into two main groups according to how choice probabilities are computed: (i) BRPs using an empirical bias function; and (ii) BRPs using a skewed theoretical probability distribution. This paper analyzes the second group and illustrates, throughout a series of numerical experiments, how these BRPs can benefit from parallel computing in order to significantly outperform heuristics and even simple metaheuristic approaches, thus providing reasonably good solutions in ‘real time’ to different problems in the areas of transportation, logistics, and scheduling.