Parallel Optimization Algorithm Research Articles

A unified view of parallel multi-objective evolutionary algorithms

This paper describes a unified view of parallel evolutionary algorithms for multi-objective optimization problems. The parallel optimization algorithms are detailed from both design and implementation aspects. The proposed taxonomy is based on three hierarchical parallel models. Moreover, various parallel architectures are taken into account. The performance assessment issue of parallel multi-objective evolutionary algorithms (MOEA) is also presented. This work can be extended to any population-based metaheuristics such as particle swarm and scatter search.

Design of a parallel genetic algorithm for continuous and pattern-free heliostat field optimization

The heliostat field of solar power tower plants can suppose up to 50% of investment costs and 40% of energy loss. Unfortunately, obtaining an optimal field requires facing a complex non-convex, continuous, large-scale, and constrained optimization problem. Although pattern-based layouts and iterative deployment are popular heuristics to simplify the problem, they limit flexibility and might be suboptimal. This work describes a new genetic algorithm for continuous and pattern-free heliostat field optimization. Considering the potential computational cost of the objective function and the necessity of broad explorations, it has been adapted to run in parallel on shared-memory environments. It relies on elitism, uniform crossover, static penalization of infeasibility, and tournament selection. Interesting experimental results show an optimization speedup up to 15 $$\times $$ with 16 threads. It could approximately reduce a one year runtime, at complete optimization, to a month only. The optimizer has also been made available as a generic C++ library.

A parallel boundary search particle swarm optimization algorithm for constrained optimization problems

During the past decade, considerable research has been conducted on constrained optimization problems (COPs) which are frequently encountered in practical engineering applications. By introducing resource limitations as constraints, the optimal solutions in COPs are generally located on boundaries of feasible design space, which leads to search difficulties when applying conventional optimization algorithms, especially for complex constraint problems. Even though penalty function method has been frequently used for handling the constraints, the adjustment of control parameters is often complicated and involves a trial-and-error approach. To overcome these difficulties, a modified particle swarm optimization (PSO) algorithm named parallel boundary search particle swarm optimization (PBSPSO) algorithm is proposed in this paper. Modified constrained PSO algorithm is adopted to conduct global search in one branch while Subset Constrained Boundary Narrower (SCBN) function and sequential quadratic programming (SQP) are applied to perform local boundary search in another branch. A cooperative mechanism of the two branches has been built in which locations of the particles near boundaries of constraints are selected as initial positions of local boundary search and the solutions of local boundary search will lead the global search direction to boundaries of active constraints. The cooperation behavior of the two branches effectively reinforces the optimization capability of the PSO algorithm. The optimization performance of PBSPSO algorithm is illustrated through 13 CEC06 test functions and 5 common engineering problems. The results are compared with other state-of-the-art algorithms and it is shown that the proposed algorithm possesses a competitive global search capability and is effective for constrained optimization problems in engineering applications.

Searching for common patterns on protein sequences by means of a parallel hybrid honey-bee mating optimization algorithm

Abstract At present, most computing systems have multiple cores accessing to a shared memory space. In the case of computing systems devoted to research, it is common to cluster several shared memory architectures together in order to go beyond the limits of a single shared memory architecture. These clusters can represent a significant resource for those researchers who accomplish to exploit them to the fullest. One of the most applied techniques to achieve this purpose is the mixed mode parallel programming which combines both OpenMP and MPI paradigms. In this paper, we present a parallel implementation of a swarm-based evolutionary algorithm designed for solving a complex biological optimization problem. In order to obtain the maximum possible performance, we have combined MPI and OpenMP. Furthermore, in order to solve the addressed problem in a realistic way, we have applied multiobjective optimization. For assessing the performance achieved by our proposal we have conducted experiments under different systems on six biological instances with different sizes. Results point out the relevance of combining mixed mode parallel programming, a swarm-based evolutionary algorithm, and multiobjective optimization from both parallelism and biology points of view.

Acoustic gradient index lens design using gradient based optimization

We demonstrate a gradient based optimization of the absolute acoustic pressure at a focal point by incrementally repositioning a set of cylindrical obstacles so that they eventually act as an effective gradient index (GRIN) lens. The gradient-based optimization algorithm maximizes the absolute pressure at the focal point by analytically evaluating its derivative with respect to the cylinder positions and then perturbatively optimizing the position of each cylinder in the GRIN lens device while taking into account acoustic multiple scattering between the cylinders. The method is presented by examples of sets of hard cylinders and sets of elastic cylindrical shells of uniform size. Designs using available cylindrical shells enable easy and realistic fabrication of acoustic GRIN lenses. Computations are performed on MATLAB using parallel optimization algorithms and a multistart optimization solver, and supplying the gradients of absolute pressure at the focal point with respect to position vectors. Providing the analytic form of the gradients enhances modeling by allowing the use of exact gradients with optimization algorithms and parallel computing. This results in reducing number of function calls and time needed to converge, and improving solution accuracy for large scale optimization problems especially at high frequencies and with a large number of scatterers.

Optimal design of energy systems using constrained grey-box multi-objective optimization

The (global) optimization of energy systems, commonly characterized by high-fidelity and large-scale complex models, poses a formidable challenge partially due to the high noise and/or computational expense associated with the calculation of derivatives. This complexity is further amplified in the presence of multiple conflicting objectives, for which the goal is to generate trade-off compromise solutions, commonly known as Pareto-optimal solutions. We have previously introduced the p-ARGONAUT system, parallel AlgoRithms for Global Optimization of coNstrAined grey-box compUTational problems, which is designed to optimize general constrained single-objective grey-box problems by postulating accurate and tractable surrogate formulations for all unknown equations in a computationally efficient manner. In this work, we extend p-ARGONAUT towards multi-objective optimization problems and test the performance of the framework, both in terms of accuracy and consistency, under many equality constraints. Computational results are reported for a number of benchmark multi-objective problems and a case study of an energy market design problem for a commercial building, while the performance of the framework is compared with other derivative-free optimization solvers.

Asynchronously parallel optimization solver for finding multiple minima

We propose and analyze an asynchronously parallel optimization algorithm for finding multiple, high-quality minima of nonlinear optimization problems. Our multistart algorithm considers all previously evaluated points when determining where to start or continue a local optimization run. Theoretical results show that when there are finitely many minima, the algorithm almost surely starts a finite number of local optimization runs and identifies every minimum. The algorithm is applicable to general optimization settings, but our numerical results focus on the case when derivatives are unavailable. In numerical tests, a Python implementation of the algorithm is shown to yield good approximations of many minima (including a global minimum), and this ability is shown to scale well with additional resources. Our implementation’s time to solution is shown also to scale well even when the time to perform the function evaluation is highly variable. An implementation of the algorithm is available in the libEnsemble library at https://github.com/Libensemble/libensemble .

HPC optimal parallel communication algorithm for the simulation of fractional-order systems

A parallel numerical simulation algorithm is presented for fractional-order systems involving Caputo-type derivatives, based on the Adams–Bashforth–Moulton predictor–corrector scheme. The parallel algorithm is implemented using several different approaches: a pure MPI version, a combination of MPI with OpenMP optimization and a memory saving speedup approach. All tests run on a BlueGene/P cluster, and comparative improvement results for the running time are provided. As an applied experiment, the solutions of a fractional-order version of a system describing a forced series LCR circuit are numerically computed, depicting cascades of period-doubling bifurcations which lead to the onset of chaotic behavior.

Parallel design of intelligent optimization algorithm based on FPGA

Intelligent optimization algorithm (IOA) has been widely studied and applied to solve various optimization problems. When scholars improve IOA with mathematical methods, they also want to seek an effective method to implement algorithms with higher real time, especially for a complex problem. Parallel design is an effective method to improve the real time of IOA. Currently, the parallel programming based on open multi-processing (OpenMP) and compute unified device architecture (CUDA) are two popular methods. To find and develop a new IOA parallel method, in this paper, a parallel design and implementation method based on field programmable gate array (FPGA) is explored. In order to validate the proposed method, parallel genetic algorithm (GA) and parallel particle swarm optimization (PSO) algorithm are realized by the proposed method. Furthermore, the performance and advantage of the proposed FPGA-based parallel IOA method are tested by comparing with OpenMP-based parallel programming and CUDA-based parallel programming, the final results show that the proposed method with highest real-time performance in IOA parallel implementation. A case study by using FPGA-based parallel simulate annealing (SA) to address job shop scheduling problem (JSSP) to illustrate the proposed method has high potential in industrial applications.

Parallel Data Partitioning Algorithms for Optimization of Data-Parallel Applications on Modern Extreme-Scale Multicore Platforms for Performance and Energy

Data partitioning algorithms aiming to minimize the execution time and the energy of computations in self-adaptable data-parallel applications on modern extreme-scale multicore platforms must address two critical challenges. First, they must take into account the new complexities inherent in these platforms such as severe resource contention and non-uniform memory access. Second, they must have low practical runtime and memory costs. The sequential data partitioning algorithms addressing the first challenge have a theoretical time complexity of O( $m * m * p * p$ ) where $m$ is the number of points in the discrete speed/energy function and $p$ is the number of available processors. They, however, exhibit high practical runtime cost and excessive memory footprint, therefore, rendering them impracticable for employment in self-adaptable applications executing on extreme-scale multicore platforms. We present, in this paper, the parallel data partitioning algorithms that address both the challenges. They take as input the functional models of performance and energy consumption against problem size and output workload distributions, which are globally optimal solutions. They have a low time complexity of O( $m * m * p$ ) thereby providing a linear speedup of O( $p$ ) and low memory complexity of O( $n$ ) where $n$ is the workload size expressed as a multiple of granularity. They employ dynamic programming approach, which also facilitates the easier integration of performance and energy models of communications. We experimentally study the practical cost of application of our algorithms in two data-parallel applications, matrix multiplication and fast Fourier transform, on a cluster in Grid’5000 platform. We demonstrate that their practical runtime and memory costs are low making them ideal for employment in self-adaptable applications. We also show that the parallel algorithms exhibit tremendous speedups over the sequential algorithms. Finally, using theoretical analysis for a forecast exascale platform, we demonstrate that the parallel algorithms have negligible execution times compared to the matrix multiplication application executing on the platform.

Parallel parameter optimization algorithm in dynamic general equilibrium models

Abstract We present a parallel parameter optimization algorithm for reproducing future projections of certain model outputs in dynamic general equilibrium models. The optimization problem is reduced to a nonlinear system of equations. The Jacobian matrix for a Newton-type solver in the problem is generated in parallel. The parameter optimization algorithm is implemented for parallel systems with distributed memory by using MPI. To achieve better performance of the parallel algorithm we use the parallel Fair – Taylor algorithm for computing an equilibrium path. Calculation of prices, input-output ratios and international trade for different time steps is carried out in parallel at each iteration of the method. The solution method is implemented for parallel systems with shared memory by using OpenMP. The effectiveness of the hybrid MPI+OpenMP parallel code for parameter optimization is demonstrated in the example of a global multi-sector energy economics model with scenarios that are used for studying climate change impacts on land use.

Improved asynchronous parallel optimization analysis for stochastic incremental methods

As data sets continue to increase in size and multi-core computer architectures are developed, asynchronous parallel optimization algorithms become more and more essential to the field of Machine L...

Local search for parallel optimization algorithms for high diminsional optimization problems

Local search algorithms perform an important role when being employed with optimization algorithms tackling numerous optimization problems since they lead to getting better solutions. However, this is not practical in many applications as they do not contribute to the search process. This was not much studied previously for traditional optimization algorithms or for parallel optimization algorithms. This paper investigates this issue for parallel optimization algorithms when tackling high dimensional subset problems. The acquired results show impressive recommendations.

A Parallelization Research for FY Satellite Rainfall Estimate Day Knock off Product Algorithm

With the development of satellite remote sensing technology, more and more requirements are put forward on the timeliness and stability of the satellite weather service system. The FY satellite rainfall estimate day knock off product algorithm runs longer, about 20 minutes, which affects the estimated rainfall product generated timeliness. Research and development of parallel optimization algorithms based on the needs of satellite meteorological services and their effectiveness in practical applications are necessary ways to enhance the high-performance and high-availability capabilities of satellite meteorological services. So aiming at this problem, we started the parallel algorithm research based on the analysis of precipitation estimation algorithm. Firstly, we explained the steps of precipitation estimated date knock off product algorithm; secondly, we analyzed the four main calculation module calculating the amount of algorithms; thirdly, multithreaded parallel algorithm and MPI parallelization was designed. Finally, the multithreaded parallel and MPI parallelization were realized. Experimental results show that the multithreaded parallel and MPI parallelization algorithm could greatly improve the overall degree of computational efficiency. And, MPI parallelization mode has a higher operating efficiency. The performance of parallel processing is closely related to the architecture of the computer. From the perspective of service scheduling and product algorithms, the MPI parallelization approach is adopted to achieve the purpose of improving service quality.

A novel parallel hurricane optimization algorithm for secure emission/economic load dispatch solution

This paper proposes a parallel hurricane optimization algorithm (PHOA) for solving economic emission load dispatch (EELD) problem in modern power systems. In PHOA, several sub-populations moving independently in the search space with the aim of simultaneously optimizing the problem objectives considering the local behavior between sub-populations. By this way, it is intended to search for the Pareto optimal solutions which are contrasting to the single optimal solution. The inherent characteristics of parallelization strategy can enhance the Pareto solutions and increase the convergence to reach the Pareto optimal solutions. Simulations are conducted on three test systems and comparisons with other optimization techniques that reported in the literature are demonstrated. The obtained results demonstrate the superiority of the proposed PHOA compared to other optimization techniques. Additional economic benefits with secure settings are fulfilled, while preserving all system constraints within their permissible limits. Added to that, two security indices are proposed from generation units and transmission lines. The highest security index from generation units reflects that the operating condition achieves more power reserve. In transmission lines, the highest security index means that the transmission lines operated beyond their congestion limits. For justification of the proposed security indices, the proposed solution methodology is employed to assure their benefits in terms of economical and environmental issues. The proposed algorithm improves the economic issue as well as enhances the power system operation in the technical point of view with acceptable levels of emissions. Moreover, design of experiments using the Taguchi approach is employed to calibrate the parameters of the algorithms. So, it can be considered as a promising alternative algorithm for solving problems in practical large-scale power systems.

A parallel hybrid optimization algorithm for some network design problems

Network design problems have been widely studied in the last decades due to the importance of ICT in our daily life and are still the subject of extensive researches. Network design covers a large family of problems, and several algorithms, both exact and heuristic methods, have been proposed to address each of them. In this paper, we consider two variants of the so-called survivable network design problem and propose a generic parallel hybrid algorithm to solve them. The algorithm is based on the hybridization of a Lagrangian relaxation algorithm, a greedy algorithm and a genetic algorithm. We present, for each variant, a computational study showing the efficiency of our approach in producing both lower and upper bounds for the optimal solution.

Model predictive controller-based multi-model control system for longitudinal stability of distributed drive electric vehicle

Distributed drive electric vehicle(DDEV) has been widely researched recently, its longitudinal stability is a very important research topic. Conventional wheel slip ratio control strategies are usually designed for one special operating mode and the optimal performance cannot be obtained as DDEV works under various operating modes. In this paper, a novel model predictive controller-based multi-model control system (MPC-MMCS) is proposed to solve the longitudinal stability problem of DDEV. Firstly, the operation state of DDEV is summarized as three kinds of typical operating modes. A submodel set is established to accurately represent the state value of the corresponding operating mode. Secondly, the matching degree between the state of actual DDEV and each submodel is analyzed. The matching degree is expressed as the weight coefficient and calculated by a modified recursive Bayes theorem. Thirdly, a nonlinear MPC is designed to achieve the optimal wheel slip ratio for each submodel. The optimal design of MPC is realized by parallel chaos optimization algorithm(PCOA)with computational accuracy and efficiency. Finally, the control output of MPC-MMCS is computed by the weighted output of each MPC to achieve smooth switching between operating modes. The proposed MPC-MMCS is evaluated on eight degrees of freedom(8DOF)DDEV model simulation platform and simulation results of different condition show the benefits of the proposed control system.

Joint capacity planning and distribution network optimization of coal supply chains under uncertainty

A two‐stage stochastic integer programming model is developed to address the joint capacity planning and distribution network optimization of multiechelon coal supply chains (CSCs) under uncertainty. The proposed model not only introduces the uses of compound real options in sequential capacity planning, but also considers uncertainty induced by both risks and ambiguities. Both strategic decisions (i.e., facility locations and initial investment, service assignment across the entire CSC, and option holding status) and scenario‐based operational decisions (i.e., facility operations and capacity expansions, outsourcing policy, and transportation and inventory strategies) can be simultaneously determined using the model. By exploiting the nested decomposable structure of the model, we develop a new distributed parallel optimization algorithm based on nonconvex generalized Bender decomposition and Lagrangean relaxation to mitigate the computation resource limitation. One of the main CSCs in China is studied to demonstrate the applicability of the proposed model and the performance of the algorithm. © 2017 American Institute of Chemical Engineers AIChE J, 64: 1246–1261, 2018

The Bernstein algorithm using the modified implicit Bernstein form and its GPU parallelization using CUDA

In this paper, we propose a Modified Implicit Bernstein Form (MIBF) for computing the Bernstein coefficients of a polynomial. The MIBF avoids several redundant computations in Smith’s Implicit Bernstein Form (IBF) (Smith in J Glob Optim 43:445–458, 2009). Based on the MIBF, we then propose a serial (or CPU based) Bernstein Algorithm for polynomial global optimization. On a set of test problems, the proposed Bernstein Algorithm is about 1.8 times faster than the one based on IBF in Dhabe and Nataraj (Int J Syst Assur Eng Manag, 2017). To obtain further speedups, we next propose a GPU parallel optimization algorithm based on the MIBF, and obtain speedups of up to 44 times with 97% reductions in computations over the serial version. Thus, we recommend the GPU parallel Bernstein algorithm based on the proposed MIBF form for solving polynomial global optimization problems.

A fast optimal parallel algorithm for a short addition chain

Given a natural number e, an addition chain for e is a finite sequence of numbers having the following properties: (1) the first number is one, (2) every element is the sum of two earlier elements, and (3) the given number occurs at the end of the sequence. We introduce a fast optimal algorithm to generate a chain of short length for the number e of n-bits. The algorithm is based on the right–left binary strategy and barrel shifter circuit. The algorithm uses $$O((\frac{n}{\log {n}})^2)$$ processors and runs in $$O((\log {n})^2)$$ time under exclusive read exclusive write parallel random access machine.