Parallel Particle Swarm Optimization Research Articles

Parallel PSO for Efficient Neural Network Training Using GPGPU and Apache Spark in Edge Computing Sets

The training phase of a deep learning neural network (DLNN) is a computationally demanding process, particularly for models comprising multiple layers of intermediate neurons.This paper presents a novel approach to accelerating DLNN training using the particle swarm optimisation (PSO) algorithm, which exploits the GPGPU architecture and the Apache Spark analytics engine for large-scale data processing tasks. PSO is a bio-inspired stochastic optimisation method whose objective is to iteratively enhance the solution to a (usually complex) problem by approximating a given objective. The expensive fitness evaluation and updating of particle positions can be supported more effectively by parallel processing. Nevertheless, the parallelisation of an efficient PSO is not a simple process due to the complexity of the computations performed on the swarm of particles and the iterative execution of the algorithm until a solution close to the objective with minimal error is achieved. In this study, two forms of parallelisation have been developed for the PSO algorithm, both of which are designed for execution in a distributed execution environment. The synchronous parallel PSO implementation guarantees consistency but may result in idle time due to global synchronisation. In contrast, the asynchronous parallel PSO approach reduces the necessity for global synchronization, thereby enhancing execution time and making it more appropriate for large datasets and distributed environments such as Apache Spark. The two variants of PSO have been implemented with the objective of distributing the computational load supported by the algorithm across the different executor nodes of the Spark cluster to effectively achieve coarse-grained parallelism. The result is a significant performance improvement over current sequential variants of PSO.

Optimal capacity configuration and operation strategy of typical industry load with energy storage in fast frequency regulation

As the potential and competent load-side resources for frequency response and control in modern power grids, typical industrial load can compensate for the deficiency of frequency response capability for new-type power systems. However, the operational flexibility is seriously enforced by the operation conditions uncertainties of industrial load. With “Online Calculation, and Real-time Matching” as the core, based on fuzzy mathematical theory, the coordinated operation strategy of typical industrial loads and energy storage systems (ESS) is proposed to finish fast frequency regulation (FFR) tasks. And an optimal capacity configuration model of industrial loads with ESSs is established to evaluate the whole economic profitability in FFR. Specially, in terms of algorithms, a novel Two-layer Parallel Particle Swarm Optimization and Genetic Algorithm (TPPSGA) is innovatively designed in this manuscript. Comparing with traditional heuristic algorithms, TPPSOGA improves computation speed by 6.84 times. Simulation results show that our proposed strategy can reasonably estimate the frequency capability of industrial load by membership degree function, and prove that the industrial load participating in FFR with the support of ESSs is able to get more revenue and stability than industrial load or ESSs separately is in regulation of FFR. The economic analyze in market price and ESS parameters can help the load agent to screen the suitable-type ESS and its capacity to serve for FFR and enhance the flexibility in load-side frequency regulation.

PPSO and Bayesian game for intrusion detection in WSN from a macro perspective

The security of wireless sensor networks is a hot topic in current research. Game theory can provide the optimal selection strategy for attackers and defenders in the attack-defense confrontation. Aiming at the problem of poor generality of previous game models, we propose a generalized Bayesian game model to analyze the intrusion detection of nodes in wireless sensor networks. Because it is difficult to solve the Nash equilibrium of the Bayesian game by the traditional method, a parallel particle swarm optimization is proposed to solve the Nash equilibrium of the Bayesian game and analyze the optimal action of the defender. The simulation results show the superiority of the parallel particle swarm optimization compared with other heuristic algorithms. This algorithm is proved to be effective in finding optimal defense strategy. The influence of the detection rate and false alarm rate of nodes on the profit of defender is analyzed by simulation experiments. Simulation experiments show that the profit of defender decreases as false alarm rate increases and decreases as detection rate decreases. Using heuristic algorithm to solve Nash equilibrium of Bayesian game provides a new method for the research of attack-defense confrontation. Predicting the actions of attacker and defender through the game model can provide ideas for the defender to take active defense.

Implementation and testing of parallel PSO to attain speedup on general purpose computer systems

Implementation and testing of parallel PSO to attain speedup on general purpose computer systems

Simultaneous scheduling of jobs and transporters in a hybrid flow shop with collision-free transporter routing: A novel parallel heuristic

A computationally potent two-stage parallel heuristic (TSPH) and a mixed integer linear programming (MILP) are suggested to address the simultaneous scheduling of jobs and transporters in a hybrid flow shop system, wherein multiple transporters, stage omission, transporter eligibility, machine eligibility, and collision-free transporter routing are assumed. To the best of our knowledge, the significance of transporter collision-free routing has not been spotlighted in the literature on flow shop systems. Collision-free routing of transporters is a requisite technical attribute as transport means may collide on routes and break down the whole system. Our MILP and TSPH assume collision-free routing to impede the issue. Being equipped with parallel computing, TSPH can deal with large problems in a considerably short time. To support, TSPH is analogized against the suggested MILP, two-step MILP (TSMILP), and an efficient parallel meta-heuristic (i.e., parallel particle swarm optimization and genetic algorithm (PPSOGA)) that outperformed many literarily prominent meta-heuristics in the literature. The benchmark results uncover that TSPH outclasses both TSMILP and PPSOGA in the quality of solutions. Eventually, utilizing the convergence plot and Nemenyi’s post-hoc procedure for Friedman’s test, it is revealed that the outcomes of TSPH are remarkably more desirable than those of TSMILP and PPSOGA.

Reconstruction of internal heat source in biological tissue using parallel particle swarm optimization

With the rapid development of engineering thermophysics, researches on human biological heat transfer phenomena has gradually shifted from qualitative to quantitative. The heat distribution information of human lesions is of great value for disease analysis, diagnosis, and treatment. It is a typical inverse problem of heat conduction that deriving the distribution of internal heat sources from the temperature distribution on the body surface. Differing from traditional numerical methods for solving heat conduction, this paper transforms such an inverse problem of bio-heat transfer into a direct one, thereby avoiding complex boundary conditions and regularization processes. To noninvasively reconstruct the internal heat source and its corresponding 3D temperature field in biological tissue, the parallel particle swarm optimization (PPSO) is used in the simulation module, where the position P(x, y, z) of point heat source in biological tissue and its corresponding temperature T are set as the optimization variables. Under a certain optimized sample, one can obtain the simulated temperature distributing on the surface of the module, then subtract the simulated temperature from the measured temperature of the same surface which was measured using a thermal infrared imager. If the absolute value of the difference is smaller, it indicates that the current sample is closer to the true location and temperature of the heat source. When the values of optimization variables are determined, the corresponding 3D temperature field is also confirmed. The simulation results show the simulated position and temperature of the heat source are very approximate with those of the real experimental module. The method presented in this paper has enormous potential and promising prospects in clinical research and application, such as tumor hyperthermia, disease thermal diagnosis technology, etc.

Parallel Hybrid Particle Swarm-Grey Wolf Algorithms for Optimal Load-Shedding in An Isolated Network

In distribution networks integrated with distributed generation (DG), disconnection from the main grid reduces the power supply significantly. The power imbalance between DG generation and load degrades network stability. This paper proposes a hybrid parallel Particle Swarm Optimization - Grey Wolf Optimizer (PSGWO) algorithm for load shedding optimization. This optimization aims to reduce the DG power not absorbed by the remaining loads and maintain the voltage within the specified limits. The performance of PSGWO is tested on an IEEE 33 bus radial distribution system, considering loading levels of 80% to 140% of the baseload. At a 100% loading level, PSGWO showed the best performance, with a load shedding of 2.2297 MW and a voltage deviation of 0.0049. These values are the smallest compared to the results of the standard PSO and GWO algorithms. The PSGWO algorithm remains superior and converges faster than standard PSO and GWO at all loading levels.

Variational Data Assimilation Method Using Parallel Dual Populations Particle Swarm Optimization Algorithm

In recent years, numerical weather forecasting has been increasingly emphasized. Variational data assimilation furnishes precise initial values for numerical forecasting models, constituting an inherently nonlinear optimization challenge. The enormity of the dataset under consideration gives rise to substantial computational burdens, complex modeling, and high hardware requirements. This paper employs the Dual-Population Particle Swarm Optimization (DPSO) algorithm in variational data assimilation to enhance assimilation accuracy. By harnessing parallel computing principles, the paper introduces the Parallel Dual-Population Particle Swarm Optimization (PDPSO) Algorithm to reduce the algorithm processing time. Simulations were carried out using partial differential equations, and comparisons in terms of time and accuracy were made against DPSO, the Dynamic Weight Particle Swarm Algorithm (PSOCIWAC), and the Time-Varying Double Compression Factor Particle Swarm Algorithm (PSOTVCF). Experimental results indicate that the proposed PDPSO outperforms PSOCIWAC and PSOTVCF in convergence accuracy and is comparable to DPSO. Regarding processing time, PDPSO is 40% faster than PSOCIWAC and PSOTVCF and 70% faster than DPSO.

Parallel Particle Swarm Optimization Algorithm for Identifying Complex Communities in Biological Networks

Identification of complex communities in biological networks is a critical and ongoing challenge since lots of network-related problems correspond to the subgraph isomorphism problem known in the literature as NP-hard. Several optimization algorithms have been dedicated and applied to solve this problem. The main challenge regarding the application of optimization algorithms, specifically to handle large-scale complex networks, is their relatively long execution time. Thus, this paper proposes a parallel extension of the PSO algorithm to detect communities in complex biological networks. The main contribution of this study is summarized in three- fold; Firstly, a modified PSO algorithm with a local search operator is proposed to detect complex biological communities with high quality. Secondly, the variability in the capability of PSO to extract community structure in biological networks is studied when different types of crossover operators are used. Finally, to reduce the computational time needed to solve this problem, especially when detecting complex communities in large-scale biological networks, we have implemented parallel computing to execute the algorithm. The performance of the proposed algorithm was tested and evaluated on two real biological networks. The experimental results showed the effective performance of the proposed algorithm when using single-point crossover operator, and its superiority over other counterpart algorithms. Moreover, the use of parallel computing in the proposed algorithm representation has greatly reduced the computational time required for its execution.

Minimizing Fuel Consumption for Surveillance Unmanned Aerial Vehicles Using Parallel Particle Swarm Optimization.

This paper presents a method based on particle swarm optimization (PSO) for optimizing the power settings of unmanned aerial vehicle (UAVs) along a given trajectory in order to minimize fuel consumption and maximize autonomy during surveillance missions. UAVs are widely used in surveillance missions and their autonomy is a key characteristic that contributes to their success. Providing a way to reduce fuel consumption and increase autonomy provides a significant advantage during the mission. The method proposed in this paper included path smoothing techniques in 3D for fixed-wing UAVs based on circular arcs that overfly the waypoints, an essential feature in a surveillance mission. It used the equations of motions and the decomposition of Newton's equation to compute the fuel consumption based on a given power setting. The proposed method used PSO to compute optimized power settings while respecting the absolute physical constraints, such as the load factor, the lift coefficient, the maximum speed and the maximum amount of fuel onboard. Finally, the method was parallelized on a multicore processor to accelerate the computation and provide fast optimization of the power settings in case the trajectory was changed in flight by the operator. Our results showed that the proposed PSO was able to reduce fuel consumption by up to 25% in the trajectories tested and the parallel implementation provided a speedup of 21.67× compared to a sequential implementation on the CPU.

Optimization of G1 Micromixer Structure in Two-Fluid Mixing Based on CFD and Response Surface Methodology

Optimizing the structure of micromixers to improve the mixing efficiency is of great significance for chemical engineering and biology fields. In this study, an optimization of the microchannel in two liquids mixing is carried out based on computational fluid dynamics (CFD) and response surface methodology. Firstly, CFD simulations were performed to investigate the mixing flow field and mixing efficiency in the microchannel by considering different process and structure parameters (e.g., feed pressure p, microchannel width w). The response surface methodology was adopted to construct a fitting surface by CFD discrete working conditions. Then, an optimized microchannel width w was searched using the parallel particle swarm optimization (PPSO) algorithm from the response surface. Lastly, the searched optimum was validated by CFD simulation again, and the final result showed that the predicted mixing efficiency from the response surface model is well confirmed by CFD simulation. On average, the new optimized microchannel width of 1.634 mm performs higher flow flux and mixing efficiency than the original width of 1.5 mm, increasing 13.51% and 2.45%, respectively.

A concurrent design optimization framework for IMSFRP composite structures considering material and structural parameters simultaneously

Injection molded short fiber reinforced polymer (IMSFRP) composite is a widely used lightweight material in the automobile industry. Due to the integrated molding characteristics of IMSFRP composite structures, its lightweight potential can be fully exploited through the simultaneous design of material and structural parameters. However, the efficient material-structure parallel optimization design method for IMSFRP composite structures has not been fully investigated. In this study, a framework of lightweight design containing the concurrent optimization of material and structure parameters is proposed, and it has been successfully applied to an automobile IMSFRP composite liftgate inner. Firstly, considering the skin-core-skin (SCS) structure of the material, a layered model is built up and a parameterized constitutive model is established to efficiently obtain material properties for various material parameters. Then, a parameter extraction and mapping method is proposed to map fiber distribution information to the structural analysis model to improve the accuracy of structural analysis. In addition, based on the Kriging model and a newly developed parallel boundary search particle swarm optimization algorithm, a lightweight design framework of composite liftgate inner is proposed to consider the material and structural design variables simultaneously. The results show that the proposed framework can solve the concurrent optimization design problem effectively and achieve a weight reduction of 10.5 % for the composite liftgate inner under multiple working conditions.

Comparison of Parallel Genetic Algorithm and Particle Swarm Optimization for Parameter Calibration in Hydrological Simulation

ABSTRACT Parameter calibration is an important part of hydrological simulation and affects the final simulation results. In this paper, we introduce heuristic optimization algorithms, genetic algorithm (GA) to cope with the complexity of the parameter calibration problem, and use particle swarm optimization algorithm (PSO) as a comparison. For large-scale hydrological simulations, we use a multilevel parallel parameter calibration framework to make full use of processor resources, and accelerate the process of solving high-dimensional parameter calibration. Further, we test and apply the experiments on domestic supercomputers. The results of parameter calibration with GA and PSO can basically reach the ideal value of 0.65 and above, with PSO achieving a speedup of 58.52 on TianHe-2 supercomputer. The experimental results indicate that using a parallel implementation on multicore CPUs makes high-dimensional parameter calibration in large-scale hydrological simulation possible. Moreover, our comparison of the two algorithms shows that the GA obtains better calibration results, and the PSO has a more pronounced acceleration effect.

Parallel-Computing-Based Calibration for Microscopic Traffic Simulation Model

Microscopic traffic simulation is vital to assess the performances of various traffic operation and management schemes. Microscopic traffic simulation is usually not parameter-free, and it relies on independent parameters to predict traffic evolution. Thus, parameter calibration is indispensable to conveying trustworthy simulation results. Heuristic algorithms are widely used for parameter calibration. Its logic is for achieving iterative optimization through continuous trial-and-error simulations. This process is time-consuming and usually takes several hours, making the calibration unable to meet the requirements of speed and efficiency. In recent years, parallel computing technology has been gradually applied in the engineering realm, which makes rapid calibration possible. Following the three steps of parallel framework selection, algorithm bottleneck identification, and subtask load balancing, this paper designs and implements the parallelization of genetic algorithm and particle swarm optimization (PSO) calibration algorithms. Finally, the proposed parallel framework is applied to simulation parameter calibration of a section of a 5 km long highway in Australia, and the effectiveness of parallel computing is evaluated from the two dimensions of reduction in calibration computational time and scalability. The results show that the proposed parallel calibration algorithm can shorten the 5 h calibration process to less than 1 h, reducing the calibration time by 80%. The parallel PSO calibration algorithm has better scalability, and its acceleration effect is better when more processors are used.

A flexible and efficient knowledge-guided machine learning data assimilation (KGML-DA) framework for agroecosystem prediction in the US Midwest

Process-based models are widely used to predict the agroecosystem dynamics, but such modeled results often contain considerable uncertainty due to the imperfect model structure, biased model parameters, and inaccurate or inaccessible model inputs. Data assimilation (DA) techniques are widely adopted to reduce prediction uncertainty by calibrating model parameters or dynamically updating the model state variables using observations. However, high computational cost, difficulties in mitigating model structural error, and low flexibility in framework development hinder its applications in large-scale agroecosystem predictions. In this study, we addressed these challenges by proposing a novel DA framework that integrates a Knowledge-Guided Machine Learning (KGML)-based surrogate with tensorized ensemble Kalman filter (EnKF) and parallelized particle swarm optimization (PSO) to effectively assimilate historical and in-season multi-source remote sensing data. Specifically, we incorporate knowledge from a process-based model, ecosys, into a Gated Recurrent Unit (GRU)-based hierarchical neural network. The hierarchical architecture of KGML-DA mimics key processes of ecosys and builds a causal relationship between target variables. Using carbon budget quantification in the US Corn-Belt as a context, we evaluated KGML-DA's performance in predicting key processes of the carbon cycle at three agricultural sites (US-Ne1, US-Ne2, US-Ne3), along with county-level (627 counties) and 30-m pixel-level (Champaign County, IL) grain yield. The site experiments show that updating the upstream variable, e.g., gross primary production (GPP), improved the prediction of downstream variables such as ecosystem respiration, net ecosystem exchange, biomass, and leaf area index (LAI), with RMSE reductions ranging from 9.2% to 30.5% for corn and 4.8% to 24.6% for soybean. Uncertainty in downstream variables was automatically constrained after correcting the upstream variables, demonstrating the effectiveness of the causality linkages in the hierarchical surrogate. We found joint use of in-season GPP and evapotranspiration (ET) products along with historical GPP and surveyed yields achieved the best prediction for county-level yields, while assimilating in-season LAI observations benefitted the prediction in extreme years. Uncertainty and error analysis of regional yield estimation demonstrated that KGML-DA could reduce prediction error by 26.5% for corn and 36.2% for soybean. Remarkably, the GPU-based tensor operation design makes this DA framework more than 7000 times faster than the PB model with a High-Performance Computing system, indicating the high potential of the proposed framework for in-season, high-resolution agroecosystem predictions.

Parallelized Particle Swarm Optimization on FPGA for Realtime Ballistic Target Tracking.

This paper addresses the problem of tracking a high-speed ballistic target in real time. Particle swarm optimization (PSO) can be a solution to overcome the motion of the ballistic target and the nonlinearity of the measurement model. However, in general, particle swarm optimization requires a great deal of computation time, so it is difficult to apply to realtime systems. In this paper, we propose a parallelized particle swarm optimization technique using field-programmable gate array (FPGA) to be accelerated for realtime ballistic target tracking. The realtime performance of the proposed method has been tested and analyzed on a well-known heterogeneous processing system with a field-programmable gate array. The proposed parallelized particle swarm optimization was successfully conducted on the heterogeneous processing system and produced similar tracking results. Also, compared to conventional particle swarm optimization, which is based on the only central processing unit, the computation time is significantly reduced by up to 3.89×.

An Improved Parallel Particle Swarm Optimization

In the area of global optimization, a variety of techniques have been developed to find the global minimum. These techniques, in most cases, require a significant amount of computational resources and time to complete and therefore there is a need to develop parallel techniques. In addition, the wide spread of parallel architectures in recent years greatly facilitates the implementation of such techniques. Among the most widely used global optimization techniques is the particle swarm optimization technique. In this work, a series of modifications are proposed in the direction of efficient parallelization for particle swarm optimization. These modifications include an innovative velocity calculation mechanism that has also been successfully used in the serial version of the method, mechanisms for propagating the best particles between parallel computing units, but also a process termination mechanism, which has been properly configured for efficient execution in parallel computing environments. The proposed technique was applied to a multitude of computational problems from the relevant literature and the results were more than promising, since it was found that increasing the computational threads can significantly reduce the required number of function calls to find the global minimum. The proposed technique is at rate of 50–70% of the required number of function calls compared to other optimization techniques. This reduction is visible even if one to two parallel processing units are used. In addition, with the increase in parallel processing units, a drastic reduction in the number of calls is observed and therefore a reduction in the required computing time, which can reach up to 70%.

A parallel particle swarm optimization framework based on a fork-join thread pool using a work-stealing mechanism

Particle Swarm Optimization (PSO) is one of the most popular optimization algorithms that has been adopted in various fields, including design, scheduling, and biochemistry. However, the algorithm is time-consuming when facing high-dimensional or multi-objective optimizing problems. Parallel PSO is thus proposed to improve its computing efficiency, and many studies have been conducted. However, the low-level system design is seldom considered in parallel programming, which may have a nonnegligible impact on computing efficiency, creating a gap between low-level optimization and high-level algorithm design. Therefore, this paper proposes a Parallel Asynchronous PSO (PAPSO) framework based on thread pools utilizing multicore processors and adopts a cross-level approach to bridge the gap. A series of experiments are designed and conducted to examine how the aforementioned method can improve the parallel execution efficiency of PSO compared with the OpenMP framework and nonparallel PSO. Results indicate that PAPSO can significantly improve PSO computing efficiency by reducing the elapsed time, approaching approximately 20% optimization compared with OpenMP. Additionally, it achieves an approximately linear speedup of up to 4.5 threads. In addition, the particle communication experiment shows that the nonblocking communication protocol is effective for maintaining the computing elapsed time in the same level facing different neighborhood sizes. Finally, the work-stealing mechanism achieves an average of 16% improvement for general computing scenarios and maintain up to 16% improvement for imbalanced workload scenarios. Generally, the major contribution of this paper is that we innovate the thread pooling management concept with the fork-join model in parallelizing PSO to make sufficient use of multiple core CPUs for parallel computing through multithread programming.

A parallel particle swarm optimization algorithm based on GPU/CUDA

Parallel computing is the main way to improve the computational efficiency of metaheuristic algorithms for solving high-dimensional, nonlinear optimization problems. Previous studies have typically only implemented local parallelism for the particle swarm optimization (PSO) algorithm. In this study, we proposed a new parallel particle swarm optimization algorithm (GPU-PSO) based on the Graphics Processing Units (GPU) and Compute Unified Device Architecture (CUDA), which uses a combination of coarse-grained parallelism and fine-grained parallelism to achieve global parallelism. In addition, we designed a data structure based on CUDA features and utilized a merged memory access mode to further improve data-parallel processing and data access efficiency. Experimental results show that the algorithm effectively reduces the solution time of PSO for solving high-dimensional, large-scale optimization problems. The speedup ratio increases with the dimensionality of the objective function, where the speedup ratio is up to 2000 times for the high-dimensional Ackley function.

Optimization of Characteristics for a Stochastic Agent-Based Model of Goods Exchange with the Use of Parallel Hybrid Genetic Algorithm

Abstract A novel approach to modeling stochastic processes of goods exchange between multiple agents is presented, considering the possibility of optimizing the environment's characteristics and individual decision-making strategies. The proposed model makes it possible to form optimal states when choosing the moments of concluding barter and monetary transactions at the individual level of each agent maximizing the utility function. A new parallel hybrid Real-Coded Genetic Algorithm and Particle Swarm Optimization (RCGA-PSO) has been developed, combining methods of evolutionary selection based on well-known heuristic operators with methods of swarm optimization and machine learning. The algorithm is characterized by the best time efficiency and accuracy in comparison with other methods. The software implementation of the developed algorithm and model has been performed using the FLAME GPU framework. The possibility of using the RCGA-PSO Algorithm to optimize the characteristics of the environment and strategies for making individual decisions by agents involved in barter and monetary interactions is demonstrated.