Parallel Optimization Research Articles

Thermodynamic analysis and comprehensive system optimization of the near zero emission hybrid power based on SOFC-ICE integrated system fueled with ammonia

The highly efficient heat and mass integrations fueled with ammonia of solid oxide fuel cell and reciprocating piston internal combustion engine are conducted from the viewpoints of characters of restrained / unrestrained chemical reaction and principle of energy cascade utilization. According to previous studies, we classified such integrations into serial and parallel topologies. Based on the detailed and verified models, the power matching limits of serial and parallel integrations are proposed first, showing a narrow scale of power matching of fuel cell and engines, due to the finite engine displacement. Furthermore, based on the design concepts of serial and parallel integrations, the improvement and optimization of combined systems were conducted, which showed that the possibly highest efficiency of serial and parallel systems can reach 70.53% and 68.40%, respectively. Additionally, the exergy analysis of systems could prove that such heat and mass integrations can effectively use remaining heats and fuels, leading to less exergy destructions and optimal designs and operation points achievement, while the maximum exergy destruction occurs at engine. Finally, the environmental analysis results showed that the corrected NO, NO2, N2O and NH3 emissions can reach 0.01 mg/kWh, 5.00 μg/kWh, 29.50 mg/kWh and <5 ppm, achieving near zero emissions.

SI on parallel system and algorithm optimization

SI on parallel system and algorithm optimization

Risk-averse two-stage stochastic programming-based closed-loop supply chain network design under uncertain demand

In recent years, there has been a growing awareness and acceptance of the concept of green environmental protection among the public, primarily due to the worsening environmental issues and increased emphasis by governments on corporate social responsibility regulations. As a result, the management of closed-loop supply chains has gained significant attention from both businesses and researchers. Most stochastic programming models addressing closed-loop supply chain (CLSC) under uncertainty are typically designed to be risk-neutral. In this paper, a risk-averse two-stage stochastic programming (RATSSP) model for the closed-loop supply chain network design (CLSCND) problem considering facility capacity level decisions under uncertain demand. It is aiming to trade off the expected total profit and the profit risk, based on a given degree of responsive risk aversion. Due to the NP-hard nature of the CLSCND problem, this paper applies simplified swarm optimization (SSO), genetic algorithm (GA), and particle swarm optimization (PSO) algorithms to solve the problem. To improve the efficiency and quality of the obtained solutions, we propose a parallel computation meta-heuristic algorithm, parallel simplified swarm optimization (PSSO), along with parallel versions of PSO and GA. Computational results show the effectiveness of our proposed model, with PSSO demonstrating the best robustness and solution quality among the considered algorithms.

AdaptMVSNet: Efficient Multi-View Stereo with adaptive convolution and attention fusion

Multi-View Stereo (MVS) is a crucial technique for reconstructing the geometric structure of a scene, given the known camera parameters. Previous deep learning-based MVS methods have mainly focused on improving the reconstruction quality but overlooked the running efficiency during the actual algorithm deployment. For example, deformable convolutions have been introduced to improve the accuracy of the reconstruction results further, however, its inability for parallel optimization caused low inference speed. In this paper, we propose AdaptMVSNet which is device-friendly and reconstruction-efficient, while preserving the original results. To this end, adaptive convolution is introduced to significantly improve the efficiency in speed and metrics compared to current methods. In addition, an attention fusion module is proposed to blend features from adaptive convolution and the feature pyramid network. Our experiments demonstrate that our proposed approach achieves state-of-the-art performance and is almost 2× faster than the recent fastest MVS method. We will release our source code.

The disjoint path cover in the data center network HSDC with prescribed vertices in each path

The n-dimensional HSDC with the logic graph Hn is one of the most attractive server-centric data center networks for high incremental scalability. The routing design in network topology is very important for information transmission. In particular, the application of disjoint path covers can solve various problems such as program code optimization and mapping parallel programs to parallel structures. Hn consists of 2n cliques and each two vertices in a clique are adjacent. For any vertex sets {x,u} and {y,v} in W and B, respectively, and any two disjoint vertex sets A1,A2 in some clique, where (W,B) is a bipartition of Hn, in this paper, we prove that, when |Ai|≠1 and Ai∩{x,y,u,v}=∅ for i=1,2, if |A1∪A2|≤n−3, then there exists an (x,y)-path P1 and a (u,v)-path P2 in Hn passing through A1 and A2, respectively, such that P1 and P2 have no common vertices and P1∪P2 contains all vertices of Hn. Then P1∪P2 is called the 2-disjoint path cover in Hn and we propose the efficient algorithms for the path design. Furthermore, we prove that our main result is optimal.

Massively Parallel Monte Carlo Sampling for Xinanjiang Hydrological Model Parameter Optimization Using CPU-GPU Computer Cluster

The Monte Carlo sampling (MCS) method is a simple and practical way for hydrological model parameter optimization. The MCS procedure is used to generate a large number of data points. Therefore, its computational efficiency is a key issue when applied to large-scale problems. The MCS method is an internally concurrent algorithm that can be parallelized. It has the potential to execute on massively parallel hardware systems such as multi-node computer clusters equipped with multiple CPUs and GPUs, which are known as heterogeneous hardware systems. To take advantage of this, we parallelize the algorithm and implement it on a multi-node computer cluster that hosts multiple INTEL multi-core CPUs and NVIDIA many-core GPUs by using C++ programming language combined with the MPI, OpenMP, and CUDA parallel programming libraries. The parallel parameter optimization method is coupled with the Xinanjiang hydrological model to test the acceleration efficiency when tackling real-world applications that have a very high computational burden. Numerical experiments indicate, on the one hand, that the computational efficiency of the massively parallel parameter optimization method is significantly improved compared to single-core CPU code, and the multi-GPU code achieves the fastest speed. On the other hand, the scalability property of the proposed method is also satisfactory. In addition, the correctness of the proposed method is also tested using sensitivity and uncertainty analysis of the model parameters. Study results indicate good acceleration efficiency and reliable correctness of the proposed parallel optimization methods, which demonstrates excellent prospects in practical applications.

Parallel transient response topology optimization based on PETSc

Improving the dynamic properties and reducing the vibration levels is extremely important for the design of many engineering structures. In this paper, we develop a dynamic topology optimization program based on parallel computing with the help of the PETSc Portable Extension Toolkit for Scientific Computing and the MPI communication protocol. The SIMP model is used as the topology optimization model, and the sensitivity analysis schemes of the dynamic compliance and displacement response as opposed to the density design variables are given in the paper. Using this optimization formulation, we developed a parallel topology optimization program for minimizing the transient response. The convergence performance of the program is shown with a relatively large-scale numerical example.

Parallel Optimization and Performance Tuning on a Kunpeng Cluster of Genetic Algorithm for Synthesis of Circulant Networks

Parallel Optimization and Performance Tuning on a Kunpeng Cluster of Genetic Algorithm for Synthesis of Circulant Networks

Falcon: Fair and Efficient Online File Transfer Optimization

Research networks provide high-speed wide-area network connectivity between research and education institutions to facilitate large-scale data transfers. However, scalability issues of legacy transfer applications such as scp and FTP hinder the effective utilization of these networks. Although researchers extended the legacy transfer applications to increase their performance by exploiting I/O and network parallelism, these solutions necessitate users to fine-tune parallelism level, a task that is challenging even for experienced users due to the dynamic nature of networks. In this paper, we propose an online optimization algorithm, Falcon, to tune the degree of parallelism for file transfers to maximize transfer throughput while keeping system overhead at a minimum. As research networks are shared infrastructures, we introduce a game theory-inspired novel utility function to evaluate the performance of various parallelism levels such that competing transfers are guaranteed to converge to a fair and stable solution. We assessed the performance of Falcon in isolated and production high-speed networks and found that it can discover optimal transfer parallelism in as little as 20 seconds and outperform the state-of-the-art solutions by more than <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$2\times$</tex-math></inline-formula> . Moreover, Falcon is guaranteed to converge to Nash Equilibrium when multiple transfers compete for the same resources with the help of its game theory-inspired utility function. Finally, we demonstrate that Falcon can also be used as a central transfer scheduler to speed up convergence time, increase stability, and enforce system/user-level resource limitations in shared networks.

Learning Causal Biological Networks with Parallel Ant Colony Optimization Algorithm.

A wealth of causal relationships exists in biological systems, both causal brain networks and causal protein signaling networks are very classical causal biological networks (CBNs). Learning CBNs from biological signal data reliably is a critical problem today. However, most of the existing methods are not excellent enough in terms of accuracy and time performance, and tend to fall into local optima because they do not take full advantage of global information. In this paper, we propose a parallel ant colony optimization algorithm to learn causal biological networks from biological signal data, called PACO. Specifically, PACO first maps the construction of CBNs to ants, then searches for CBNs in parallel by simulating multiple groups of ants foraging, and finally obtains the optimal CBN through pheromone fusion and CBNs fusion between different ant colonies. Extensive experimental results on simulation data sets as well as two real-world data sets, the fMRI signal data set and the Single-cell data set, show that PACO can accurately and efficiently learn CBNs from biological signal data.

Parallel optimization method of train scheduling and shunting at complex high‐speed railway stations

AbstractThe train operations of large stations are critical in determining the efficiency of the railway network. Large high‐speed railway stations often have more than two adjacent stations running in multiple directions and must address highly complex train operation patterns. To develop a more efficient operation plan than the existing route‐based representations for modeling train conflicts, a more systematic track‐based resource network (TRN) representation for a railway station that simulates fixed equipment is used in this paper. Based on this high‐fidelity TRN, an improved operation scheduling model (OSM) is introduced for the integrated scheduling of train operations, including shunting operations. Besides, two approaches based on Lagrangian relaxation (LR) and the Alternating Direction Method of Multipliers (ADMM) are described in detail to address the operation scheduling problem (OSP) for stations, and an efficient heuristic rule‐based algorithm is also designed to generate lower bound solutions. To improve the algorithm efficiency, we designed a parallelization strategy for the proposed approaches. Based on real‐world case studies, the computing results demonstrate that the parallelized ADMM‐based approach can always find high‐quality feasible solutions for complex stations and generate conflict‐free station operation plans with significantly improved efficiency.

Hydrogen as a Renewable Energy Carrier in a Hybrid Configuration of Distributed Energy Systems: Bibliometric Mapping of Current Knowledge and Strategies

Storing energy in hydrogen deposits balances the operation of energy systems and is an effective tool in the process of energy transformation towards achieving Sustainable Development Goals. To assess the validity of its use as an alternative renewable energy carrier in dispersed energy systems of hybrid configuration, a comprehensive review of scientific literature was conducted in this study, based on bibliometric analysis. The bibliographic database used in the study was the international Web of Science database. This review contributes to a better understanding of the characteristics of the selected research area. The evolution of research trends implemented in the design of energy systems associated with hydrogen technologies is revealed, clearly indicating that it is a developing field. In recent years, there has been an increase in the number of publications, although the territorial range of research (mainly simulation) conducted in the domain does not include areas with the most favourable infrastructural conditions. The analysis reveals weak cooperation between South American, African, East Asian, and Oceanic countries. In the light of earlier, thematically similar literature reviews, several research gaps are also identified and proposals for future research are presented. They concern, in particular, the parallel implementation and optimization of the operation of hydrogen (HRES—Hybrid Renewable Energy System and HESS—Hybrid Energy Storage System) solutions in terms of economics, ecology, lifespan, and work efficiency, as well as their feasibility analysis. With the support of other researchers and those involved in the subject matter, this review may contribute to the further development of hybrid hydrogen systems in terms of increasing competitiveness and promoting the implementation of these technologies.

Exploring synchronization and lift suppression in fluid flow around vibrating cylinder: a parallel CFD and global optimization investigation

In this study, we employ a parallel Computational Fluid Dynamics (CFD) code integrated with the VTDIRECT95 algorithm, a parallel deterministic global optimization method, to conduct global optimization for an oscillating circular cylinder. We conduct numerical simulations for the flow at a Reynolds number of 500 within the parameter range of 0.1≤Ax≤0.3 and 0.5fst≤fex≤2.5fst where Ax represents the inline oscillating amplitude, fex denotes the forcing oscillation frequency, and fst corresponds to the frequency of a stationary cylinder. To enhance computational efficiency, a combination of VTdirect and a CFD solver is utilized to efficiently identify the synchronization region, thereby reducing computational resources. The results reveal a significant reduction in the lift coefficient within the synchronized region compared to unsynchronized regimes. Furthermore, the study delves into the underlying flow physics behind synchronization and lift suppression. By synchronizing the shedding of vortices, their detrimental effects are nullified, resulting in a reduction in lift. Moreover, the research examines the influence of three-dimensional (3-D) flow by comparing 2-D and 3-D simulations at two different Reynolds numbers. It demonstrates that accounting for 3-D effects yields more accurate predictions of fluid behavior. Synchronization maps and root mean square (rms) lift coefficient plots illustrate the impact of Reynolds number and movement frequency on lift suppression. The findings indicate that achieving synchronization in 3-D flow necessitates stronger amplitudes and higher frequencies. At higher Reynolds numbers, the wake structures become unstable, leading to intricate vortical patterns. Consequently, the synchronization curve shifts towards higher amplitudes and frequencies in 3-D simulations. Understanding these phenomena is vital for reducing lift force in practical applications. This research significantly contributes to knowledge regarding synchronization and lift suppression in fluid flow around vibrating cylinders.

A new parallel bat colony optimization algorithm and its application for evolving CNN architectures with artificial selection

A new parallel bat colony optimization algorithm and its application for evolving CNN architectures with artificial selection

Automatic Pipeline Parallelism: A Parallel Inference Framework for Deep Learning Applications in 6G Mobile Communication Systems

With the rapid development of wireless communication, achieving the neXt generation Ultra-Reliable and Low-Latency Communications (xURLLC) in 6G mobile communication systems has become a critical problem. Among many applications in xURLLC, deep learning model inference requires improvement over its efficiency. Due to the heterogeneous hardware environment in 6G, parallel schedules from distributed machine learning and edge computing has been borrowed to tackle the efficiency problem. However, traditional parallel schedules suffer from high latency, low throughput, and low device utility. In this paper, we propose Automatic Pipeline Parallelism ( <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">AP</i> <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> ), a parallel inference framework for deep learning applications in 6G mobile communication systems, to improve the model inference efficiency while maintaining reliability. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">AP</i> <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> contains three sub-modules. A task-device affinity predictor predicts a task’s expected execution time on a given device. The parallel inference arrangement optimizer finds the most suitable device for each task. The parallel inference scheduler converts the arrangement to a schedule that can be directly executed in the system. The experimental results show that <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">AP</i> <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> can achieve better latency, throughput, reliability, and device utility than other parallel schedules. Also, the priority of the sub-module designs has been approved through the experiments.

Raman spectral pattern recognition of breast cancer: A machine learning strategy based on feature fusion and adaptive hyperparameter optimization

Raman spectroscopy, as a kind of molecular vibration spectroscopy, provides abundant information for measuring components and molecular structure in the early detection and diagnosis of breast cancer. Currently, portable Raman spectrometers have simplified and made equipment application more affordable, albeit at the cost of sacrificing the signal-to-noise ratio (SNR). Consequently, this necessitates a higher recognition rate from pattern recognition algorithms. Our study employs a feature fusion strategy to reduce the dimensionality of high-dimensional Raman spectra and enhance the discriminative information between normal tissues and tumors. In the conducted random experiment, the classifier achieved a performance of over 96% for all three average metrics: accuracy, sensitivity, and specificity. Additionally, we propose a multi-parameter serial encoding evolutionary algorithm (MSEA) and integrate it into the Adaptive Local Hyperplane K-nearest Neighbor classification algorithm (ALHK) for adaptive hyperparameter optimization. The implementation of serial encoding tackles the predicament of parallel optimization in multi-hyperparameter vector problems. To bolster the convergence of the optimization algorithm towards a global optimal solution, an exponential viability function is devised for nonlinear processing. Moreover, an improved elitist strategy is employed for individual selection, effectively eliminating the influence of probability factors on the robustness of the optimization algorithm. This study further optimizes the hyperparameter space through sensitivity analysis of hyperparameters and cross-validation experiments, leading to superior performance compared to the ALHK algorithm with manual hyperparameter configuration.

Multitask Particle Swarm Optimization With Dynamic Transformation

Multitask optimization (MTO) mainly utilizes knowledge transfer among tasks to address multiple optimization problems in parallel. However, the decision space dimensions of different tasks often differ, which leads to the failure of knowledge transfer. Therefore, it is a challenging problem to transfer knowledge among tasks with different dimensions to achieve parallel optimization of multiple tasks. To address this problem, multitask particle swarm optimization with a dynamic transformation strategy (MTPSO-DTS) is proposed to improve the performance of MTO. First, an intertask similarity index based on gradient information and location information is designed to dynamically assess the degree of similarity among different tasks. Then, the intertask similarity is determined to assist the transformation of dimensions. Second, a dynamic transformation strategy based on intertask similarity is developed to achieve a uniform representation of different dimension knowledge, including dimension supplementation and dimension reduction. Then, the MTPSO-DTS algorithm, which takes advantage of the search characteristics of particle swarm optimization, can facilitate knowledge transfer among tasks with different dimensions. Third, the convergence of the MTPSO-DTS algorithm is analyzed to verify the validity theoretically. Finally, numerical comparison experiments on benchmark problems and a practical application are carried out to demonstrate the effectiveness of the MTPSO-DTS algorithm. The results indicate that the proposed MTPSO-DTS algorithm can facilitate knowledge transfer among tasks of different dimensions to promote parallel optimization of multiple tasks.

Full-Stack Optimizing Transformer Inference on ARM Many-Core CPU

The past several years have witnessed tremendous success of transformer models in natural language processing (NLP), and their current landscape is increasingly diverse. Although GPU gradually becomes the dominating workhorse and de facto standard for deep learning, there are still many scenarios where using CPU remains a prevalent choice. Recently, ARM many-core processor starts emigrating to cloud computing and high-performance computing, which is promising to deploy transformer inference. In this paper, we identify several performance bottlenecks of existing inference runtime on many-core CPU including low-core usage, isolated thread configuration, inappropriate implementation of general matrix multiply (GEMM), and redundant computations for variable-length inputs. To tackle these problems, full-stack and cross-layer optimizations are conducted for these challenges from deep learning service level to the neural network operator level. We explore multi-instance parallelization at the service level to improve CPU core usage. To improve parallel efficiency of the inference runtime, we design NUMA-aware thread scheduling and a look-up table for optimal parallel configurations. The GEMM implementation is tailored for some critical modules such self-attention module to exploit the characteristics of transformer workload. To eliminate redundant computations, a novel storage format is designed and implemented to pack sparse data and a load balancing strategy is proposed for tasks with different sparsity. Experimental results show that our implementation can outperform existing solutions by 1.1x to 6x for different transformer-based models with fixed-length inputs. For variable-length inputs, it achieves 1.9x to 6x speedups on Kungpeng920 processor and 3x to 8x speedups on Ampere Altra processor depending on different sequence lengths and batch sizes.

Parallel algorithm design and optimization of geodynamic numerical simulation application on the Tianhe new-generation high-performance computer

Parallel algorithm design and optimization of geodynamic numerical simulation application on the Tianhe new-generation high-performance computer

A parallel particle swarm optimization framework based on a fork-join thread pool using a work-stealing mechanism

Particle Swarm Optimization (PSO) is one of the most popular optimization algorithms that has been adopted in various fields, including design, scheduling, and biochemistry. However, the algorithm is time-consuming when facing high-dimensional or multi-objective optimizing problems. Parallel PSO is thus proposed to improve its computing efficiency, and many studies have been conducted. However, the low-level system design is seldom considered in parallel programming, which may have a nonnegligible impact on computing efficiency, creating a gap between low-level optimization and high-level algorithm design. Therefore, this paper proposes a Parallel Asynchronous PSO (PAPSO) framework based on thread pools utilizing multicore processors and adopts a cross-level approach to bridge the gap. A series of experiments are designed and conducted to examine how the aforementioned method can improve the parallel execution efficiency of PSO compared with the OpenMP framework and nonparallel PSO. Results indicate that PAPSO can significantly improve PSO computing efficiency by reducing the elapsed time, approaching approximately 20% optimization compared with OpenMP. Additionally, it achieves an approximately linear speedup of up to 4.5 threads. In addition, the particle communication experiment shows that the nonblocking communication protocol is effective for maintaining the computing elapsed time in the same level facing different neighborhood sizes. Finally, the work-stealing mechanism achieves an average of 16% improvement for general computing scenarios and maintain up to 16% improvement for imbalanced workload scenarios. Generally, the major contribution of this paper is that we innovate the thread pooling management concept with the fork-join model in parallelizing PSO to make sufficient use of multiple core CPUs for parallel computing through multithread programming.