Asynchronous Parallel Research Articles

Double-Barrel Nitinol Stent Placement for Iliocaval Reconstruction: The Effect of Deployment Sequence and Direction on Final Configuration

Double-barrel stent placement across the iliocaval confluence is commonly used for the treatment of chronic bilateral iliocaval occlusion. The difference in the deployment outcomes of synchronous parallel stent deployment versus asynchronous or antiparallel deployment and the underlying stent interactions are poorly understood. In this study, 3 strategies of double-barrel nitinol self-expanding stent deployment across the iliocaval confluence (synchronous parallel, asynchronous parallel, and synchronous antiparallel) were contrasted in vivo in 3 swine followed by assessment of the explanted stent construct. Synchronous parallel stent deployment achieved a desired double-barrel configuration. The asynchronous parallel and antiparallel deployment strategies both resulted in a crushed stent despite subsequent simultaneous balloon angioplasty. These animal model results suggested that in patients who undergo double-barrel iliocaval reconstruction, synchronous parallel stent deployment may provide the desired stent conformation and increase the chance for clinical success.

Load balanced locality-aware parallel SGD on multicore architectures for latent factor based collaborative filtering

We investigate the parallelization of Stochastic Gradient Descent (SGD) for matrix completion on multicore architectures. We provide an experimental analysis of current SGD algorithms to find out their bottlenecks and limitations. Grid-based methods suffer from load imbalance among 2D blocks of the rating matrix, especially when datasets are skewed and sparse. Asynchronous methods, on the other hand, can face cache issues due to their memory access pattern. We propose bin-packing-based block balancing methods that are alternative to the recently proposed BaPa method. We then introduce Locality Aware SGD (LASGD), a grid-based asynchronous parallel SGD algorithm that efficiently utilizes cache by changing nonzero update sequence without affecting factor update order and carefully arranging latent factor matrices in the memory. Combined with our proposed load balancing methods, our experiments show that LASGD performs significantly better than alternative approaches in parallel shared-memory systems.

An asynchronous parallel high-throughput model calibration framework for crystal plasticity finite element constitutive models

An asynchronous parallel high-throughput model calibration framework for crystal plasticity finite element constitutive models

Integrating complete disassembly planning with deconstructability assessment to facilitate designing deconstructable buildings

ABSTRACT As one of the most resource-intensive sectors, the building industry has a significant role in environmental degradation and global emissions. Deconstruction can highly enhance resource efficiency by closing materials loops and enabling a circular economy model as a sustainable and environmentally friendly paradigm. This study presents an integrated method for complete disassembly planning and deconstructability assessment (DA) as a novel strategy to facilitate designing deconstructable buildings. The asynchronous parallel disassembly planning (aPDP) approach from the manufacturing industry is utilized to develop a complete disassembly planning method for building assemblies. This is the first study on the complete disassembly planning of buildings in which both sequential and parallel modes are supported. Moreover, a new DA approach based on the environmental impact (i.e. the global warming potential) of components is devised to show the deconstructability percentage. The method can create valid and feasible complete disassembly plans, and the higher complexity does not decrease its performance. Besides, the DA approach is able to raise environmental awareness and draw more attention to the profound implications of the material selection procedure. The method's provided information can assist decision-makers in designing buildings with a higher deconstructability degree by enriching the deconstruction plan, reducing the lack of disassembly information, and enhancing decisions’ quality in DA.

Enhancing Evolutionary Algorithm Performance with Knowledge Transfer and Asynchronous Parallelism

This two-part PhD thesis (by Eric O. Scott, Advisors: Kenneth A. De Jong and Sean Luke) investigates asynchronous steady-state evolutionary algorithms and evolutionary knowledge transfer, respectively, as ways to address challenges that can make it difficult to apply evolutionary algorithms (EAs) to computationally intensive applications. A full copy of the dissertation can be found via ProQuest.

Numerical study on the water-entry characteristics of asynchronous parallel projectiles at an oblique impact angle

This study investigates the water-entry problems of asynchronous parallel projectiles with oblique entry speeds of 200 m/s for different water-entry intervals and two different water-entry sequences numerically. The experiment of the water-entry of the projectile with an oblique impact angle is conducted to verify the numerical method. The influences of the oblique asynchronous parallel water-entry on the evolutions of the cavitating flow and the movement characteristics under different working conditions are studied in detail. It can be found that for the minimum interval, the first launch projectile is not able to generate a complete cavity to encapsulate its body due to the intense compression of the cavity of the following launch projectile, and its movement is seriously affected and ultimately destabilized. Simultaneously, compared with the first launch projectile entering the water from the side close to the initial free surface, the cavity of the first launch projectile entering the water from the side away from the initial free surface is squeezed more, wetted earlier and destabilized faster. With the increase of the interval, the influence of the following launch projectile on the cavity evolution and movement stability of the first launch projectile weakens. Besides, for the following launch projectile, due to the continuous impact of the cavitating flow of the first launch projectile, the profile of the cavity of the following launch projectile presents asymmetry. For the minimum interval, the projectile deflects faster and towards the inner side of the projectile, while for the larger intervals, the projectile deflects to the outer side of the projectile and becomes unstable.

ANFIS multi-tasking algorithm implementation scheme for ball-on-plate system stabilization

This paper presents the design and realization of a ball-on-plate system using a 3-degree-of-freedom parallel robot controlled by an adaptive neuro-fuzzy in-ference system. The ball-on-plate system is nonlinear, multivariable, with an under-actuated feature. Initially, the parallel robot is designed using SolidWorks and mechanized using a computer numerical control machine. Followed by the presentation of the ball-on-plate system mathematical model and the simplified model obtained. Afterwards, the inverse kinematics are performed to derive the appropriate angle for each servomotor. Eventually, the controller is designed and implemented in a double loop feedback scheme. A comparison between the proposed controller and a conventional proportional–integral–derivative controller in terms of time response, overshoot, and steady-state error is carried out. Furthermore, a comparison between sequential and asynchronous parallel processing is conducted for two different scenarios. The first scenario is when moving the ball to the origin while the second is for disturbance rejection. Simulation and experimental results show that the adaptive neuro-fuzzy inference system implemented using asynchronous parallel processing improves the real-time system stability by considerably decreasing oscillations as well as enhancing the ball movement smoothness with a small stead-state error.

On the convergence analysis of asynchronous SGD for solving consistent linear systems

Parallel and distributed stochastic algorithms have drawn significant attention in the realm of big data and machine learning. While the synchronous versions of these algorithms are well understood in terms of their convergence, the convergence analyses of their asynchronous counterparts are not widely studied. In this paper, we propose and analyze a distributed, asynchronous parallel algorithm to solve an arbitrary, consistent linear system by reformulating the system into a stochastic optimization problem as Richtárik and Takác̃ in [1]. We compare the convergence rates of our asynchronous algorithm with the synchronous parallel algorithm proposed by Richtárik and Takáč in [1] under different choices of the hyperparameters—the stepsize, the damping factor, the number of processors, and the delay factor. We show that our asynchronous parallel algorithm enjoys a global linear convergence rate, similar to the synchronous parallel algorithm in [1] under the same setup. We also show that our asynchronous algorithm improves upon the synchronous algorithm in [1] with a better convergence rate when the number of processors is larger than four. Furthermore, our asynchronous parallel algorithm performs asymptotically better than its synchronous counterpart for certain linear systems. Finally, we compute the minimum number of processors required for those systems for which our asynchronous algorithm is better and find that this number can be as low as two for some ill-conditioned problems.

A frequency-based parent selection for reducing the effect of evaluation time bias in asynchronous parallel multi-objective evolutionary algorithms

Parallel evolutionary algorithms (PEAs) have been studied for reducing the execution time of evolutionary algorithms by utilizing parallel computing. An asynchronous PEA (APEA) is a scheme of PEAs that increases computational efficiency by generating a new solution immediately after a solution evaluation completes without the idling time of computing nodes. However, because APEA gives more search opportunities to solutions with shorter evaluation times, the evaluation time bias of solutions negatively affects the search performance. To overcome this drawback, this paper proposes a new parent selection method to reduce the effect of evaluation time bias in APEAs. The proposed method considers the search frequency of solutions and selects the parent solutions so that the search progress in the population is uniform regardless of the evaluation time bias. This paper conducts experiments on multi-objective optimization problems that simulate the evaluation time bias. The experiments use NSGA-III, a well-known multi-objective evolutionary algorithm, and compare the proposed method with the conventional synchronous/asynchronous parallelization. The experimental results reveal that the proposed method can reduce the effect of the evaluation time bias while reducing the computing time of the parallel NSGA-III.

A two-stage interactive evolutionary algorithm for multi-objective asynchronous parallel selective disassembly sequence planning problem

The existing studies of selective disassembly sequence planning (SDSP) focus on sequential SDSP, which is inefficient especially when disassembling complex products. Moreover, energy efficiency has not received much attention in SDSP, which is the critical manifestation of environmental impact. Thus, this work focuses on parallel SDSP and further proposes a profit-oriented and energy-efficient multi-objective asynchronous parallel SDSP (PEAPSDSP) for optimizing disassembly time, energy consumption, and disassembly profit simultaneously. In PEAPSDSP, multiple components can be removed simultaneously and without synchronization requirement. Then, a two-stage interactive multi-objective evolutionary algorithm (TS-IMOEA) is proposed to address PEAPSDSP. The potential non-dominated solutions (NDSs) are obtained in first stage, and a novel search strategy is designed to further improve the quality of NDSs and population quality in second stage. Numerical results in three different scale cases demonstrate the superiority of TS-IMOEA in solving PEAPSDSP, and the benefit of considering selective disassembly in value recovery.

Efficient Gradient Support Pursuit With Less Hard Thresholding for Cardinality-Constrained Learning.

Recently, stochastic hard thresholding (HT) optimization methods [e.g., stochastic variance reduced gradient hard thresholding (SVRGHT)] are becoming more attractive for solving large-scale sparsity/rank-constrained problems. However, they have much higher HT oracle complexities, especially for high-dimensional data or large-scale matrices. To address this issue and inspired by the well-known Gradient Support Pursuit (GraSP) method, this article proposes a new Relaxed Gradient Support Pursuit (RGraSP) framework. Unlike GraSP, RGraSP only requires to yield an approximation solution at each iteration. Based on the property of RGraSP, we also present an efficient stochastic variance reduction-gradient support pursuit algorithm and its fast version (called stochastic variance reduced gradient support pursuit (SVRGSP+). We prove that the gradient oracle complexity of both our algorithms is two times less than that of SVRGHT. In particular, their HT complexity is about κ∧s times less than that of SVRGHT, where κ∧s is the restricted condition number. Moreover, we prove that our algorithms enjoy fast linear convergence to an approximately global optimum, and also present an asynchronous parallel variant to deal with very high-dimensional and sparse data. Experimental results on both synthetic and real-world datasets show that our algorithms yield superior results than the state-of-the-art gradient HT methods.

Multi-Rate Parallel Real-Time Simulation Method for Doubly Fed Wind Power Systems Based on FPGA–CPU

A multi-rate parallel real-time simulation method based on FPGA–CPU is studied to realize the asynchronous co-simulation of the converter of doubly fed wind power systems with the wind turbine and external power grid. The doubly fed wind power system is partitioned by simulation step length, and the partitioned small-step-length data are processed using integral homogenization. For large-step data, an improved delay-compensated linear interpolation method combined with Newton interpolation is proposed for processing. The general small time-step (GST) model method is used to implement the FPGA modeling of the small-step converter, and resource optimization is achieved through timing time-division multiplexing. Asynchronous parallel co-simulation of a doubly fed wind power system is implemented on an FPGA–CPU co-simulation platform. Among them, the FPGA realizes the development of the converter HDL with a small step of 1 μs, while the CPU completes the simulation of the wind turbine and power grid synchronously with a large step of 50 μs. Finally, by comparing with MATLAB/Simulink offline simulation and analyzing the error, it is concluded that the simulation accuracy of the improved method in this paper is higher than that of the un-interpolated parallel simulation, which verifies the real-time performance and accuracy of the modeling and improved method in this paper.

Comparison of parallel optimization algorithms on computationally expensive groundwater remediation designs

Contaminated groundwater resources threaten human health and destroy ecosystems in many areas worldwide. Groundwater remediation is crucial for environmental recovery; however, it can be cost prohibitive. Planning a cost-effective remediation design can take a long time, as it may involve the evaluation of many management decisions, and the corresponding simulation models are computationally demanding. Parallel optimization can facilitate much faster management decisions for cost-effective groundwater remediation design using complex pollutant transport models. However, the efficiency of different parallel optimization algorithms varies depending on both the search strategy and parallelism. In this paper, we show the performance of a parallel surrogate-based optimization algorithm called parallel stochastic radial basis function (p-SRBF), which has not been previously used on contaminant remediation problems. The two case problems involve two superfund sites (i.e., the Umatilla Aquifer and Blaine Aquifer), and one objective evaluation takes 5 and 30 min for the two problems, respectively. Exceptional speedup (superlinear) is achieved with 4 to 16 cores, and excellent speedup is achieved using up to 64 cores, obtaining a good solution at 80 % efficiency. We compare our p-SRBF with three different parallel derivative-free optimization algorithms, including genetic algorithm, mesh adaptive direct search, and asynchronous parallel pattern search optimization, in terms of objective quality, computational reduction and robust behavior across multiple trials. p-SRBF outperforms the other algorithms, as it finds the best solution in both the Umatilla and Blaine cases and reduces the computational budget by at least 50 % in both problems. Additionally, statistical comparisons show that the p-SRBF results are better than those of the alternative algorithms at the 5 % significant level. This study enriches theoretical real-world groundwater remediation methods. The results demonstrate that p-SRBF is promising for environmental management problems that involve computationally expensive models.

Mathematical Formulations for Asynchronous Parallel Disassembly Planning of End-of-Life Products

Disassembly is one of the most time-consuming and labor-intensive activities during the value recovery of end-of-life (EOL) products. The completion time (makespan) of disassembling EOL products is highly associated with the allocation of operators, especially in parallel disassembly. In this paper, asynchronous parallel disassembly planning (APDP), which avoids the necessity to synchronize disassembly tasks of manipulators during the parallel disassembly process, is studied to optimize the task assignment of manipulators for minimal makespan. We utilize four mixed integer linear programming (MILP) formulations to identify the optimal solutions. A set of different-sized instances are used to test and compare the performance of the proposed models, including some real-world cases. Finally, the proposed exact algorithm is further compared with the existing approach to solving APDP. Results indicate that a significant difference exists in terms of the computational efficiency of the MILP models, while three of four MILP formulations can efficiently achieve better solutions than that of the existing approach.

An asynchronous parallel explicit solver based on scaled boundary finite element method using octree meshes

Explicit time integration methods are an integral part of solving structural dynamics problems such as the propagation of elastic and acoustic waves, impact scenarios including crash tests and many more. One common limitation of this class of time integrators is, however, their conditional stability, meaning that highly distorted, small, or stiff finite elements typically govern the critical time step size. That is to say, even a small number of such elements will result in a significantly decrease of the feasible time step, which in turn drastically increases the computational costs of solving the semi-discrete equations of motion. This is especially true for non-uniform and unstructured meshes. Therefore, an asynchronous explicit solver in parallel is proposed in this article. The idea is to assign different time step sizes to different parts of the mesh, such that the overall computational effort is minimized. To improve the performance of the proposed solver, it is combined with a sophisticated octree meshing framework, where the balanced octrees are used, meaning that the ratio of the element sizes of adjacent elements cannot exceed a value of two. Thus, the critical time step of each element can be estimated straightforwardly. To avoid issues with hanging nodes, a special polyhedral element formulation, the scaled boundary finite element method, is employed. Since there are only a limited number of cell patterns in a balanced octree, the stiffness and mass matrices are pre-computed, significantly reducing the computational cost. Moreover, exploiting an element-by-element technique, the assembly of global stiffness matrices can be avoided. The main advantage of the described methodology is that the asynchronous explicit solver can be easily implemented in a high-performance computing environment. By means of several numerical examples, the accuracy and efficiency of the proposed method are demonstrated, as well as its versatility in handling complex engineering problems.

Adaptive synchronous strategy for distributed machine learning

In distributed machine learning training, bulk synchronous parallel (BSP) and asynchronous parallel (ASP) are two main synchronization methods to help achieve gradient aggregation. However, BSP needs longer training time due to “stragglers” problem, while ASP sacrifices the accuracy due to “gradient staleness” problem. In this article, we propose a distributed training paradigm on parameter server framework called adaptive synchronous strategy (A2S) which improves the BSP and ASP paradigms by adaptively adopting different parallel training schemes for workers with different training speeds. Based on the stale value between the fastest and slowest worker, A2S adaptively adds a relaxed synchronous barrier for fast workers to alleviate gradient staleness, where a differentiated weighting gradient aggregation method is used to reduce the impact of slow gradients. Simultaneously, A2S adopts ASP training for slow workers to eliminate stragglers. Hence, A2S not only improves the “gradient staleness” and “stragglers” problems, but also obtains convergence stability and synchronous gain through synchronous and asynchronous parallel, respectively. Specially, we theoretically proved the convergence of A2S by deriving the regret bound. Moreover, experiment results show that A2S improves accuracy by up to 2.64% and accelerates training by up to 41% more than the state-of-the-art synchronization methods BSP, ASP, stale synchronous parallel (SSP), dynamic SSP, and Sync-switch.

Asynchronous Parallel Expected Improvement Matrix-Based Constrained Multi-Objective Optimization for Analog Circuit Sizing

In this brief, we propose an efficient asynchronous parallel expected improvement matrix-based (EIM) constrained multi-objective optimization approach with mixed models for analog circuit sizing. In contrast to other asynchronous methods for single-objective optimization, our method aims to solve multi-objective problems with multiple constraints. The proposed new asynchronous strategy can parallelize not only circuit simulations but also the process of building models. We introduce an EIM criterion for objectives as the acquisition function and build radial basis function models for constraints. The proposed method is based on prescreening instead of internal optimization to further reduce the computational cost of models. Experimental results on two real-world circuits illustrate that when the batch size is 15, our proposed method provides with better trade-off information and reduces the total runtime of the optimization by up to 92X compared with NSGA-II. There is a speedup of up to 8X and 4X compared with the state-of-the-art synchronous approach and asynchronous approach in terms of the runtime.

Analysis of nonlinear soil-structure interaction using partitioned method

For the current state-of-practice in soil-structure interaction (SSI) of nuclear facilities, the strain-dependent characteristics of soil are considered indirectly via equivalent linear methods in the frequency domain, typically represented by the SASSI program. SSI analysis in the time-domain, although directly involving material nonlinearity of the soil, is inefficient in practice up to now. Based on Partitioned Analysis of SSI (PASSI), a method for nonlinear soil-structure interaction analysis is developed in this paper, by applying implicit time-step integration and explicit one with different time steps to nuclear island and to soil, respectively. The incremental equilibrium equation and explicit decoupling method are used to analyze the soil nonlinearity described by the Davidenkov model with simplified loading-reloading rules, which avoids solving algebraic equations and iteration processes. An asynchronous parallel algorithm for nonlinear SSI analysis is given. A simple example is given to verify the partitioned approach with explicit-implicit co-computation against the full-explicit approach. Seismic response characteristics of nuclear power plants are investigated by comparing responses of nonlinear analysis with linear analysis. Different results address the importance of nonlinear SSI analysis for the seismic performance of nuclear structures.

Asynchronous simulated annealing on the placement problem: A beneficial race condition

Race conditions, which occur when compute workers do not synchronise correctly, are considered undesirable in parallel computing, as they introduce often-unintended stochastic behaviour. This study presents an asynchronous parallel algorithm with a race condition, and demonstrates that it reaches a superior solution faster than the equivalent synchronous algorithm without the race condition. Specifically, a parallel simulated annealing algorithm that solves a graph mapping problem (placement) is used to explore this. This paper illustrates how problem size and degree of parallelism affects both the collision rate caused by the race condition, and convergence time. The asynchronous approach reaches a superior solution in half the time of the equivalent synchronous approach. The solver presented here can be applied to application deployment in distributed systems, and the concept can be applied to problems solvable by global optimisation methods, where fitness errors can be tolerated in exchange for faster execution.

Beyond backpropagate through time: Efficient model‐based training through time‐splitting

Model-based policy gradient (MBPG) has been employed to seek an approximate solution to the optimal control problem. However, there is coupling between adjacent states due to temporal dependencies, making the training time grow linearly with the time horizon. This paper reshapes the training process of MBPG with the time-splitting technique to establish a time-independent algorithm called Training Through Time-Splitting (T3S). First, copy the coupled variables to obtain two independent variables. Meanwhile, an extra variable together with an equivalence constraint is introduced for problem consistency. Then, the transformed problem divides into subproblems with carefully derived loss functions. Subproblems own decoupled variables and shared policy networks, which means they can be optimized concurrently. Guided by the algorithm design, this paper further proposes an asynchronous parallel training scheme to accelerate training efficiency. Numerical simulation shows that the T3S algorithm outperforms the MBPG algorithm by 83.6% in wall-clock time with a trajectory tracking task.