Spatial Parallelization Research Articles

Spatially encoded pure-shift diffusion-ordered NMR spectroscopy yielded by chirp excitation

Spatially-encoded diffusion-ordered NMR spectroscopy (SPEN-DOSY) has emerged as a new time-efficient tool for the analysis of mixtures of small molecules in solution. Time efficiency is achieved using the concept of spatial parallelization of the effective gradient area, instead of the sequential incrementation used in conventional diffusion experiments. The data acquired with such sequences are then usually processed to extract diffusion coefficients, but cases when peak overlap in the 1H spectrum are difficult to address. Such limitation in conventional diffusion experiments is addressed via using the Pure Shift Yielded by CHirp Excitation (PSYCHE)-iDOSY sequence. Here we have adapted the PSYCHE-iDOSY sequence by using echo planar spectroscopic imaging (EPSI) to acquire SPEN-DOSY data. The pure shift mode of PSYCHE separates the overlapped components and a modified Stejskal-Tanner equation is used to extract the corresponding diffusion coefficient. The primary results obtained with the above-mentioned mixtures seem to open the possibility of separating complex mixtures in less time than PSYCHE-iDOSY.

Parallel-in-time multiple shooting for optimal control problems governed by the Navier–Stokes equations

In the past decades, multiple shooting methods have proven to be a promising direction to speed up the optimization process, especially in the context of ODE-based optimization. Very recently, Fang et al. (Journal of Computational Physics, vol. 452, 110926, 2022) proposed a multiple shooting algorithm for large-scale PDE-constrained optimization. The current paper continues this line of work and explores the potential of multiple shooting methods for optimal control problems governed by the three-dimensional Navier–Stokes equations. Similar to Fang et al., the augmented Lagrangian (AL) method is used to solve the resulting equality-constrained optimization problem, and we employ the classical limited-memory BFGS method for the unconstrained subproblems inside the AL loop. In the current work, we exploit the multiple shooting paradigm in full by processing the shooting windows parallel-in-time, allowing for significant parallel speed-ups compared to single shooting. The proposed method is validated on a velocity tracking case, using up to 100 windows. Our analysis shows that the multiple shooting algorithm allows for considerable algorithmic and parallel speed-ups. While algorithmic speed-up depends on the exact tracking case and initialization of the shooting windows, the multiple shooting algorithm always outperforms single shooting in terms of computational time (if the number of windows is sufficiently high) due to the parallel-in-time implementation. For a given amount of resources, we also show that the proposed parallel-in-time strategy can outperform spatial parallelization alone, especially when the spatial parallelization is saturated.

Time parallelism and Newton-adaptivity of the two-derivative deferred correction discontinuous Galerkin method

In this work, we consider a high-order discretization of compressible viscous flows allowing parallelization both in space and time.The discontinuous Galerkin spectral element method, which is well-suited for massively parallel simulations, is used for spatial discretization. The main novelty in this work is the additional demonstration of time-parallel capabilities within an implicit two-derivative timestepping procedure to further increase the parallel speedup. Temporal parallelism is made possible by a predictor-corrector-type time discretization that allows to split the associated workload onto multiple processors.We identify a homogeneous load balance with respect to the linear (GMRES) iterations on each processor as a key for parallel efficiency. To homogenize the load and to enable practical simulations, an adaptive strategy for Newton’s method is introduced. It is shown that the time-parallel method provides a parallel efficiency of approx. 60−70% on 4−7 computational partitions. Moreover, the capabilities of the novel method for the simulation of large-scale problems are illustrated with a mixed temporal and spatial parallelization on more than 1000 processors.

Coherent Optical Transceivers Scaling and Integration Challenges

The advancement of digital coherent technologies has dramatically increased the system capacity per single-core single-mode fiber to the point that we can now approach the Shannon limit by utilizing high-order modulation formats and high-coding gain forward error correction (FEC) codes. Because the required energy per bit increases exponentially the closer we get to the Shannon limit, extending the available optical bandwidth by using ultrawideband wavelength-division multiplexing (WDM) and/or spatial-division multiplexing (SDM) is indispensable for increasing the system capacity with high energy efficiency. However, simple extensions of wavelength resources and spatial parallelization dramatically increase the number of transceivers (TxRxs) in proportion to the wavelength/spatial multiplicity. The key to achieving cost- and energy-efficient systems is to reduce the system complexity by using high-density integration and broadband optelectronics. In this article, we overview and discuss the recent advances of coherent optical transceivers integrated with an optical front end and digital signal processing (DSP)/application-specific integrated circuit (ASIC). We then present the transponder architectures and the challenges involved in applying them for massive parallelized transmission systems.

Parallel-in-time simulation of biofluids

We extend Parareal, a parallel-in-time method that alternates between a serial sweep and a parallel sweep, to simulate the fluid flow around bio-inspired, dynamic structures over a period of time. Our main contributions include demonstrating the applicability of Parareal to the simulation of biofluids and developing novel solvers for the serial sweeps of Parareal. We propose to construct non-intrusive solvers by extrapolating a parametrized family of existing solvers. Compared to the existing solvers, they either allow the use of larger time steps, have a higher order of accuracy in time, or both. They are also straightforward to implement and parallelize. Numerical results show that when the number of biological structures is small or the number of computer cores employed is sufficiently large, the proposed variant of Parareal can achieve a significantly higher parallel speedup than the more commonly used spatial parallelization.

Spatial parallel disturbance region update method with OpenMP for steady compressible flows

We present the spatial parallelization of the disturbance region update method (DRUM) with OpenMP, named the spatial parallel disturbance region update method (spDRUM). DRUM is an acceleration methodology for compressible flows, employing dynamic computational domains (DCDs) to eliminate the worthless computational effort of the conventional global-update method. However, the DCDs may cause a load-imbalance issue, making DRUM difficult to execute efficiently in parallel. In this work, parallel strategies of steps in spDRUM are introduced, and a dynamic load balancer that is established based on the equivalent number of cells in the DCDs and a combined grid partitioning method is proposed to exploit the full acceleration potential of spDRUM. Besides, a concept of sub-blocks that takes advantage of the memory-shared characteristic of OpenMP is embedded in the data structure to simplify the inner-boundary treatment. Numerical test cases demonstrate that spDRUM accomplishes remarkable convergence speed for solving all steady compressible flow problems; benefiting from the coupled contribution of parallelism and DCDs, the total speed-up ratio of spDRUM could be significantly greater than that of the ideal case ignoring parallel overheads and that of the global-update method with the same number of threads; the proposed grid partitioning method could generate various types of partitions efficiently and flexibly, keeping load balanced for multiblock structured grids; embedding the concept of sub-blocks in the data structure makes the number of inner-boundary cells unchanged and remains optimal, regardless of the variation in partitions.

Rapid analytical CEST spectroscopy of competitive host–guest interactions using spatial parallelization with a combined approach of variable flip angle, keyhole and averaging (CAVKA)

A serious limitation of high resolution 129Xe chemical exchange saturation transfer (CEST) NMR spectroscopy for comparing competitive host-guest interactions from different samples is the long acquisition time due to step-wise encoding of the chemical shift dimension. A method of optimized use of 129Xe spin magnetization to enable the accelerated and simultaneous acquisition of CEST spectra from multiple samples or regions in a setup is described. The method is applied to investigate the host-guest system of commercially available cucurbit[7]uril (CB7) and xenon with competing guests: cis-1,4-bis(aminomethyl)cyclohexane, cadaverine, and putrescine. Interactions with the different guests prove that the observed CEST signal is from a CB6 impurity and that CB7 itself does not produce a CEST signal. Instead, rapid interactions between xenon and CB7 manifest in the spectrum as a broad saturation response that could be suppressed by cis-1,4-bis(aminomethyl)cyclohexane. This guest prevents interactions at the CB7 portals. The suggested method represents a type of spectroscopic imaging that is capable of capturing the exchange kinetics information of systems that otherwise suffer from shortened T2 times and yields multiple spectra for comparing exchange conditions with a reduction of >95% in acquisition time. The spectral quality is sufficient to perform quantitative analysis and quantifications relative to a CB6 standard as well as relative to a known blocker concentration (putrescine) that both reveal an unexpectedly high CB6 impurity of ca. 8%.

Scalable and distributed strategies for socially distanced human mobility

COVID-19 is a global health crisis that has caused ripples in every aspect of human life. Amid widespread vaccinations testing, manufacture and distribution efforts, nations still rely on human mobility restrictions to mitigate infection and death tolls. New waves of infection in many nations, indecisiveness on the efficacy of existing vaccinations, and emerging strains of the virus call for intelligent mobility policies that utilize contact pattern and epidemiological data to check contagion. Our earlier work leveraged network science principles to design social distancing optimization approaches that show promise in slowing infection spread however, they prove to be computationally prohibitive and require complete knowledge of the social network. In this work, we present scalable and distributed versions of the optimization approaches based on Markov Chain Monte Carlo Gibbs sampling and grid-based spatial parallelization that tackle both the challenges faced by the optimization strategies. We perform extensive simulation experiments to show the ability of the proposed strategies to meet necessary network science measures and yield performance comparable to the optimal counterpart, while exhibiting significant speed-up. We study the scalability of the proposed strategies as well as their performance in realistic scenarios when a fraction of the population temporarily flouts the location recommendations.

An accurate and time-parallel rational exponential integrator for hyperbolic and oscillatory PDEs

Rational exponential integrators (REXI) are a class of numerical methods that are well suited for the time integration of linear partial differential equations with imaginary eigenvalues. Since these methods can be parallelized in time (in addition to the spatial parallelization that is commonly performed) they are well suited to exploit modern high performance computing systems. In this paper, we propose a novel REXI scheme that drastically improves accuracy and efficiency. The chosen approach will also allow us to easily determine how many terms are required in the approximation in order to obtain accurate results. We provide comparative numerical simulations for a shallow water equation that highlight the efficiency of our approach and demonstrate that REXI schemes can be efficiently implemented on graphic processing units.

A parallel inverted dual time stepping method for unsteady incompressible fluid flow and heat transfer problems

The simulation of unsteady incompressible fluid flow and heat transfer problems is a huge time-consuming task and has been becoming one of the hottest topics in the fields of computational fluid dynamics and numerical heat transfer. Pure spatial parallelization has been so far the most widely used parallel strategy to speed up unsteady simulations but suffers from the problem of speedup saturation as parallel scale increases. In order to further speed up unsteady simulations and exploit the concurrency in temporal dimension, we propose a novel temporal parallelization strategy, i.e., parallel inverted dual time stepping method, by simply inverting the sequence of the inner loop (pseudo time) and the outer loop (physical time) within the dual time stepping framework. The parallel performance of the proposed method is verified and validated with three steady/unsteady problems. Parallel results of the proposed method agree well with those of the dual time stepping method and the available benchmark solutions. The parallel efficiency of the proposed method is about 96%, 81%, 62%, 51%, 45% for 4, 8, 12, 16, 20 CPU cores, respectively, which is considerably higher than that of the widely used parareal method. The proposed method has potential in further increasing parallel scale as spatial parallelization is saturated.

A spatial decomposition parallel algorithm for a concurrent atomistic-continuum simulator and its preliminary applications

This paper presents the development of a spatial decomposition parallel algorithm and its implementation into a concurrent atomistic-continuum (CAC) method simulator for multiscale modeling of dislocations in metallic materials. The scalability and parallel efficiency of the parallelized CAC are tested using up to 512 processors. With a modest computational resource, a single crystalline f.c.c. sample containing 10.6 billion atoms is modeled using only 4,809,108 finite elements in a CAC model at a fraction of the cost of full molecular dynamics (MD). The simulation demonstrates a nearly ideal scalability of the newly parallelized CAC simulator. The parallel efficiency of the newly parallelized CAC is shown to be higher than 90% when using 512 processors in the high performance computing cluster at Iowa State University. This parallel efficiency is comparable to the state-of-the-art atomistic simulator. Moreover, the newly parallelized CAC simulator employing a uniform coarse mesh is capable of capturing important atomistic features of dislocations, including dislocation nucleation, migration, stacking faults as well as the formation of Lomer-Cottrell locks, in a billion-atom system. The spatial decomposition-based parallelization algorithm developed in this work is general and can be transferable to many other existing concurrent multiscale simulation tools.

A decomposition approach to the static traffic assignment problem

This paper describes a spatial parallelization scheme for the static traffic assignment problem. In this scheme, which we term a decomposition approach to the static traffic assignment problem (DSTAP), the network is divided into smaller networks, and the algorithm alternates between equilibrating these networks as subproblems, and master iterations using a simplified version of the full network. The simplified network used for the master iterations is based on linearizations to the equilibrium solution for each subnetwork obtained using sensitivity analysis techniques. We prove that the DSTAP method converges to the equilibrium solution on the full network, and demonstrate computational savings of 35–70% on the Austin network. Natural applications of this method are statewide or national assignment problems, or cities with rivers or other geographic features where subnetworks can be easily defined.

A high-performance finite-volume algorithm for solving partial differential equations governing compressible viscous flows on structured grids

This work focuses on the development of a high-performance fourth-order finite-volume method to solve the nonlinear partial differential equations governing the compressible Navier–Stokes equations on a Cartesian grid with adaptive mesh refinement. The novelty of the present study is to introduce the loop chaining concept to this complex fourth-order fluid dynamics algorithm for significant improvement in code performance on parallel machines. Specific operations involved in the algorithm include the finite-volume formulation of fourth-order spatial discretization stencils and optimal inter-loop parallelization strategies. Numerical fluxes of the Navier–Stokes equations comprise the hyperbolic (inviscid) and elliptic (viscous) ​components. The hyperbolic flux is evaluated using high-resolution Godunov’s method and the elliptic flux is based on fourth-order centered-difference methods everywhere in the computational domain. The use of centered-difference methods everywhere supports the idea of fusing modular codes to achieve high efficiency on modern computers. Temporal discretization is performed using the standard fourth-order ​Runge–Kutta method. The fourth-order accuracy of solution in space and time is verified with a transient Couette flow problem. The algorithm is applied to solve the Sod’s shock tube and the transient flat-plate boundary layer flow. The numerical predictions are validated by comparing to the analytical solutions. The performance of the baseline code is compared to that of the fused scheme which fuses modular codes via loop chaining concept and a significant improvement in execution time is observed.

Numerical simulation of skin transport using Parareal

In-silico investigation of skin permeation is an important but also computationally demanding problem. To resolve all scales involved in full detail will not only require exascale computing capacities but also suitable parallel algorithms. This article investigates the applicability of the time-parallel Parareal algorithm to a brick and mortar setup, a precursory problem to skin permeation. The C++ library Lib4PrM implementing Parareal is combined with the UG4 simulation framework, which provides the spatial discretization and parallelization. The combination's performance is studied with respect to convergence and speedup. It is confirmed that anisotropies in the domain and jumps in diffusion coefficients only have a minor impact on Parareal's convergence. The influence of load imbalances in time due to differences in number of iterations required by the spatial solver as well as spatio-temporal weak scaling is discussed.

A stencil-based implementation of Parareal in the C++ domain specific embedded language STELLA

In view of the rapid rise of the number of cores in modern supercomputers, time-parallel methods that introduce concurrency along the temporal axis are becoming increasingly popular. For the solution of time-dependent partial differential equations, these methods can add another direction for concurrency on top of spatial parallelization. The paper presents an implementation of the time-parallel Parareal method in a C++ domain specific language for stencil computations (STELLA). STELLA provides both an OpenMP and a CUDA backend for a shared memory parallelization, using the CPU or GPU inside a node for the spatial stencils. Here, we intertwine this node-wise spatial parallelism with the time-parallel Parareal. This is done by adding an MPI-based implementation of Parareal, which allows us to parallelize in time across nodes. The performance of Parareal with both backends is analyzed in terms of speedup, parallel efficiency and energy-to-solution for an advection–diffusion problem with a time-dependent diffusion coefficient.

Parallel algorithm and its convergence of spatial domain decomposition of discrete ordinates method for solving radiation heat transfer problem

To improve the computational efficiency and hold calculation accuracy at the same time, we study the parallel computation for radiation heat transfer. In this paper, the discrete ordinates method (DOM) and the spatial domain decomposition parallelization (DDP) are combined by message passing interface (MPI) language. The DDP–DOM computation of the radiation heat transfer within the rectangular furnace is described. When the result of DDP–DOM along one-dimensional direction is compared with that along multi-dimensional directions, it is found that the result of the latter one has higher precision without considering the medium scattering. Meanwhile, an in-depth study of the convergence of DDP–DOM for radiation heat transfer is made. Analyzing the cause of the weak convergence, we relate the total number of iteration steps when the convergence is obtained to the number of sub-domains. When we decompose the spatial domain along one-, two- and three-dimensional directions, different linear relationships between the number of total iteration steps and the number of sub-domains will be possessed separately, then several equations are developed to show the relationships. Using the equations, some phenomena in DDP–DOM can be made clear easily. At the same time, the correctness of the equations is verified.

A comparison of continuous mass‐lumped finite elements with finite differences for 3‐D wave propagation

ABSTRACTThe finite‐difference method on rectangular meshes is widely used for time‐domain modelling of the wave equation. It is relatively easy to implement high‐order spatial discretization schemes and parallelization. Also, the method is computationally efficient. However, the use of finite elements on tetrahedral unstructured meshes is more accurate in complex geometries near sharp interfaces. We compared the standard eighth‐order finite‐difference method to fourth‐order continuous mass‐lumped finite elements in terms of accuracy and computational cost. The results show that, for simple models like a cube with constant density and velocity, the finite‐difference method outperforms the finite‐element method by at least an order of magnitude. Outside the application area of rectangular meshes, i.e., for a model with interior complexity and topography well described by tetrahedra, however, finite‐element methods are about two orders of magnitude faster than finite‐difference methods, for a given accuracy.

Cloud-Based Spatial Information Service Architecture within LBS

Location Based Services (LBS) have become a popular technology to retrieve information about the surroundings of a mobile user which results in ubiquitous demand of spatial information service with diverse needs of different types of users. The aim of this paper is to reveal the potential of cloud-based spatial information service architecture that plays an integral role in LBS design and practice. This paper analyzes the characteristics of the spatial information of cloud services, such as data access transparency, spatial analysis parallelization, service capabilities flexible, information services standardization, service aggregation visualization, re-development flexible. This paper provides a possible solution to overcome the LBS service issues. In this paper, the short review of LBS and Cloud computing are given first and then the possibility of LBS design with cloud computing are analyzed. At last, cloud-based spatial information service architecture is proposed.

Full Core Reactor Analysis: Running Denovo on Jaguar

Fully consistent, full core, three-dimensional, deterministic neutron transport simulations using the orthogonal mesh code Denovo were run on the massively parallel computing architecture Jaguar XT5. Using energy and spatial parallelization schemes, Denovo was able to efficiently scale to more than 160 000 processors. Cell-homogenized cross sections were used with step characteristics, linear discontinuous finite element, and trilinear discontinuous finite element spatial methods. It was determined that using the finite element methods gave considerably more accurate eigenvalue solutions for large–aspect ratio meshes than using step characteristics.

Parallelization in time of numerical simulations of fully-developed plasma turbulence using the parareal algorithm

It is shown that numerical simulations of fully-developed plasma turbulence can be successfully parallelized in time using the parareal algorithm. The result is far from trivial, and even unexpected, since the exponential divergence of Lagrangian trajectories as well as the extreme sensitivity to initial conditions characteristic of turbulence set these type of simulations apart from the much simpler systems to which the parareal algorithm has been applied to this day. It is also shown that the parallel gain obtainable with this method is very promising (close to an order of magnitude for the cases and implementations described), even when it scales with the number of processors quite differently to what is typical for spatial parallelization.