Efficient Parallel Computation Research Articles

Distributed matrix computing system for big data

In order to solve the problem of low computing efficiency in big data analysis and model construction, this paper intended to deeply explore the big data analysis programming model, DAG (Directed Acyclic Graph) and other contents, and on this basis, it adopted a distributed matrix computing system Octopus for big data analysis. Octopus is a universal matrix programming framework that provides a programming model based on matrix operations, which can conveniently analyze and process large-scale data. By using Octopus, users can extract functions and data from multiple platforms and operate through a unified matrix operation interface. The distributed matrix representation and storage layer can design data storage formats for distributed file systems. Each computing platform in OctMatrix provides its own matrix library, and it provides a matrix library written in R language for the above users. SymboMatrix provides a matrix interface to OctMatrix that is consistent with OctMatrix. However, SymboMatrix also retains the flow diagram for matrix operations in the process, and it also supports logical and physical optimization of the flow diagram on a DAG. For the DAG computational flow graph generated by SymbolMatrix, this paper divided it into two parts: logical optimization and physical optimization. This paper adopted a distributed file system based on line matrix, and obtained the corresponding platform matrix by reading the documents based on line matrix. In the evaluation of system performance, it was found that the distributed matrix computing system had a high computing efficiency, and the average CPU (central processing unit) usage reached 70%. This system can make full use of computing resources and realize efficient parallel computing.

Parallel finite layer method for land subsidence and its homotopy parameter inversion

Land subsidence is caused by the interaction between groundwater flow and soil deformation. At present, in the field of land subsidence calculation, there is an increasing tendency to use Biot's consolidation theory to precisely simulate the coupling processes of flow and deformation. However, there are some practical problems with large-scale numerical computation, such as large computational loads, low efficiency, and difficulty in obtaining parameters. The Finite Layer Method employs orthogonality of analytical functions to achieve decoupling among various series terms. And the method can be enhanced through parallel computing to optimize the efficiency of solving complex and extensive problems due to its decoupling property. Based on this, combined with the nonlinear homotopy method, a new approach for the reversal of land subsidence is proposed. The validity of the parallel program is confirmed through comparison with established solutions, and the impact of various factors on the efficiency of parallel computing is analyzed. The results indicate that the parallel approach enhances computational efficiency, while the homotopy inversion technique exhibits broad convergence characteristics.

Point convolutional neural network algorithm for Ising model ground state research based on spring vibration

The ground state search of the Ising model can be used to solve many combinatorial optimization problems. Under the current computer architecture, an Ising ground state search algorithm suitable for hardware computing is necessary for solving practical problems. Inspired by the potential energy conversion of the springs, we propose the Spring-Ising Algorithm, a point convolutional neural network algorithm for ground state search based on the spring vibration model. Spring-Ising Algorithm regards the spin as a moving mass point connected to a spring and establishes the equation of motion for all spins. Spring-Ising Algorithm can be mapped on AI chips through the basic structure of the neural network for fast and efficient parallel computing. The algorithm has shown promising results in solving the Ising model and has been tested in the recognized test benchmark K2000. The optimal results of this algorithm after 10,000 steps of iteration are 2.9% of all results. The algorithm introduces the concept of dynamic equilibrium to achieve a more detailed local search by dynamically adjusting the weight of the Ising model in the spring oscillation model. Spring-Ising Algorithm offers the possibility to calculate the Ising model on a chip which focuses on accelerating neural network calculations.

PARALLELPROJ—an open-source framework for fast calculation of projections in tomography

In this article, we introduce parallelproj, a novel open-source framework designed for efficient parallel computation of projections in tomography leveraging either multiple CPU cores or GPUs. This framework efficiently implements forward and back projection functions for both sinogram and listmode data, utilizing Joseph’s method, which is further extended to encompass time-of-flight (TOF) PET projections. Our evaluation involves a series of tests focusing on PET image reconstruction using data sourced from a state-of-the-art clinical PET/CT system. We thoroughly benchmark the performance of the projectors in non-TOF and TOF, sinogram, and listmode employing multi CPU-cores, hybrid CPU/GPU, and exclusive GPU mode. Moreover, we also investigate the timing of non-TOF sinogram projections calculated in STIR (Software for Tomographic Image Reconstruction) which recently integrated parallelproj as one of its projection backends. Our results indicate that the exclusive GPU mode provides acceleration factors between 25 and 68 relative to the multi-CPU-core mode. Furthermore, we demonstrate that OSEM listmode reconstruction of state-of-the-art real-world PET data sets is achievable within a few seconds using a single consumer GPU.

A Novel Method for State of Charge Estimation in Lithium-Ion Batteries Using Temporal Convolutional Network and Multi-Verse Optimization

This study presents a novel data-driven method for state-of-charge estimation in lithium-ion batteries. It integrates a temporal convolutional network with multi-verse optimization to enhance the accuracy of predicting the state of charge. The temporal convolutional network possesses advantages such as an extended memory window and efficient parallel computation, exhibiting exceptional performance in time-series tasks for state of charge estimation. Its hyperparameters are optimized by adopting multi-verse optimization to obtain better model performance. The driving model utilizes various measurable data as inputs, including battery terminal voltage, current, and surface temperature. To validate the effectiveness of the proposed method, extensive datasets from diverse dynamic working conditions at different ambient temperatures are employed for model training, validation, and testing. The numerical outcomes provide evidence of the proposed method’s superior performance compared to the other two methods, providing a more robust and accurate solution for the state of charge estimation in lithium-ion batteries.

Comprehensive holographic parallel beam modulation inside material based on automatic differentiation

Holographic beam modulation is widely applied in optical tweezers, hard-brittle material marking, high-density storage, etc. To generate high-fidelity arbitrary 3-dimensional (3D) parallel multifoci inside the material, the spherical aberration compensated automatic differentiation (SACAD) algorithm is presented. All polarization components are included and the spherical aberration compensation is embedded in the physical model. The technique of automatic differentiation is used in the error backpropagation procedure, ensuring efficient parallel computing of pixel-by-pixel gradients. In several simulation tests, the root mean square errors of the generated 3D multifoci distributions are all less than 0.01 and the diffraction efficiencies are all beyond 90%, outperforming the results of the established algorithms. In the experiments, we have verified the advantage of SACAD algorithm in complicated 3D internal marking with spherical aberration compensation. Since the SACAD algorithm can achieve high fidelity and efficiency phase retrieval with a straightforward procedure, it has the potential to become a well-received solution for internal parallel beam modulation.

A Rapid Parallel Mosaicking Algorithm for Massive Remote Sensing Images Utilizing Read Filtering

Mosaicking is a crucial step in the application of remote sensing images. The amount of remote sensing image data has grown rapidly, along with the expansion of observed areas and increased image resolution. As a result, traditional serial mosaicking techniques are facing significant challenges. In recent times, various studies have utilized high-performance computing to hasten image mosaicking and attain favorable outcomes. Nevertheless, the current research only accelerates mosaicking through external technology, without optimizing from the perspective of algorithm flow, which introduces unnecessary data I/O and slows down the mosaicking. This paper introduces a rapid parallel remote sensing image mosaicking algorithm utilizing read filtering. To begin with, the target images are divided into blocks and stored in a distributed file system. Subsequently, the image blocks are read and filtered based on a designated input format. Finally, the overlapping and non-overlapping areas are read and processed asynchronously, reducing the data I/O and computing overhead, thereby improving the efficiency of parallel computing. The experiments indicate that the mosaicking algorithm introduced in this paper enhances throughput and speedup by an average of 1.38 MB/S and 0.87 relative to the current techniques, respectively, concerning various datasets and cores. This study provides a theoretical foundation and novel ideas for processing remote sensing images on cluster platforms.

STADIA: Photonic Stochastic Gradient Descent for Neural Network Accelerators

Deep Neural Networks (DNNs) have demonstrated great success in many fields such as image recognition and text analysis. However, the ever-increasing sizes of both DNN models and training datasets make deep leaning extremely computation- and memory-intensive. Recently, photonic computing has emerged as a promising technology for accelerating DNNs. While the design of photonic accelerators for DNN inference and forward propagation of DNN training has been widely investigated, the architectural acceleration for equally important backpropagation of DNN training has not been well studied. In this paper, we propose a novel silicon photonic-based backpropagation accelerator for high performance DNN training. Specifically, a general-purpose photonic gradient descent unit named STADIA is designed to implement the multiplication, accumulation, and subtraction operations required for computing gradients using mature optical devices including Mach-Zehnder Interferometer (MZI) and Mircoring Resonator (MRR), which can significantly reduce the training latency and improve the energy efficiency of backpropagation. To demonstrate efficient parallel computing, we propose a STADIA-based backpropagation acceleration architecture and design a dataflow by using wavelength-division multiplexing (WDM). We analyze the precision of STADIA by quantifying the precision limitations imposed by losses and noises. Furthermore, we evaluate STADIA with different element sizes by analyzing the power, area and time delay for photonic accelerators based on DNN models such as AlexNet, VGG19 and ResNet. Simulation results show that the proposed architecture STADIA can achieve significant improvement by 9.7× in time efficiency and 147.2× in energy efficiency, compared with the most advanced optical-memristor based backpropagation accelerator.

Investigation and Implementation of Parallelism Resources of Numerical Algorithms

This article is devoted to an approach to solving a problem of the efficiency of parallel computing. The theoretical basis of this approach is the concept of a Q -determinant. Any numerical algorithm has a Q -determinant. The Q -determinant of the algorithm has clear structure and is convenient for implementation. The Q -determinant consists of Q -terms. Their number is equal to the number of output data items. Each Q -term describes all possible ways to compute one of the output data items based on the input data. We also describe a software Q -system for studying the parallelism resources of numerical algorithms. This system enables to compute and compare the parallelism resources of numerical algorithms. The application of the Q -system is shown on the example of numerical algorithms with different structures of Q -determinants. Furthermore, we suggest a method for designing of parallel programs for numerical algorithms. This method is based on a representation of a numerical algorithm in the form of a Q -determinant. As a result, we can obtain the program using the parallelism resource of the algorithm completely. Such programs are called Q - effective . The results of this research can be applied to increase the implementation efficiency of numerical algorithms, methods, as well as algorithmic problems on parallel computing systems.

Restoration of Atmospheric Turbulence-Degraded Short-Exposure Image Based on Convolution Neural Network

Ground-based remote observation systems are vulnerable to atmospheric turbulence, which can lead to image degradation. While some methods can mitigate this turbulence distortion, many have issues such as long processing times and unstable restoration effects. Furthermore, the physics of turbulence is often not fully integrated into the image reconstruction algorithms, making their theoretical foundations weak. In this paper, we propose a method for atmospheric turbulence mitigation using optical flow and convolutional neural networks (CNN). We first employ robust principal component analysis (RPCA) to extract a reference frame from the images. With the help of optical flow and the reference frame, the tilt can be effectively corrected. After correcting the tilt, the turbulence mitigation problem can be simplified as a deblurring problem. Then, we use a trained CNN to remove blur. By utilizing (i) a dataset that conforms to the turbulence physical model to ensure the restoration effect of the CNN and (ii) the efficient parallel computing of the CNN to reduce computation time, we can achieve better results compared to existing methods. Experimental results based on actual observed turbulence images demonstrate the effectiveness of our method. In the future, with further improvements to the algorithm and updates to GPU technology, we expect even better performance.

Adaptation of Parallel SaaS to Heterogeneous Co-Located Cloud Resources

Cloud computing has received increasing attention due to its promise of delivering on-demand, scalable, and virtually unlimited resources. However, heterogeneity or co-location of virtual cloud resources can cause severe degradation of the efficiency of parallel computations because of a priori unknown application-specific performance metrics, load imbalance, and limitations of memory bandwidth. This paper presents the runtime adaptation of parallel discrete element method (DEM) Software as a Service (SaaS) to heterogeneous or co-located resources of the OpenStack cloud. The computational workload is adapted by using weighted repartitioning and runtime measured performance of parallel computations on Docker containers. The high improvement in performance up to 48.7% of the execution time is achieved, applying the runtime adapted repartitioning when the load imbalance is high enough. The low load imbalance leads to the close values of computational load, when small variations in the system load and performance can cause oscillations in subsets of particles. Memory stress tests cause heterogeneity of non-isolated containers, which reduces the performance of memory bandwidth bound DEM SaaS on the co-located resources. The runtime adapted repartitioning handles the constant and periodically variable performance of non-isolated containers and decreases the total execution time of DEM SaaS.

Intelligent Information Technology for Inductive Modeling of Complex Processes on the Basis of Recurrent-and-Parallel Computations

The paper develops a novel intelligent information technology for inductive modeling of complex processes by experimental data, the high level of productivity of which is achieved by applying a new concept of combining the efficiency of recurrent and parallel computations. The implementation of such technology in modern intelligent information-and-analytical systems provides a significant increase in the efficiency and validity of making managerial decisions in the tasks of operational management of complex processes. An example is done of using the developed technology for evaluation and forecast of the investment activity in Ukraine.

A Recompressed Nested Cross Approximation for Electrically Large Bodies

A recompressed nested cross approximation (rNCA) based closely on the recent fast nested cross approximation (fNCA) algorithm is formulated in this article. The proposed method builds on previous work in which the fNCA was formulated in a purely algebraic and kernel-independent fashion, using a top-down recursive application of the adaptive cross-approximation (ACA). Our proposed method employs ACA recompression to avoid the need to compute low-rank approximations of excessively large far-field matrices, and thus mitigates the effects of high-frequency rank growth on run-time scaling for electrically large models. The low run-time and memory cost allows for efficient parallel computation of <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\mathcal {H}^{2}$ </tex-math></inline-formula> -matrices for systems of excessive electrical sizes. Radar cross sections (RCSs) are evaluated for electrically large instances of a perfectly conducting sphere and the NASA Almond. We observe near-linear scaling of memory cost and construction time.

A local data assimilation method (Local DA v1.0) and its application in a simulated typhoon case

Abstract. Integrating the hybrid and multiscale analyses and the parallel computation is necessary for current data assimilation schemes. A local data assimilation method, Local DA, is designed to fulfill these needs. This algorithm follows the grid-independent framework of the local ensemble transform Kalman filter (LETKF) and is more flexible in hybrid analysis than the LETKF. Local DA employs an explicitly computed background error correlation matrix of model variables mapped to observed grid points/columns. This matrix allows Local DA to calculate static covariance with a preset correlation function. It also allows the conjugate gradient (CG) method to be used to solve the cost function and allows localization to be performed in model space, observation space, or both spaces (double-space localization). The Local DA performance is evaluated with a simulated multiscale observation network that includes sounding, wind profiler, precipitable water vapor, and radar observations. In the presence of a small-size time-lagged ensemble, Local DA can produce a small analysis error by combining multiscale hybrid covariance and double-space localization. The multiscale covariance is computed using error samples decomposed into several scales and independently assigning the localization radius for each scale. Multiscale covariance is conducive to error reduction, especially at a small scale. The results further indicate that applying the CG method for each local analysis does not result in a discontinuity issue. The wall clock time of Local DA implemented in parallel is halved as the number of cores doubles, indicating a reasonable parallel computational efficiency of Local DA.

CitcomSVE: A Three‐Dimensional Finite Element Software Package for Modeling Planetary Mantle’s Viscoelastic Deformation in Response to Surface and Tidal Loads

CitcomSVE: A Three‐Dimensional Finite Element Software Package for Modeling Planetary Mantle’s Viscoelastic Deformation in Response to Surface and Tidal Loads

Resource and delay aware fine-grained service offloading in collaborative edge computing

Fine-grained service offloading in collaborative edge computing can realize full use of the limited resources of edge nodes to achieve efficient parallel computing. The existing research mainly focuses on service delay but pays insufficient attention to the network status, which will easily cause unbalanced resource utilization. Therefore, we propose a resource and delay aware fine-grained service offloading mechanism. First, we propose a novel network-adaptive service graph reconstruction algorithm to reduce the complexity of service offloading and the transmission delay, which includes service graph partition, dependency conflict detection and elimination, and service graph re-creation. Next, to better balance link and node resource utilization respectively, we propose original graph-based and association graph-based service graph mapping algorithms based on graph neural networks. A goal-directed affinity-based loss function is explored for them, which aims to address the difficulty of label generation in supervised learning. We conduct extensive simulation experiments with different numbers of subtasks, edge nodes and service requests under different network resource statuses. The experimental results show that the proposed service graph reconstruction method can balance network resource utilization, while reducing the service transmission delay and algorithm execution time for complex services. Moreover, the service graph mapping algorithms can improve the resource utilization balance degree while satisfying service constraints with start-end node location, resources and delay in various scenarios, especially in the case of unbalanced user distribution. Generally, our fine-grained service offloading mechanism enables short execution time and strong scalability, and is applicable to dynamic edge networks.

Integrate-and-Fire Neuron With Li-Based Electrochemical Random Access Memory Using Native Linear Current Integration Characteristics

Neuromorphic computing has gained a considerable research interest due to its potential in realizing highly efficient parallel computations. However, the existing neuromorphic architectures face various drawbacks. In this study, we present an integrate-and-fire (I&F) neuron using a Li-based electrochemical random access memory (Li-ECRAM) to achieve exceptional area efficiency and low-power neuromorphic computing. The proposed Li-ECRAM neuron employs a significantly reduced number of transistors when compared to other novel nonvolatile memory-based I&F neurons due to linear current integration characteristics and a high linear conductance response to the input current. As the integration-type Li-ECRAM is linear, it eliminates the requirement of a nonlinear compensating circuit. Therefore, a Li-ECRAM-based neuron has a simple structure comprising Li-ECRAM, reset transistor, inverter, and pulse generator. Furthermore, we also evaluate the operation speed and energy consumption of the proposed neuron, demonstrating the potential for high-speed and low-power operation. The proposed neuron can be applied in large-scale neuromorphic hardware applications due to the scalability and low energy consumption of Li-ECRAM.

Heterogeneous Big Data Parallel Computing Optimization Model using MPI/OpenMP Hybrid and Sensor Networks

For the heterogeneous big data parallel computing model, two levels of parallelism between nodes are not considered, resulting in low efficiency of heterogeneous big data parallel computing and bandwidth to send and receive information, high communication overhead, long model running time and small computational volume. In the paper, we propose an optimization model of heterogeneous big data parallel computing based on a hybrid Multi Point Interface (MPI)/Open Multi-Processing (OpenMP) and Sensor Networks. First, the processor characteristics of heterogeneous big data architecture is analyzed, the parallel tasks among processors are divided, collect the heterogeneous big data to be computed and cluster them, and use the processing results as the input items of the model. Then, a parallel load balancing mechanism is established to optimally divide the parallel computing load of heterogeneous big data, and a parallel computing optimization program is written by combining the hybrid programming mode of MPI and OpenMP and using the hybrid MPI/OpenMP, and finally, the parallel computing optimization of heterogeneous big data is realized by optimizing the parallel communication and determining the model parameters. The results show that the proposed model has a communication bandwidth of 510Mbps, a computational volume of 1.16GB, a model runtime of 24s, and an improved network bandwidth utilization of 93%, which can effectively reduce the communication overhead, and improve the efficiency of parallel computing and bandwidth sending and receiving information in sensor networks, and shorten the model running time.

Ultra-Fast Approximate Inference Using Variational Functional Mixed Models

ABSTRACT While Bayesian functional mixed models have been shown effective to model functional data with various complex structures, their application to extremely high-dimensional data is limited due to computational challenges involved in posterior sampling. We introduce a new computational framework that enables ultra-fast approximate inference for high-dimensional data in functional form. This framework adopts parsimonious basis to represent functional observations, which facilitates efficient compression and parallel computing in basis space. Instead of performing expensive Markov chain Monte Carlo sampling, we approximate the posterior distribution using variational Bayes and adopt a fast iterative algorithm to estimate parameters of the approximate distribution. Our approach facilitates a fast multiple testing procedure in basis space, which can be used to identify significant local regions that reflect differences across groups of samples. We perform two simulation studies to assess the performance of approximate inference, and demonstrate applications of the proposed approach by using a proteomic mass spectrometry dataset and a brain imaging dataset. Supplementary materials for this article are available online.

A load-decoupling parallel strategy based on shared memory architecture for DSMC to simulate near-continuum gases

Direct Simulation Monte Carlo (DSMC) method is gradually being applied to simulate near-continuum flowfields. However, the large amount of computation hinders its application. Parallelization is the primary way to solve this problem. Nevertheless, when the Knudsen number (Kn) is less than 1.0×10−2, it is hard to gain good load balance while parallelized since the overall computational load of DSMC is challenging to determine and also changes with cases. This paper proposed a fine-grain parallel strategy based on the shared memory architecture. This strategy is designed to gain good load balance through reasonably decoupling computational load, which means that the load of each module of the program is treated separately, not as a whole. Implemented through Open Multi-Processing (OpenMP), a “dual-decomposition” method is adopted to achieve the load-decoupling strategy. In the “dual-decomposition” method, the computation domain is decomposed with a different weight according to whether one module is particle-dominated or collision-dominated. Besides, a multi-segment partition algorithm with low overhead is proposed. Two cases, including one unsteady shock-bubble interaction (SBI) case and the other steady supersonic jet case, are used to verify our strategy's correctness and test the performance under different conditions. The results show the strategy newly proposed can reduce more than 70% load imbalance relative to our previous PartPlusColl strategy when Kn<1.0×10−2. Simultaneously, the results prove that our load-decoupling strategy can improve parallel efficiency by about 11.17%−21.80%, compared with Sparta [1]. This strategy sets the fundamental for improving parallel efficiency of large-scale parallel computing of DSMC.