High-level Parallel Research Articles

A Scalable Accelerator for Local Score Computation of Structure Learning in Bayesian Networks

A Bayesian network is a powerful tool for representing uncertainty in data, offering transparent and interpretable inference, unlike neural networks’ black-box mechanisms. To fully harness the potential of Bayesian networks, it is essential to learn the graph structure that appropriately represents variable interrelations within data. Score-based structure learning, which involves constructing collections of potentially optimal parent sets for each variable, is computationally intensive, especially when dealing with high-dimensional data in discrete random variables. Our proposed novel acceleration algorithm extracts high levels of parallelism, offering significant advantages even with reduced reusability of computational results. In addition, it employs an elastic data representation tailored for parallel computation, making it FPGA-friendly and optimizing module occupancy while ensuring uniform handling of diverse problem scenarios. Demonstrated on a Xilinx Alveo U50 FPGA, our implementation significantly outperforms optimal CPU algorithms and is several times faster than GPU implementations on an NVIDIA TITAN RTX. Furthermore, the results of performance modeling for the accelerator indicate that, for sufficiently large problem instances, it is weakly scalable, meaning that it effectively utilizes increased computational resources for parallelization. To our knowledge, this is the first study to propose a comprehensive methodology for accelerating score-based structure learning, blending algorithmic and architectural considerations.

An efficient controller-based architecture for AES algorithm using FPGA

An efficient controller-based architecture for AES algorithm using FPGA

LSH SimilarityJoin Pattern in FastFlow

Similarity joins are recognized to be among the most used data processing and analysis operations. We introduce a C++-based high-level parallel pattern implemented on top of FastFlow Building Blocks to provide the programmer with ready-to-use similarity join computations. The SimilarityJoin pattern is implemented according to the MapReduce paradigm enriched with locality sensitive hashing (LSH) to optimize the whole computation. The new parallel pattern can be used with any C++ serializable data structure and executed on shared- and distributed-memory machines. We present experimental validations of the proposed solution considering two different clusters and small and large input datasets to evaluate in-core and out-of-core executions. The performance assessment of the SimilarityJoin pattern has been conducted by comparing the execution time against the one obtained from the original hand-tuned Hadoop-based implementation of the LSH-based similarity join algorithms as well as a Spark-based version. The experiments show that the SimilarityJoin pattern: (1) offers a significant performance improvement for small and medium datasets; (2) is competitive also for computations using large input datasets producing out-of-core executions.

Cloud-based urgent computing for forest fire spread prediction

Forest fires cause every year damages to biodiversity, atmosphere, and economy activities. Forest fire simulation have improved significantly, but input data describing fire scenarios are subject to high levels of uncertainty. In this work the two-stage prediction scheme is used to adjust unknown parameters. This scheme relies on an input data calibration phase, which is carried over following a genetic algorithm strategy. The calibrated inputs are then pipelined into the actual prediction phase. This two-stage prediction scheme is leveraged by the cloud computing paradigm, which enables high level of parallelism on demand, elasticity, scalability and low-cost. In this paper, all the models designed to properly allocate cloud resources to the two-stage scheme in a performance-efficient and cost-effective way are described. This Cloud-based Urgent Computing (CuCo) architecture has been tested using, as study case, an extreme wildland fire that took place in California in 2018 (Camp Fire).

Multivariate nearest‐neighbors Gaussian processes with random covariance matrices

AbstractWe propose a non‐stationary spatial model based on a normal‐inverse‐Wishart framework, conditioning on a set of nearest‐neighbors. The model, called nearest‐neighbor Gaussian process with random covariance matrices is developed for both univariate and multivariate spatial settings and allows for fully flexible covariance structures that impose no stationarity or isotropic restrictions. In addition, the model can handle duplicate observations and missing data. We consider an approach based on integrating out the spatial random effects that allows fast inference for the model parameters. We also consider a full hierarchical approach that leverages the sparse structures induced by the model to perform fast Monte Carlo computations. Strong computational efficiency is achieved by leveraging the adaptive localized structure of the model that allows for a high level of parallelization. We illustrate the performance of the model with univariate and bivariate simulations, as well as with observations from two stationary satellites consisting of albedo measurements.

Dynamics of two neuron-like generators with memristive connection

At present one of the most urgent tasks of interdisciplinary science is the design and research of neuromorphic devices. Such devices are most often used to create systems for processing various kinds of information with algorithms similar to the data processing algorithms of the human brain or the brain of animals. The development of such neuromorphic electronics will allow computing devices and information processing systems to be built based on new principles and with a high level of parallelism [1]. Neuromorphic devices require the development of electronic components: neurons and synapses. The paper [2] proposed a phase-locked loop system with a bandpass filter in the control circuit. A more detailed study of the mathematical model of such a system has shown missing equilibrium states corresponding to the synchronization mode of the phase-locked loop system, but there are self-oscillating modes of varying complexity. Self-oscillations observed in such system are similar to spike and burst oscillation of the neuron’s membrane potential. Hardware implementation [3] of the considered neuron like generator in the form of an electronic device demonstrated the possibility of reproducing the same dynamic modes as in the mathematical model [2]. A fundamental disadvantage of the proposed model [2] and its experimental implementation [3] – the absence of an excitable mode (by excitable we mean a dynamic system with a stable equilibrium state and a periodic pseudoorbit of large amplitude, passing in the vicinity of the equilibrium state), when pulse generation would only respond to external disturbance. At the same time, the vast majority of brain neurons are in the excitable subthreshold mode, and their generation is primarily caused by presence of multiple connection. One of the tasks of this work was to modification the existing model of the neuron-like generator in order to preserve the known dynamics and add a mode of excited oscillator. While solving this problem, a modification of the neuron like generator based on the phase-locked loop system with a band-pass filter in the control circuit was proposed and implemented as an electronic circuit. The modification eliminates the basic drawback of the initial model – inability to work in the excitable mode. The new dynamic mode with the absence of self-oscillations was obtained by adding an electronically controlled switch between the low- and high-pass filters in the control loop. Existence of the excitable mode and the existence of previously known self-oscillating modes of varying complexity was demonstrated experimentally: spike, burst, and chaotic modes was confirmed [4]. Another task in this work was to explore the dynamics of two neuron-like generators with memristive coupling. A second-order memristor model based on Chua's memristor was used as a model of synaptic connection. When solving this problem, nonlinear frequency dependences of the conductivity of the memristive element were found. This dependence has the same character for self-oscillating modes of varying complexity: spike and burst. In addition, it was demonstrated synchronization of two neuron-like generators connected through a memristive element. Synchronization of two coupled neuron-like generators is interim in nature and strongly depends on the current state of the memristive element [5].

Efficient GPU parallelization of adaptive mesh refinement technique for high-order compressible solver with immersed boundary

A new, highly parallelized, adaptive mesh refinement (AMR) library, equipped with an accurate immersed boundary (IB) method for solving the compressible Navier–Stokes system is presented. The library, named ADAM, is designed to efficiently exploit modern exascale GPU-accelerated supercomputers and it is implemented with a highly modular structure in order to make easy to leverage it for a wide range of CFD applications. The structured Cartesian grids at the basis of (octree) AMR approach allows to implement very high order accurate models retaining a low computational cost and high level of parallelization. The accurate IB method coupled with efficient AMR technique enables the simulation flows with complex (possibly moving and deforming) geometries. The library is applied to the simulation of a strong shock diffraction over a solid sphere and a detailed discussion concerning the physical results and the parallel performance obtained is presented.

Optimized Implementation of Argon2 Utilizing the Graphics Processing Unit

In modern information technology systems, secure storage and transmission of personal and sensitive data are recognized as important tasks. These requirements are achieved through secure and robust encryption methods. Argon2 is an advanced cryptographic algorithm that emerged as the winner in the Password Hashing Competition (PHC), offering a concrete and secure measure. Argon2 also provides a secure mechanism against side-channel attacks and cracking attacks using parallel processing (e.g., GPU). In this paper, we analyze the existing GPU-based implementation of the Argon2 algorithm and further optimize the implementation by improving the performance of the hashing function during the computation process. The proposed method focuses on enhancing performance by distributing tasks between CPU and GPU units, reducing the data transfer cost for efficient GPU-based parallel processing. By shifting several stages from the CPU to the GPU, the data transfer cost is significantly reduced, resulting in faster processing times, particularly when handling a larger number of passwords and higher levels of parallelism. Additionally, we optimize the utilization of the GPU’s shared memory, which enhances memory access speed, especially in the computation of the hash value generation process. Furthermore, we leverage the parallel processing capabilities of the GPU to perform efficient brute-force attacks. By computing the H function on the GPU, the proposed implementation can generate initial blocks for multiple inputs in a single operation, making brute-force attacks in an efficient way. The proposed implementation outperforms existing methods, especially when processing a larger number of passwords and operating at higher levels of parallelism.

Improving Seed-Based FPGA Packing with Indirect Connection for Realization of Neural Networks

FPGAs are gaining favor among researchers in fields including artificial intelligence and big data due to their configurability and high level of parallelism. As the packing methods indisputably affect the implementation performance of FPGA chips, packing techniques play an important role in the design automation flow of FPGAs. In this paper, we propose a quantitative rule for packing priority of neural network circuits, and optimize the traditional seed-based packing methods with special primitives. The experiment result indicates that the proposed packing method achieves an average decrease of 8.45% in critical path delay compared to the VTR8.0 on Koios deep learning benchmarks.

A Novel NoC-Based Neural Networks Design for Implementation of A Biosensor Solution

Graphene-based nanosensors are used in many applications to monitor and survey physical or chemical phenomena. The use of Graphene improves the mechanical, electrochemical, optical, and magnetic properties of nanosensors due to its properties such as physical properties. But there are some characteristics and measurements that must be taken to manufacture such sensors. In this work, a Neural Network (NN) has been proposed that can predict the values of UV (ultraviolet) measurements, which are wavelength and absorption, based on XRD (X Ray Diffraction) measurements, namely diameter (in millimeters) and angle (theta in degrees). The proposed NN was developed on Network-on-Chip (NoC) to take advantage of the high level of parallelism and computing power provided by NoC. In addition, we used an adaptive XY routing algorithm due to its simplicity and allows exploiting multiple paths to route packets to their destination which reduces the number of lost packets due to the high injection rate without introducing any latency. Moreover, we proposed a mapping algorithm to extract maximum performance from the adaptive XY algorithm. The obtained results show the efficiency of the proposed architecture, as it allows packets to take different paths. Thus, the traffic is distributed across the network and significantly reduces packet loss. Moreover, the equal length of these paths allows avoidance of latency. HIGHLIGHTS Network-on-chip (NoC) is a powerful tool for implementing artificial neural networks, thanks to its high level of parallelism and computing power The Adaptive XY algorithm eliminates the need for a routing table and allows traffic to be distributed over the NoC The proposed mapping algorithm allows for better traffic management within the NoC, thereby enhancing performance in terms of reducing lost packets GRAPHICAL ABSTRACT

Multi-Functional Resource-Constrained Elliptic Curve Cryptographic Processor

With the rising data evolution, the demand for secured communications over networks is rising immensely. Elliptic Curve Cryptography (ECC) provides an attractive solution to fulfill the requirements of modern network applications. Many proposals published over the year over different variants of ECC satisfied some of the issues. Nevertheless, modern network applications such as Internet-of-Thing (IoT) and Software-Defined Networking (SDN) put the requirements on various aspects and can only be solved by different ECC algorithms. Looking at this point of view, an efficient architecture that could combine multiple ECC algorithms becomes an urgent request. In addition, even though many investigations of ECC on Field-Programmable Gate Arrays (FPGA), an efficient architecture that could be well-deployed on Application-Specific Integrated Circuit (ASIC) needs to get more study. Therefore, this paper proposes an area-efficient ECC hardware design that could integrate multiple ECC algorithms. The proposed design is deployed on both ASIC and FPGA platforms. Four well-known ECC-based Digital Signature Algorithms (DSAs), which are the Edwards-curve Digital Signature Algorithm (EdDSA) with Curve25519, Elliptic Curve Digital Signature Algorithm (ECDSA) with National Institute of Standards and Technology (NIST) Curve P-256, P-384, and P-521, are implemented. Furthermore, the design supports all DSA schemes: public-key generation, signature generation, and verification. We also provide optimized calculation flows for modular multiplication, modular inversion, point addition, point doubling, and Elliptic Curve Point Multiplication (ECPM) for two different elliptic curves: the NIST curve and Edward curve, on a unique architecture. The calculation processes are designed in projective coordinates and optimized in time and space to achieve a high level of parallelism. The proposed ECC processor could run up to 102-MHz on ASIC 180-nm and 109.7-MHz on Xilinx Virtex-7. In terms of area, The processor occupies 377,471 gates with 4.87- <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$mm^{2}$ </tex-math></inline-formula> on the ASIC platform and 11,401 slices on the FPGA platform. The experimental results show that our combinational design achieves area-efficient even when compared with other single-functional architecture on both ASIC and FPGA.

Accelerating Electromagnetic Field Simulations Based on Memory-Optimized CPML-FDTD with OpenACC

Although GPUs can offer higher computing power at low power consumption, their low-level programming can be relatively complex and consume programming time. For this reason, directive-based alternatives such as OpenACC could be used to specify high-level parallelism without original code modification, giving very accurate results. Nevertheless, in the FDTD method, absorbing boundary conditions are commonly used. The key to successful performance is correctly implementing the boundary conditions that play an essential role in memory use. This work accelerates the simulations of electromagnetic wave propagation that solve the Maxwell curl equations by FDTD using CMPL boundary in TE mode using OpenACC directives. A gain of acceleration optimizing the use of memory is shows, checking the loops intensities, and the use of single precision to improve the performance is also analyzed, producing an acceleration of around 5X for double precision and 11X for single precision respectively, comparing with the serial vectorized version, without introducing errors in long-term simulations. The scenarios of simulation established are common of interest and are solved at different frequencies supported by a Mid-range cards GeForce RTX 3060 and Titan RTX.

FELIX: A Ferroelectric FET Based Low Power Mixed-Signal In-Memory Architecture for DNN Acceleration

Today, a large number of applications depend on deep neural networks (DNN) to process data and perform complicated tasks at restricted power and latency specifications. Therefore, processing-in-memory (PIM) platforms are actively explored as a promising approach to improve the throughput and the energy efficiency of DNN computing systems. Several PIM architectures adopt resistive non-volatile memories as their main unit to build crossbar-based accelerators for DNN inference. However, these structures suffer from several drawbacks such as reliability, low accuracy, large ADCs/DACs power consumption and area, high write energy, and so on. In this article, we present a new mixed-signal in-memory architecture based on the bit-decomposition of the multiply and accumulate (MAC) operations. Our in-memory inference architecture uses a single FeFET as a non-volatile memory cell. Compared to the prior work, this system architecture provides a high level of parallelism while using only 3-bit ADCs. Also, it eliminates the need for any DAC. In addition, we provide flexibility and a very high utilization efficiency even for varying tasks and loads. Simulations demonstrate that we outperform state-of-the-art efficiencies with 36.5 TOPS/W and can pack 2.05 TOPS with 8-bit activation and 4-bit weight precision in an area of 4.9 mm 2 using 22 nm FDSOI technology. Employing binary operation, we obtain 1169 TOPS/W and over 261 TOPS/W/mm 2 on system level.

GraphA: An efficient ReRAM-based architecture to accelerate large scale graph processing

Graph analytics is the basis for many modern applications, e.g., machine learning and streaming data problems. With an unprecedented increase in data size of many emerging domains such as the social networks, which generate a lot of images and documents, real-time big data processing is crucial. Graph processing on traditional computer architectures faces irregular memory accesses that lead to significant data movements and waste a large amount of energy and time. ReRAM-based Processing-in-memory (PIM) is a novel technology that addresses the memory wall problem. Additionally, it provides a high parallelism level and a significant reduction in energy consumption with negligible leakage power. In this paper, we propose a ReRAM-based PIM architecture, named GraphA, which includes a novel reordering algorithm and mapping data to ReRAM Graph Engines (RGE) that cause RGEs to be used with high utilization. Furthermore, we present a compressed format, a memory layout, and proper graph partitioning for graph traversal to eliminate extra communication and useless computation. Moreover, we investigate the computation patterns of graph processing to find a suitable preprocessing model for the proposed GraphA architecture based on reorganizing classified supernode graphs and offering a runtime execution that fits it. Evaluations of GraphA on various real-world graphs show an average performance enhancement and energy saving of 5.3x and 6.0x, respectively, compared with the state-of-the-art GraphR architecture.

A Deep Neural Network Accelerator using Residue Arithmetic in a Hybrid Optoelectronic System

The acceleration of Deep Neural Networks (DNNs) has attracted much attention in research. Many critical real-time applications benefit from DNN accelerators but are limited by their compute-intensive nature. This work introduces an accelerator for Convolutional Neural Network (CNN) , based on a hybrid optoelectronic computing architecture and residue number system (RNS) . The RNS reduces the optical critical path and lowers the power requirements. In addition, the wavelength division multiplexing (WDM) allows high-speed operation at the system level by enabling high-level parallelism. The proposed RNS compute modules use one-hot encoding, and thus enable fast switching between the electrical and optical domains. We propose a new architecture that combines residue electrical adders and optical multipliers as the matrix-vector multiplication unit. Moreover, we enhance the implementation of different CNN computational kernels using WDM-enabled RNS based integrated photonics. The area and power efficiency of the proposed accelerator are 0.39 TOPS/s/mm 2 and 3.22 TOPS/s/W, respectively. In terms of computation capability, the proposed chip is 12.7× and 4.02× better than other optical implementation and memristor implementation, respectively. Our experimental evaluation using DNN benchmarks illustrates that our architecture can perform on average more than 72 times faster than GPU under the same power budget.

HPC-enabling technologies for high-fidelity combustion simulations

With the increase in computational power in the last decade and the forthcoming Exascale supercomputers, a new horizon in computational modelling and simulation is envisioned in combustion science. Considering the multiscale and multiphysics characteristics of turbulent reacting flows, combustion simulations are considered as one of the most computationally demanding applications running on cutting-edge supercomputers. Exascale computing opens new frontiers for the simulation of combustion systems as more realistic conditions can be achieved with high-fidelity methods. However, an efficient use of these computing architectures requires methodologies that can exploit all levels of parallelism. The efficient utilization of the next generation of supercomputers needs to be considered from a global perspective, that is, involving physical modelling and numerical methods with methodologies based on High-Performance Computing (HPC) and hardware architectures. This review introduces recent developments in numerical methods for large-eddy simulations (LES) and direct-numerical simulations (DNS) to simulate combustion systems, with focus on the computational performance and algorithmic capabilities. Due to the broad scope, a first section is devoted to describe the fundamentals of turbulent combustion, which is followed by a general description of state-of-the-art computational strategies for solving these problems. These applications require advanced HPC approaches to exploit modern supercomputers, which is addressed in the third section. The increasing complexity of new computing architectures, with tightly coupled CPUs and GPUs, as well as high levels of parallelism, requires new parallel models and algorithms exposing the required level of concurrency. Advances in terms of dynamic load balancing, vectorization, GPU acceleration and mesh adaptation have permitted to achieve highly-efficient combustion simulations with data-driven methods in HPC environments. Therefore, dedicated sections covering the use of high-order methods for reacting flows, integration of detailed chemistry and two-phase flows are addressed. Final remarks and directions of future work are given at the end.

Dynamic Memory Management in Massively Parallel Systems: A Case on GPUs.

Due to the high level of parallelism, there are unique challenges in developing system software on massively parallel hardware such as GPUs. One such challenge is designing a dynamic memory allocator whose task is to allocate memory chunks to requesting threads at runtime. State-of-the-art GPU memory allocators maintain a global data structure holding metadata to facilitate allocation/deallocation. However, the centralized data structure can easily become a bottleneck in a massively parallel system. In this paper, we present a novel approach for designing dynamic memory allocation without a centralized data structure. The core idea is to let threads follow a random search procedure to locate free pages. Then we further extend to more advanced designs and algorithms that can achieve an order of magnitude improvement over the basic idea. We present mathematical proofs to demonstrate that (1) the basic random search design achieves asymptotically lower latency than the traditional queue-based design and (2) the advanced designs achieve significant improvement over the basic idea. Extensive experiments show consistency to our mathematical models and demonstrate that our solutions can achieve up to two orders of magnitude improvement in latency over the best-known existing solutions.

Towards Efficient Sparse Matrix Vector Multiplication on Real Processing-In-Memory Architectures

Several manufacturers have already started to commercialize near-bank Processing-In-Memory (PIM) architectures, after decades of research efforts. Near-bank PIM architectures place simple cores close to DRAM banks. Recent research demonstrates that they can yield significant performance and energy improvements in parallel applications by alleviating data access costs. Real PIM systems can provide high levels of parallelism, large aggregate memory bandwidth and low memory access latency, thereby being a good fit to accelerate the Sparse Matrix Vector Multiplication (SpMV) kernel. SpMV has been characterized as one of the most significant and thoroughly studied scientific computation kernels. It is primarily a memory-bound kernel with intensive memory accesses due its algorithmic nature, the compressed matrix format used, and the sparsity patterns of the input matrices given. This paper provides the first comprehensive analysis of SpMV on a real-world PIM architecture, and presents SparseP, the first SpMV library for real PIM architectures. We make two key contributions. First, we design efficient SpMV algorithms to accelerate the SpMV kernel in current and future PIM systems, while covering a wide variety of sparse matrices with diverse sparsity patterns. Second, we provide the first comprehensive analysis of SpMV on a real PIM architecture. Specifically, we conduct our rigorous experimental analysis of SpMV kernels in the UPMEM PIM system, the first publicly-available real-world PIM architecture. Our extensive evaluation provides new insights and recommendations for software designers and hardware architects to efficiently accelerate the SpMV kernel on real PIM systems. For more information about our thorough characterization on the SpMV PIM execution, results, insights and the open-source SparseP software package [21], we refer the reader to the full version of the paper [3, 4]. The SparseP software package is publicly and freely available at https://github.com/CMU-SAFARI/SparseP.

Stochastic-HD: Leveraging Stochastic Computing on the Hyper-Dimensional Computing Pipeline.

Brain-inspired Hyper-dimensional(HD) computing is a novel and efficient computing paradigm. However, highly parallel architectures such as Processing-in-Memory(PIM) are bottle-necked by reduction operations required such as accumulation. To reduce this bottle-neck of HD computing in PIM, we present Stochastic-HD that combines the simplicity of operations in Stochastic Computing (SC) with the complex task solving capabilities of the latest HD computing algorithms. Stochastic-HD leverages deterministic SC, which enables all of HD operations to be done as highly parallel bitwise operations and removes all reduction operations, thus improving the throughput of PIM. To this end, we propose an in-memory hardware design for Stochastic-HD that exploits its high level of parallelism and robustness to approximation. Our hardware uses in-memory bitwise operations along with associative memory-like operations to enable a fast and energy-efficient implementation. With Stochastic-HD, we were able to reach a comparable accuracy with the Baseline-HD. Furthermore, by proposing an integrated Stochastic-HD retraining approach Stochastic-HD is able to reduce the accuracy loss to just 0.3%. We additionally accelerate the retraining process in our hardware design to create an end-to-end accelerator for Stochastic-HD. Finally, we also add support for HD Clustering to Stochastic-HD, which is the first to map the HD Clustering operations to the stochastic domain. As compared to the best PIM design for HD, Stochastic-HD is also 4.4% more accurate and 43.1× more energy-efficient.

Optimized implementation of an improved KNN classification algorithm using Intel FPGA platform: Covid-19 case study

The improved k-nearest neighbor (KNN) algorithm based on class contribution and feature weighting (DCT-KNN) is a highly accurate approach. However, it requires complex computational steps which consumes much time for the classification process. A field programmable gate array (FPGA) can be used to solve this drawback. However, using traditional hardware description language (HDL) to implement FPGA-based accelerators requires a high design time. Fortunately, the open computing language (OpenCL) high level parallel programming tool allows rapid and effective design on FPGA-based hardware accelerators. In this study, OpenCL has been used to examine speeding up the DCT-KNN algorithm on the FPGA parallel computing platform through applying numerous parallelization and optimization techniques. The optimized approach of the improved KNN could be used in various engineering problems that require a high speed of classification process. Classification of the COVID-19 disease is the case study used to examine this work. The experimental results show that implementing the DCT-KNN algorithm on the FPGA platform (Intel De5a-net Arria-10 device was used) gives an extremely high performance when compared to the traditional single-core-CPU based implementation. The execution time for our optimized design on the FPGA accelerator is 44 times faster than the conventional design implemented on the regular CPU-based computational platform.