Massive Parallel Computing Research Articles

All‐Solid‐State Vertical Three‐Terminal N‐Type Organic Synaptic Devices for Neuromorphic Computing

AbstractArtificial synaptic devices are the basic composition units for neuromorphic computing processors that realize massive parallel computing. However, the n‐type organic transistors have failed to achieve good performance as an artificial synaptic device for neuromorphic computing until now. Here, a vertical three‐terminal n‐type organic artificial synapse (TNOAS) using a lithium ion‐based organic dielectric and the n‐type donor–acceptor (D–A) conjugated polymer‐naphthalene‐1,4,5,8‐tetracarboxylic‐diimide‐thiophene‐vinyl‐thiophene (NDI‐gTVT) as the channel is proposed. The TNOAS achieves nonvolatile conductance modulation with high current density operation (≈10 KA cm−2) at low voltage and mimics the basic functions of biological synapses, such as long‐term synaptic plasticity and paired‐pulse facilitation. The minimum energy consumption of a response event triggered by a single action potential is 6.16 pJ, which can be comparable with p‐type counterparts. Moreover, simulation using handwritten digital datasets exhibit a high recognition accuracy of 94%.

Equivalence classes and conditional hardness in massively parallel computations

The Massively Parallel Computation (MPC) model serves as a common abstraction of many modern large-scale data processing frameworks, and has been receiving increasingly more attention over the past few years, especially in the context of classical graph problems. So far, the only way to argue lower bounds for this model is to condition on conjectures about the hardness of some specific problems, such as graph connectivity on promise graphs that are either one cycle or two cycles, usually called the one cycle versus two cycles problem. This is unlike the traditional arguments based on conjectures about complexity classes (e.g., textsf {P}ne textsf {NP}), which are often more robust in the sense that refuting them would lead to groundbreaking algorithms for a whole bunch of problems. In this paper we present connections between problems and classes of problems that allow the latter type of arguments. These connections concern the class of problems solvable in a sublogarithmic amount of rounds in the MPC model, denoted by textsf {MPC}(o(log N)), and the standard space complexity classes textsf {L} and textsf {NL}, and suggest conjectures that are robust in the sense that refuting them would lead to many surprisingly fast new algorithms in the MPC model. We also obtain new conditional lower bounds, and prove new reductions and equivalences between problems in the MPC model. Specifically, our main results are as follows.Lower bounds conditioned on the one cycle versus two cycles conjecture can be instead argued under the textsf {L}nsubseteq textsf {MPC}(o(log N)) conjecture: these two assumptions are equivalent, and refuting either of them would lead to o(log N)-round MPC algorithms for a large number of challenging problems, including list ranking, minimum cut, and planarity testing. In fact, we show that these problems and many others require asymptotically the same number of rounds as the seemingly much easier problem of distinguishing between a graph being one cycle or two cycles.Many lower bounds previously argued under the one cycle versus two cycles conjecture can be argued under an even more robust (thus harder to refute) conjecture, namely textsf {NL}nsubseteq textsf {MPC}(o(log N)). Refuting this conjecture would lead to o(log N)-round MPC algorithms for an even larger set of problems, including all-pairs shortest paths, betweenness centrality, and all aforementioned ones. Lower bounds under this conjecture hold for problems such as perfect matching and network flow.

A High Static Performance Hierarchical Three-Dimensional Shifted Completely Connected Network

In recent years, there has been a significant shift toward the adoption of the “Internet of Things (IoT)” era, in which vast amounts of data are collected and processed, and Artificial intelligence (AI) is used to make critical choices. When acting in real-time, it is vital to use devices with a lot of computing power. MPC (Massively Parallel Computer) Two-dimensional (2D) design systems are the most powerful computers available. Unfortunately, they can’t meet the need for the advanced computing operation required by many applications. As a result, it is critical to invest in developing high operational capacity systems that will meet the current computational power consumption demands. This paper developed the architecture of the two-dimensional Shifted Completely Connected Network (2D-SCCN) and proposed a three-dimensional (3D) network to achieve greater capacity and performance. The shortest path protocol on three-dimensional Shifted Completely Connected Network (3D- SCCN) was established, tested by computer simulator, and compared based on “shortest diameter”, “shortest average distance”, “lowest cost”, “moderate bisection width”, and “high arc-connectivity” among other characteristics. As a result, “3D-SCCN” was the most efficient method of transmitting a message between two nodes, increasing the system’s real-time response capability.

ASUCA: The JMA Operational Non-hydrostatic Model

The non-hydrostatic numerical weather prediction (NWP) model ASUCA developed by the Japan Meteorological Agency (JMA) was launched into operation as 2 and 5 km-resolution regional models in 2015 and 2017, respectively. This paper outlines specifications of ASUCA with focus on the dynamical core and its configuration/accuracy as an operational model. ASUCA is designed for high computational stability and efficiency, mass conservation and forecast accuracy. High computational stability is achieved via a time-split integration scheme to compute acoustic terms and an advection scheme with a flux-limiter function to avoid numerical oscillation. In addition, vertical advection and sedimentation are calculated together with another exclusive time-splitting technique. ASUCA adopts hybrid parallelization using Message Passing Interface (MPI) and Open Multi Processing (OpenMP) for high computational efficiency on massive parallel scalar computers. The three-dimensional arrays are allocated such that the vertical direction is the stride-one innermost dimension to make effective use of cache and multi-thread parallelization. This is particularly advantageous for physical processes evaluated in a vertical column. To ensure mass conservation, density rather than pressure is integrated as a prognostic variable in flux-form fully compressible governing equations. ASUCA exhibited better performance than the previous operational model in idealized and NWP tests.

Interactive High-Resolution Simulation of Granular Material

We introduce a particle-based simulation method for granular material in interactive frame rates. We divide the simulation into two decoupled steps. In the first step, a relatively small number of particles is accurately simulated with a constraint-based method. Here, all collisions and the resulting friction between the particles are taken into account. In the second step, the small number of particles is significantly increased by an efficient sampling algorithm without creating additional artifacts. The method is particularly robust and allows relatively large time steps, which makes it well suited for real-time applications. With our method, up to 500k particles can be computed in interactive frame rates on consumer CPUs without relying on GPU support for massive parallel computing. This makes it well suited for applications where a lot of GPU power is already needed for render tasks.

High resolution 3D ultrasonic breast imaging by time-domain full waveform inversion

Ultrasound tomography (UST) scanners allow quantitative images of the human breast’s acoustic properties to be derived with potential applications in screening, diagnosis and therapy planning. Time domain full waveform inversion (TD-FWI) is a promising UST image formation technique that fits the parameter fields of a wave physics model by gradient-based optimization. For high resolution 3D UST, it holds three key challenges: firstly, its central building block, the computation of the gradient for a single US measurement, has a restrictively large memory footprint. Secondly, this building block needs to be computed for each of the 103–104 measurements, resulting in a massive parallel computation usually performed on large computational clusters for days. Lastly, the structure of the underlying optimization problem may result in slow progression of the solver and convergence to a local minimum. In this work, we design and evaluate a comprehensive computational strategy to overcome these challenges: firstly, we exploit a gradient computation based on time reversal that dramatically reduces the memory footprint at the expense of one additional wave simulation per source. Secondly, we break the dependence on the number of measurements by using source encoding (SE) to compute stochastic gradient estimates. Also we describe a more accurate, TD-specific SE technique with a finer variance control and use a state-of-the-art stochastic LBFGS method. Lastly, we design an efficient TD multi-grid scheme together with preconditioning to speed up the convergence while avoiding local minima. All components are evaluated in extensive numerical proof-of-concept studies simulating a bowl-shaped 3D UST breast scanner prototype. Finally, we demonstrate that their combination allows us to obtain an accurate 442 × 442 × 222 voxel image with a resolution of 0.5 mm using Matlab on a single GPU within 24 h.

A Novel Tradeoff Analysis between Traffic Congestion and Packing Density of Interconnection Networks for Massively Parallel Computers

From disaster prevention to mitigation, drug analysis to drug design, agriculture to food security, IoT to AI, and big data analysis to knowledge or sentiment mining, a high computation power is a prime necessity at present. As such, massively parallel computer (MPC) systems comprising a large number of nodes are gaining popularity. To interconnect these huge numbers of nodes efficiently, hierarchical interconnection networks are an attractive and feasible option. A Tori-connected flattened butterfly network (TFBN) has been proposed by the authors in a prior work for future generation MPC systems. In the previous study, the static network performance and static cost-effectiveness were evaluated. In this research, a novel trade-off factor named message traffic congestion vs. packing density trade-off factor has been proposed, which characterizes the message congestion in the network and its packing density. The factor is used to statically assess the suitability of the implementation of an interconnection network. The message traffic density, packing density, and new factor have been evaluated for the proposed network and similar competitive networks such as TTN, TESH, 2D-Mesh, 3D-Mesh, 2D-Torus, and 3D-Torus. It has been found that the performance of the TFBN is superior to the other networks.

Mapping Hebbian Learning Rules to Coupling Resistances for Oscillatory Neural Networks.

Oscillatory Neural Network (ONN) is an emerging neuromorphic architecture with oscillators representing neurons and information encoded in oscillator's phase relations. In an ONN, oscillators are coupled with electrical elements to define the network's weights and achieve massive parallel computation. As the weights preserve the network functionality, mapping weights to coupling elements plays a crucial role in ONN performance. In this work, we investigate relaxation oscillators based on VO2 material, and we propose a methodology to map Hebbian coefficients to ONN coupling resistances, allowing a large-scale ONN design. We develop an analytical framework to map weight coefficients into coupling resistor values to analyze ONN architecture performance. We report on an ONN with 60 fully-connected oscillators that perform pattern recognition as a Hopfield Neural Network.

Massively Parallel Computation via Remote Memory Access

We introduce the Adaptive Massively Parallel Computation (AMPC) model, which is an extension of the Massively Parallel Computation (MPC) model. At a high level, the AMPC model strengthens the MPC model by storing all messages sent within a round in a distributed data store. In the following round, all machines are provided with random read access to the data store, subject to the same constraints on the total amount of communication as in the MPC model. Our model is inspired by the previous empirical studies of distributed graph algorithms [8, 30] using MapReduce and a distributed hash table service [17]. This extension allows us to give new graph algorithms with much lower round complexities compared to the best-known solutions in the MPC model. In particular, in the AMPC model we show how to solve maximal independent set in O (1) rounds and connectivity/minimum spanning tree in O (log log m / n n rounds both using O ( n δ ) space per machine for constant δ < 1. In the same memory regime for MPC, the best-known algorithms for these problems require poly log n rounds. Our results imply that the 2-C YCLE conjecture, which is widely believed to hold in the MPC model, does not hold in the AMPC model.

Lagrangian multiscale simulation of complex flows

A general multiscale and multiphysics simulation framework for inhomogeneous viscoelastic and elastoplastic complex flows, such as nanobubble flows, blood vessel flows, or polymer composite flows, is presented for use on massive parallel computers. Our simulation methodology is based on a particle simulation of macroscopic flows where a microscopic simulator is embedded in each of the hydrodynamic particles of macroscopic flow simulations to evaluate the local stress as a function of its flow history from the microscopic point of view. We developed a platform named MSSP (MultiScale Simulation Platform for complex flows) by combining the smoothed particle hydrodynamics (SPH) method and the microscopic molecular simulators. In such a technique, the main difficulty is the large amount of computation cost due to a large number of microscopic particles (typically of the order of 109−1010), and the inhomogeneity of the flow. To solve this problem, we propose a dynamical switching of the microscopic models between realistic particle simulations and linearized constitutive relations. An appropriate boundary condition for moving boundaries is also introduced in the SPH simulations that enables us to simulate complex flows with deformable objects such as phase-separated domains or biomembranes. A benchmark test of MSSP has been done by simulating nonlinear and non-Markovian fluids passing by an obstacle, giving good quantitative agreement with experiments in the same situation.

Graph Sparsification for Derandomizing Massively Parallel Computation with Low Space

The Massively Parallel Computation (MPC) model is an emerging model that distills core aspects of distributed and parallel computation, developed as a tool to solve combinatorial (typically graph) problems in systems of many machines with limited space. Recent work has focused on the regime in which machines have sublinear (in n , the number of nodes in the input graph) space, with randomized algorithms presented for the fundamental problems of Maximal Matching and Maximal Independent Set. However, there have been no prior corresponding deterministic algorithms. A major challenge underlying the sublinear space setting is that the local space of each machine might be too small to store all edges incident to a single node. This poses a considerable obstacle compared to classical models in which each node is assumed to know and have easy access to its incident edges. To overcome this barrier, we introduce a new graph sparsification technique that deterministically computes a low-degree subgraph, with the additional property that solving the problem on this subgraph provides significant progress towards solving the problem for the original input graph. Using this framework to derandomize the well-known algorithm of Luby [SICOMP’86], we obtain O (log Δ + log log n )-round deterministic MPC algorithms for solving the problems of Maximal Matching and Maximal Independent Set with O ( n ɛ ) space on each machine for any constant ɛ > 0. These algorithms also run in O (log Δ) rounds in the closely related model of CONGESTED CLIQUE, improving upon the state-of-the-art bound of O (log 2 Δ) rounds by Censor-Hillel et al. [DISC’17].

Improved MPC Algorithms for Edit Distance and Ulam Distance

Edit distance is one of the most fundamental problems in combinatorial optimization to measure the similarity between strings. Ulam distance is a special case of edit distance where no character is allowed to appear more than once in a string. Recent developments have been very fruitful for obtaining fast and parallel algorithms for both edit distance and Ulam distance. In this work, we present an almost optimal MPC (massively parallel computation) algorithm for Ulam distance and improve MPC algorithms for edit distance. Our algorithm for Ulam distance is almost optimal in the sense that (1) the approximation factor of our algorithm is 1+ε1+ε, (2) the round complexity of our algorithm is constant, (3) the total memory of our algorithm is almost linear (~O <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">ε</sub> (n)Õε(n)), and (4) the overall running time of our algorithm is almost linear which is the best known for Ulam distance. We also improve the work of Hajiaghayi et al. for edit distance in terms of total memory. The best previously known MPC algorithm for edit distance requires ~O(n <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2x</sup> )Õ(n2x) machines when the memory of each machine is bounded by ~O(n <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1-x</sup> )Õ(n1-x). In this work, we improve the number of machines to ~O(n <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">(9/5)x</sup> )Õ(n(9/5)x) while keeping the memory limit intact. Moreover, the round complexity of our algorithm is constant and the total running time of our algorithm is truly subquadratic. However, our improvement comes at the expense of a constant factor in the approximation guarantee of the algorithm. This improvement is inspired by the recent techniques of Boroujeni et al. and Chakraborty et al. for obtaining truly subquadratic time algorithms for edit distance.

Neural Network Optimization Method and Its Application in Information Processing

Neural network theory is the basis of massive information parallel processing and large-scale parallel computing. Neural network is not only a highly nonlinear dynamic system but also an adaptive organization system, which can be used to describe the intelligent behavior of cognition, decision-making, and control. The purpose of this paper is to explore the optimization method of neural network and its application in information processing. This paper uses the characteristic of SOM feature map neural network to preserve the topological order to estimate the direction of arrival of the array signal. For the estimation of the direction of arrival of single-source signals in array signal processing, this paper establishes a uniform linear array and arbitrary array models based on the distance difference vector to detect DOA. The relationship between the DDOA vector and the direction of arrival angle is regarded as a mapping from the DDOA space to the AOA space. For this mapping, through derivation and analysis, it is found that there is a similar topological distribution between the two variables of the sampled signal. In this paper, the network is trained by uniformly distributed simulated source signals, and then the trained network is used to perform AOA estimation effect tests on simulated noiseless signals, simulated Gaussian noise signals, and measured signals of sound sources in the lake. Neural network and multisignal classification algorithms are compared. This paper proposes a DOA estimation method using two-layer SOM neural network and theoretically verifies the reliability of the method. Experimental research shows that when the signal-to-noise ratio drops from 20 dB to 1 dB in the experiment with Gaussian noise, the absolute error of the AOA prediction is small and the fluctuation is not large, indicating that the prediction effect of the SOM network optimization method established in this paper does not vary. The signal-to-noise ratio drops and decreases, and it has a strong ability to adapt to noise.

A 3D graphics rendering pipeline implementation based on the openCL massively parallel processing

Recently, massively-parallel computing libraries and devices are much widely used, in addition to the traditional 3D graphics systems. In this paper, we present a full 3D fixed-function graphics pipeline, based on the OpenCL, which is one of the most widely used massively-parallel computing library. The full 3D graphics features including WebGL, Web3D and others can be implemented on the massively-parallel computations, without underlying 3D graphics hardware support. Many previous works focused on another massively-parallel system of CUDA, which has a drawback of limited availability. In contrast, we designed and implemented a new architecture with OpenCL, which is now available on various computing devices, including most CPUs, GPUs, and at least theoretically, special-purpose embedded FPGAs. Our work provides full 3D graphics features on OpenCL-capable systems, without dedicated 3D graphics hardware, to finally make 3D graphics features ubiquitous. Technically, we used a top-down approach in its rendering, from the whole screen to precise pixels. At each stage, we tuned our OpenCL implementations and also their global and local parameter spaces. We present the details of our design and also the final result of our implementation, and show its correctness and efficiency.

Midimew Connected Torus Network for Next Generation Massively Parallel Computer System

Important scientific and engineering applications need high performance computing. This can be provided by massively parallel computer (MPC) systems. A sensitive step of maintaining such systems is the interconnection network used to interconnect the computing nodes. The topology used effects the network costs and performance significantly. Hierarchal interconnection networks (HIN) were introduced having several attractive features including low latency, low cost, and high fault tolerance. This paper proposes a new HIN called Midimew connect Torus Network (MTN) that provides constant node degree, high arc connectivity, high fault tolerance and a reasonable bisection width. Static network performance evaluation for the proposed MTN has been conducted and compared with other networks. The comparison included conventional topologies such as 2D Mesh and 2D Torus and hierarchal ones which are TESH and TTN. The comparison of result shows that with the cost of extra communication links MTN attains higher fault tolerance than that of TESH and mesh networks, and equal to that of TTN, and less than that of torus network. Also, the hierarchical networks such as MTN, TTN, and TESH yield moderate bisection width; and the bisection width of MTN is lower than that of mesh and torus networks. This paper shows the superiority of the proposed MTN by comparative study of static network performance.

The Static Performance Effect of Hybrid- Hierarchical Interconnection by Shifted Completely Connected Network

Massively parallel computer (MPC) systems execute many operations based on internal networks called interconnection networks. The performance of these networks is affected by their topologies.There are many topologies of interconnection networks for MPC systems, unfortunately, they faced many drawbacks. Expanding the size of the network degrades the performance of these topologies. That is why this current paper presents a hybrid-hierarchical interconnection network (HIN) topology by Shifted Completely Connected Network (SCCN) to circumvent the drawbacks of the existing topologies. An experimental evaluation involving the design and development of a hierarchical network was carried out. A two-dimensional higher level networks has been produced and its static network performance parameters evaluated through simulators. The finding of the simulations has shown some good performances compared to many previous designed networks. SCCN is better than all conventional networks in terms of diameter, cost and average distance.

Превосходство квантовой программной инженерии в интеллектуальной робототехнике

A new approach for implementing quantum massive parallel computations is presented, using methods of circuit implementation of quantum algorithmic gates. Methods for designing fast quantum operators such as superposition, entanglement, andinterference are considered. The presented methodsallow you to reduce the number of actions that must be performed. The implementation is presented as a support tool for SW&amp;HW supercomputer accelerator for modeling quantum algorithms. In particular, a newquantum-genetic and quantum-fuzzy inference algorithm for intelligent robotic control has been implemented. Also, a new method for performing Grover's inference without operations with the productis presented.

CMOS Image Sensor Featuring Current-Mode Focal-Plane Image Compression

The interest in focal-plane processing techniques, by which image processing is carried out at the pixel level, has increased since the advent of active pixel sensors in the middle 90’s. By sharing processing circuitry by a group of neighboring pixels such techniques enable high-speed imaging operation and massive parallel computation. Focal-plane image compression is particularly interesting, because it allows for further reduction in data rates. The proposed approach also benefits from processing currents rather than voltages, which not only suits current-mode APS imagers, but also enables the circuits to operate at low voltage supply levels and achieve high speed. Moreover, arithmetic computations such as additions and scaling are easily implemented in current mode. Whereas current-mode imaging architectures produce higher fixed pattern noise (FPN) figures than their voltage-mode counterparts, low FPN can be achieved by applying correlated double sampling (CDS) and gain correction techniques. This work presents a 32 × 32 gray-level imaging integrated circuit featuring focal plane image compression, such that for each 4 × 4 pixel block, analog circuits implement differential pulse-code modulation, linear transform, and vector quantization. Other processing functions implemented in the chip are CDS and A/D conversion. Theoretical details are described, as well as the test setup of the chip fabricated in a 0.35 μm CMOS process. To validate the proposed technique, experimental results and captured photographs are shown. The CMOS imager compresses captured images at 0.94 bits/pixel for an overall power consumption below 40 mW (white image), which is equivalent to approximately 36 μW per pixel. Using photographs taken from bar-target pattern inputs, it is shown that details up to 2 cycles/c mare preserved in the decoded images.

TFBN: A Cost Effective High Performance Hierarchical Interconnection Network

In order to fulfill the increasing demand for computation power to process a boundless data concurrently within a very short time or real-time in many areas such as IoT, AI, machine learning, smart grid, and big data analytics, we need exa-scale or zetta-scale computation in the near future. Thus, to have this level of computation, we need a massively parallel computer (MPC) system that shall consist of millions of nodes; and, for the interconnection of these massive numbers of nodes, conventional topologies are infeasible. Thus, a hierarchical interconnection network (HIN) is a rational way to connect huge nodes. Through this article, we are proposing a new HIN, which is a tori-connected flattened butterfly network (TFBN) for the next generation MPC system. Numerous basic modules are hierarchically interconnected as a toroidal connection, whereby the basic modules are flattened butterfly networks. We have studied the network architecture, static network performance, and static cost-effectiveness of the proposed TFBN in detail; and compared static network and cost-effectiveness performance of the TFBN to those of TTN, torus, TESH, and mesh networks. It is depicted that TFBN possesses low diameter and average distance, high arc connectivity, and temperate bisection width. It also has better cost-effectiveness and cost-performance trade-off factor compared to those of TTN, torus, TESH, and mesh networks. The only shortcoming is that the complexity of wiring of the TFBN is higher than that of those networks; this is because the basic module necessitates some extra short length link to form the flattened butterfly network. Therefore, TFBN is a high performance and cost-effective HIN, and it will be a good option for the next generation MPC system.

Breaking the linear-memory barrier in [formula omitted]: Fast [formula omitted] on trees with strongly sublinear memory

Recently, studying fundamental graph problems in the Massively Parallel Computation (MPC) framework, inspired by the MapReduce paradigm, has gained a lot of attention. An assumption common to a vast majority of approaches is to allow Ω˜(n) memory per machine, where n is the number of nodes in the graph and Ω˜ hides polylogarithmic factors. However, as pointed out by Karloff et al. [SODA'10] and Czumaj et al. [STOC'18], it might be unrealistic for a single machine to have linear or only slightly sublinear memory.In this paper, we thus study a more practical variant of the MPC model which only requires substantially sublinear or even subpolynomial memory per machine. In contrast to the linear-memory MPC model and also to streaming algorithms, in this low-memory MPC setting, a single machine will only see a small number of nodes in the graph. We introduce a new and strikingly simple technique to cope with this imposed locality.In particular, we show that the Maximal Independent Set (MIS) problem can be solved efficiently, that is, in O(log3⁡log⁡n) rounds, when the input graph is a tree. This constitutes an almost exponential speed-up over the low-memory MPC algorithm in O˜(log⁡n) rounds in a concurrent work by Ghaffari and Uitto [SODA'19] and substantially reduces the local memory from Ω˜(n) required by the recent O(log⁡log⁡n)-round MIS algorithm of Ghaffari et al. [PODC'18] to nε for any ε>0, without incurring a significant loss in the round complexity. Moreover, it demonstrates how to make use of the all-to-all communication in the MPC model to almost exponentially improve on the corresponding bound in the LOCAL and PRAM models by Lenzen and Wattenhofer [PODC'11].