Parallel Computer Architectures Research Articles

A Hybrid Parallel Computing Architecture Based on CNN and Transformer for Music Genre Classification

Music genre classification (MGC) is the basis for the efficient organization, retrieval, and recommendation of music resources, so it has important research value. Convolutional neural networks (CNNs) have been widely used in MGC and achieved excellent results. However, CNNs cannot model global features well due to the influence of the local receptive field; these global features are crucial for classifying music signals with temporal properties. Transformers can capture long-range dependencies within an image thanks to adopting the self-attention mechanism. Nevertheless, there are still performance and computational cost gaps between Transformers and existing CNNs. In this paper, we propose a hybrid architecture (CNN-TE) based on CNN and Transformer encoder for MGC. Specifically, we convert the audio signals into mel spectrograms and feed them into a hybrid model for training. Our model employs a CNN to initially capture low-level and localized features from the spectrogram. Subsequently, these features are processed by a Transformer encoder, which models them globally to extract high-level and abstract semantic information. This refined information is then classified using a multi-layer perceptron. Our experiments demonstrate that this approach surpasses many existing CNN architectures when tested on the GTZAN and FMA datasets. Notably, it achieves these results with fewer parameters and a faster inference speed.

Nanowatt all-optical 3D perception for mobile robotics.

Three-dimensional (3D) perception is vital to drive mobile robotics' progress toward intelligence. However, state-of-the-art 3D perception solutions require complicated postprocessing or point-by-point scanning, suffering computational burden, latency of tens of milliseconds, and additional power consumption. Here, we propose a parallel all-optical computational chipset 3D perception architecture (Aop3D) with nanowatt power and light speed. The 3D perception is executed during the light propagation over the passive chipset, and the captured light intensity distribution provides a direct reflection of the depth map, eliminating the need for extensive postprocessing. The prototype system of Aop3D is tested in various scenarios and deployed to a mobile robot, demonstrating unprecedented performance in distance detection and obstacle avoidance. Moreover, Aop3D works at a frame rate of 600 hertz and a power consumption of 33.3 nanowatts per meta-pixel experimentally. Our work is promising toward next-generation direct 3D perception techniques with light speed and high energy efficiency.

An open-source parallel EMT simulation framework

As the integration level of inverter-based resources (IBRs) increases, ensuring the reliable operation of the bulk power systems requires the use of electromagnetic transient (EMT) simulation tools to identify and mitigate system-wide stability risks. Conducting EMT studies for large-scale, IBR-rich grids, however, is challenging due to the inherent computational bottleneck caused by the underlying high-fidelity models and required small time steps. This paper introduces ParaEMT: an open-source, generic EMT simulation framework designed to accelerate simulations by leveraging advanced parallel computational technologies, such as high-performance computers. This paper presents a comprehensive exposition of ParaEMT, covering its modeling library, simulation strategy, framework structure, operational procedures, and auxiliary features, alongside its extensible parallel computational architecture. Notably, ParaEMT is a publicly accessible and modularized framework written in Python, thereby facilitating future development and the integration of new models and algorithms. The accuracy and efficiency of ParaEMT are demonstrated by rigorous validations via multiple case studies.

Minimum Linear Arrangement of Generalized Sierpinski Graphs

The creation of scientific supercomputers is one of the most pressing issues confronting technology today. Experts in computer science anticipate that future supercomputers will be built on large-scale parallel processing. A system with multiple processors and memories will be used in such a computer. The interconnection network that allows communication between the system’s processors and memories is a critical component of such systems. In the topic of interconnection networks for parallel computer architectures, graph embedding problems have grown in relevance. Network embedding has been recognized as a valuable method for developing efficient algorithms and simulating various architectures in parallel and distributed computing. In this paper, we obtain the maximum subgraph of the generalized Sierpinski graphs $S(n, m), n\geq 2, m\geq 3$, and calculate the minimum linear arrangement of generalized Sierpinski graphs by graph embeddings.

Parallel-in-time optimization of induction motors

Parallel-in-time (PinT) methods were developed to accelerate time-domain solution of evolutionary problems using modern parallel computer architectures. In this paper we incorporate one of the efficient PinT approaches, in particular, the asynchronous truncated multigrid-reduction-in-time algorithm, into a bound constrained optimization procedure applied to an induction machine. Calculation of an optimal motor geometry with respect to its efficiency in the steady state is thus parallelized at each iteration of the optimization algorithm. As a result, a more efficient motor model is obtained about 11 times faster compared to optimization using the standard sequential time stepping.

A massively parallel implementation of multilevel Monte Carlo for finite element models

The Multilevel Monte Carlo (MLMC) method has proven to be an effective variance-reduction statistical method for Uncertainty Quantification (UQ) in Partial Differential Equation (PDE) models, combining model computations at different levels to create an accurate estimate. Still, the computational complexity of the resulting method is extremely high, particularly for 3D models, which requires advanced algorithms for the efficient exploitation of High Performance Computing (HPC). In this article we present a new implementation of the MLMC in massively parallel computer architectures, exploiting parallelism within and between each level of the hierarchy. The numerical approximation of the PDE is performed using the finite element method but the algorithm is quite general and could be applied to other discretization methods. The two key ingredients of the implementation are a good processor partition scheme together with a good scheduling algorithm to assign work to different processors. We introduce a multiple partition of the set of processors that permits the simultaneous execution of different levels and we develop a dynamic scheduling algorithm to exploit it. The problem of finding the optimal scheduling of distributed tasks in a parallel computer is an NP-complete problem. We propose and analyze a new greedy scheduling algorithm to assign samples and we show that it is a 2-approximation, which is the best that may be expected under general assumptions. On top of this result we design a distributed memory implementation using the Message Passing Interface (MPI) standard. Finally we present a set of numerical experiments illustrating its scalability properties.

A parallel computing architecture based on cellular automata for hydraulic analysis of water distribution networks

Water distribution networks (WDNs) are one of the largest infrastructures in society. Various methods for formulation and hydraulic analysis of water distribution networks, including numerical and non-numerical methods, have been previously proposed. Due to the complexity, the nonlinearity of the hydraulic equations of water distribution networks, and the need for multiple executions and uncertainties in parameters, solving the hydraulic model of water distribution networks has high time complexity. In this paper, a parallel computational architecture based on the concept of cellular automata is proposed to accelerate the numerical solution of the steady-state water distribution network model. Taylor series is proposed to solve hydraulic equations. The presented architecture was implemented as a parallel hardware platform on a field-programmable gate array. The performance of the proposed method was compared with EPANET software for networks with different complexities and topologies. The results show that the proposed parallel algorithm can accelerate the hydraulic analysis of regular water distribution networks up to 700 times and 250 times for small and large networks, respectively.

Multi-Tier Hybrid Offloading for Computation-Aware IoT Applications in Civil Aircraft-Augmented SAGIN

Satellites and civil aircrafts (CAs) with computing ability are valuable access platforms, making it possible for Internet of Things (IoT) devices to offload their computation-intensive tasks in remote areas without network infrastructures. Unlike existing works mainly focused on the static scenarios or the interaction between any two types of local, edge and cloud nodes, we propose an innovative multi-tier hybrid parallel computation architecture in CA-augmented space-air-ground integrated networks (CAA-SAGIN). Specifically, devices perform local computing, CAs and satellites act as edge servers, and ground stations of satellite networks operate cloud computing. Aiming to minimize the weighted sum of end-to-end (E2E) delay and energy consumption, we formulate a partial computation offloading problem by jointly considering access strategy, transmit power, computing resource allocation, offloading ratio and delay tolerance. The platform selection exists both within and between layers, and there are inner- and inter-coupling relationships between communication and computing resources. The issue is solved by the proposed multi-tier partial task offloading (MPTO) algorithm. The original problem is firstly decomposed into primal and master subproblems by generalized benders decomposition (GBD) method, and parallel successive convex approximation (SCA) theory is utilized to transform the multi-variable NP-hard master problem into a convex one. Simulation results demonstrate the convergence and optimality of the MPTO algorithm and the advantages of this multi-tier hybrid computation offloading system. Also, the optimal tradeoff between E2E delay and energy consumption can be achieved by the MPTO algorithm.

Capacitated production planning by parallel genetic algorithm for a multi-echelon and multi-site TFT-LCD panel manufacturing supply chain

Capacitated production planning of thin-film transistor liquid crystal display (TFTLCD) manufacturing is challenging to optimize owing to its complex sequential operations of array, cell and module plants, which constitute a multi-echelon supply chain. To address this issue, this study proposes a capacitated production planning model for such multi-echelon and multi-site supply chain in the TFTLCD manufacturing industry. Major domain characteristics, such as simultaneous allocation for capacity and transportation, multiple simultaneous resources, glass substrate slicing economics, capacity transformation, and the trade-off between inventories and backorders, are incorporated to maximize the profit of supply chain. To deal the high problem complexity, a parallel programming-based genetic algorithm accompanied with a repair operator is developed to gain the result rapidly. Experiment results indicate that the proposed algorithm outperforms the other benchmark algorithms. The impact of demand uncertainty is also investigated to quantify the information value of the demand. The contribution of the present study lies on the extension of existing literature by proposing a generalized mathematical model for the capacitated production planning problem in the multi-echelon TFTLCD-PM SC, with consideration of key characteristics in such industry. Further, this study improved a regular GA by designing a repair operator and employing a parallel computation architecture to solve a large-scale problem in finite time.

Molecular Simulation Approaches to the Study of Thermotropic and Lyotropic Liquid Crystals

Over the last decade, the availability of computer time, together with new algorithms capable of exploiting parallel computer architectures, has opened up many possibilities in molecularly modelling liquid crystalline systems. This perspective article points to recent progress in modelling both thermotropic and lyotropic systems. For thermotropic nematics, the advent of improved molecular force fields can provide predictions for nematic clearing temperatures within a 10 K range. Such studies also provide valuable insights into the structure of more complex phases, where molecular organisation may be challenging to probe experimentally. Developments in coarse-grained models for thermotropics are discussed in the context of understanding the complex interplay of molecular packing, microphase separation and local interactions, and in developing methods for the calculation of material properties for thermotropics. We discuss progress towards the calculation of elastic constants, rotational viscosity coefficients, flexoelectric coefficients and helical twisting powers. The article also covers developments in modelling micelles, conventional lyotropic phases, lyotropic phase diagrams, and chromonic liquid crystals. For the latter, atomistic simulations have been particularly productive in clarifying the nature of the self-assembled aggregates in dilute solution. The development of effective coarse-grained models for chromonics is discussed in detail, including models that have demonstrated the formation of the chromonic N and M phases.

The cycle-structure connectivity of crossed cubes

Crossed cube is a kind of network structure as a basis for distributed memory parallel computer architecture. Connectivity plays an important role in measuring the fault-tolerance of an interconnection network. As a generalization, Lin et al. [6] introduced the concept of structure connectivity in 2016. For connected graphs G and T, the T-structure connectivity κ(G;T) of G is the minimum cardinality of a set of subgraphs F of G that each member is isomorphic to T so that G−F is a disconnected or a trivial graph. In this paper, we study cycle-structure connectivity of n-dimensional crossed cubes CQn. We show that κ(CQn;C2k−1)=⌈nk−1⌉ for 4⩽k⩽n and κ(CQn;C2k)=⌊nk⌋+1 for even k with 4⩽k⩽n. Furthermore, we also obtain that κ(CQn;C6)=⌊n4⌋+⌈n−3⌊n4⌋2⌉ for n⩾4.

Graphics processing unit accelerated lattice Boltzmann method simulations of dilute gravity currents

Lattice Boltzmann method models offer a novel framework for the simulation of high Reynolds number dilute gravity currents. The numerical algorithm is well suited to acceleration via implementation on massively parallel computer architectures. Here, we present two lattice Boltzmann method models of lock-exchange dilute gravity currents in which the largest turbulent length scales are directly resolved. The three-dimensional simulations are accelerated by exporting computations to a graphics processing unit and are validated against experiments and high-resolution simulations for Reynolds numbers up to 30 000. The lattice Boltzmann method models achieve equivalent accuracy to conventional large-eddy simulation models in the prediction of key flow properties. A conservative analysis of computational performance relative to conventional methods indicates that the presented framework reduces simulation times by two orders of magnitude. Therefore, it can be used as a foundation for the development of depth-resolving models that capture more of the complexity of environmental gravity currents.

Isotropic finite-difference approximations for phase-field simulations of polycrystalline alloy solidification

Phase-field models of microstructural pattern formation during alloy solidification are commonly solved numerically using the finite-difference method, which is ideally suited to carry out computationally efficient simulations on massively parallel computer architectures such as Graphic Processing Units. However, one known drawback of this method is that the discretization of differential terms involving spatial derivatives introduces a spurious lattice anisotropy that can influence the solid-liquid interface dynamics. We find that this influence is significant for the case of polycrystalline dendritic solidification, where the crystal axes of different grains do not generally coincide with the reference axes of the finite-difference lattice. In particular, we find that with the commonly used finite-difference implementation of the quantitative phase-field model of binary alloy solidification, both the operating state of the dendrite tip and the dendrite growth orientation are strongly affected by the lattice anisotropy. To circumvent this problem, we use known methods in both real and Fourier space to derive finite-difference approximations of leading differential terms in 2D and 3D that are isotropic at order h2 of the lattice spacing h. Importantly, those terms include the divergence of the anti-trapping current that is found to have a critical influence on pattern selection. The 2D and 3D discretizations use an approximated form of the anti-trapping current that facilitates the Fourier-space derivation of the associated isotropic differential operator at O(h2), but we also derive a 2D discretization of the standard form of this current. Finally, we present 2D and 3D phase-field simulations of alloy solidification, showing that the isotropic finite-difference implementations dramatically reduce spurious lattice anisotropy effects, yielding both the tip operating state and growth direction of the dendrite that are nearly independent of the angle between the crystal and finite-difference lattice axes.

Toward an Automated-Algebra Framework for High Orders in the Virial Expansion of Quantum Matter

The virial expansion provides a non-perturbative view into the thermodynamics of quantum many-body systems in dilute regimes. While powerful, the expansion is challenging as calculating its coefficients at each order n requires analyzing (if not solving) the quantum n-body problem. In this work, we present a comprehensive review of automated algebra methods, which we developed to calculate high-order virial coefficients. The methods are computational but non-stochastic, thus avoiding statistical effects; they are also for the most part analytic, not numerical, and amenable to massively parallel computer architectures. We show formalism and results for coefficients characterizing the thermodynamics (pressure, density, energy, static susceptibilities) of homogeneous and harmonically trapped systems and explain how to generalize them to other observables such as the momentum distribution, Tan contact, and the structure factor.

Ultralow‐Power Synaptic Transistors Based on Ta2O5/Al2O3 Bilayer Dielectric for Algebraic Arithmetic

AbstractMultifarious artificial synaptic devices are extensively proposed in the field of neuromorphic hardware systems for their applicability in promising parallel computer architecture, which is preferred to classical Von Neumann architecture in numerous and complex information processing. Besides the ability to mimic typical biological synaptic behaviors, low power consumption is critical for the synaptic devices in the neuromorphic hardware system.In this paper, ultralow‐power Ta2O5/Al2O3 bilayer‐gate‐dielectric synaptic transistors (TABSTs) with low‐temperature atomic layer deposited dielectric are proposed. The TABSTs show power consumption as low as 19.9 aJ per synaptic event successfully at a low drain voltage of 0.001 V and a short pulse width of 1 ms. Essential synaptic behaviors including excitatory postsynaptic current, inhibitory postsynaptic current, spike‐amplitude‐dependent plasticity, spike‐duration‐dependent plasticity, paired pulse facilitation, long‐term potentiation, long‐term depression, the transition from short‐term memory to long‐term memory as well as learning and forgetting abilities are well mimicked by the TABSTs. Moreover, algebraic arithmetic operations such as addition, subtraction, multiplication, and division are also implemented by the TABSTs. This work provides a promising approach to emerging neuromorphic systems.

Parallel Discrete Convolutions on Adaptive Particle Representations of Images.

We present data structures and algorithms for native implementations of discrete convolution operators over Adaptive Particle Representations (APR) of images on parallel computer architectures. The APR is a content-adaptive image representation that locally adapts the sampling resolution to the image signal. It has been developed as an alternative to pixel representations for large, sparse images as they typically occur in fluorescence microscopy. It has been shown to reduce the memory and runtime costs of storing, visualizing, and processing such images. This, however, requires that image processing natively operates on APRs, without intermediately reverting to pixels. Designing efficient and scalable APR-native image processing primitives, however, is complicated by the APR's irregular memory structure. Here, we provide the algorithmic building blocks required to efficiently and natively process APR images using a wide range of algorithms that can be formulated in terms of discrete convolutions. We show that APR convolution naturally leads to scale-adaptive algorithms that efficiently parallelize on multi-core CPU and GPU architectures. We quantify the speedups in comparison to pixel-based algorithms and convolutions on evenly sampled data. We achieve pixel-equivalent throughputs of up to 1TB/s on a single Nvidia GeForce RTX 2080 gaming GPU, requiring up to two orders of magnitude less memory than a pixel-based implementation.

Genetic programming for iterative numerical methods

We introduce GPLS (Genetic Programming for Linear Systems) as a GP system that finds mathematical expressions defining an iteration matrix. Stationary iterative methods use this iteration matrix to solve a system of linear equations numerically. GPLS aims at finding iteration matrices with a low spectral radius and a high sparsity, since these properties ensure a fast error reduction of the numerical solution method and enable the efficient implementation of the methods on parallel computer architectures. We study GPLS for various types of system matrices and find that it easily outperforms classical approaches like the Gauss–Seidel and Jacobi methods. GPLS not only finds iteration matrices for linear systems with a much lower spectral radius, but also iteration matrices for problems where classical approaches fail. Additionally, solutions found by GPLS for small problem instances show also good performance for larger instances of the same problem.

Porting hypre to heterogeneous computer architectures: Strategies and experiences

Linear systems are occurring in many applications, and solving them can take a large amount of the total simulation time. The high performance library hypre provides a variety of interfaces and linear solvers, including various multigrid methods, that have achieved good scalability on a variety of homogeneous parallel computer architectures. Heterogeneous architectures with nodes that have both CPUs and accelerators provide new challenges, since they require more fine-grained parallelism and reduced data movement between different memories on a single node as well as across nodes. We will discuss our experiences and strategies to port hypre to heterogeneous computers with accelerators, including the design of a new memory model, the use of abstractions, the BoxLoop macros in the structured and semi-structured interfaces, and the restructuring of algebraic multigrid (AMG) into modular components. We present numerical experiments comparing CPU and GPU performance for several test problems.

Implementation of Relativistic Coupled Cluster Theory for Massively Parallel GPU-Accelerated Computing Architectures.

In this paper, we report reimplementation of the core algorithms of relativistic coupled cluster theory aimed at modern heterogeneous high-performance computational infrastructures. The code is designed for parallel execution on many compute nodes with optional GPU coprocessing, accomplished via the new ExaTENSOR back end. The resulting ExaCorr module is primarily intended for calculations of molecules with one or more heavy elements, as relativistic effects on the electronic structure are included from the outset. In the current work, we thereby focus on exact two-component methods and demonstrate the accuracy and performance of the software. The module can be used as a stand-alone program requiring a set of molecular orbital coefficients as the starting point, but it is also interfaced to the DIRAC program that can be used to generate these. We therefore also briefly discuss an improvement of the parallel computing aspects of the relativistic self-consistent field algorithm of the DIRAC program.

Program diagramming and fundamental programming patterns for a polymorphic computing dataflow processor

In this era where the growth of computer performance is significantly flattening, new parallel computer architectures are needed to continue to fuel that growth. One candidate architecture to fuel this growth is a recently proposed dataflow processor that is meant to be tiled together into a fabric to create a polymorphic computer (i.e. a reconfigurable computer whose architecture can be changed to match the software) where program instructions can be independently migrated in a fine-grained manner around the fabric to logically change the computer architecture. This work provides a summary of the dataflow processor’s instruction set and details how to write programs for this processor. The techniques that are presented to write programs for this processor are diagramming and 12 building block programming patterns. The 12 patterns are: Operations with a Constant, Simple Copy, Pitch (Discard), Counters, Critical Sections, Funnels (Gathers), Distribution Tree (Scatters), Lookup Tables, Single Value Selection, Single Value Steering, Switch (Multi-Value Steering), and Loops. These patterns are expected to benefit future compilers. In addition, the ideal timing results of several test algorithms that are implemented in this processor and that utilize the above patterns are presented. • Background summary of a polymorphic computing dataflow processor’s instruction set • Program diagramming methods for a polymorphic computing dataflow processor • 12 Programming patterns in a polymorphic computing dataflow processor • Ideal timing results of several test algorithms in polymorphic computing fabric