Data Memory Access Research Articles

High-performance convolutional neural network emulation via FPGA-integrated memristive circuitry

Although memristors address the potential data flow and memory access bottlenecks in traditional von Neumann architectures by moving computational operations into the storage elements, thereby improving computational efficiency and reducing energy consumption, the current costs of memristors are relatively high, and they face issues such as low reliability and short lifespan. Therefore, this paper proposes a simulation circuit model based on FPGA, which can provide a low-cost solution for rapid prototyping design and exploration of the design space. The memristor simulation circuit is deployed on the Xilinx ZYNO-7000 FPGA XC7Z020 with a resource utilization rate of less than 1%, and it has been successfully integrated and verified. Furthermore, this paper also constructs a convolutional neural network for MNIST image recognition using a memristor model. With a basis of 512 parameters, it achieves an accuracy of 90.22%.

CPU and GPU oriented optimizations for LiDAR data processing

Digital Terrain Models (DTM) can be accurately obtained from clouds of LiDAR points but the corresponding cloud processing time can be prohibitive. This paper describes several optimization techniques that have been applied to the Overlap Window Method (OWM) that is a key component in DTM applications. OWM was originally implemented in R which translates into serious limitations in terms of the size of the LiDAR point cloud that can be processed. We have ported the code to C++, significantly optimized the data structure to minimize memory accesses, and developed parallel implementations for CPU and GPU commodity devices using oneAPI libraries and tools. This results in CPU and GPU versions that are up to 19x and 83x faster, respectively, than an OpenMP baseline that uses eight CPU cores. Most importantly, the proposed optimizations for CPU and GPU can be paramount to get the most out of other LiDAR-based algorithms in which the careful selection of the right data structure, parallelization strategies and memory access reduction techniques will certainly result in significant performance improvements.

Hardware acceleration of number theoretic transform for zk‐SNARK

AbstractZk‐SNARK unleashes the great potential of ZKP (zero‐knowledge proof) in the blockchain, distributed storage, and so forth. However, the proof‐generation of zk‐SNARK is excessively time intensive, making it a challenge to deploy a high‐performance zk‐SNARK in most real applications. As a result, NTT (Number Theoretic Transform), one of the most time‐consuming parts in proof‐generation, needs to be accelerated significantly. To address this issue, we propose a novel and efficient “data reordering" technique to enable a highly pipelined architecture, on which an FPGA‐based hardware accelerator is designed to support the large‐bitwidth and large‐scale NTT tasks in zk‐SNARK. This two‐level pipelined architecture can effectively reduce the data dependency and memory access requirements, meanwhile, can be flexibly scaled to different scales of FPGAs. To balance computing efficiency and flexibility, the OpenCL equipped with HLS is used to implement the heterogeneous acceleration system. We prototype the accelerator on the AMD‐Xilinx Alveo U50 card (UltraScale+ XCU50 FPGA). The evaluation results show that (1) our accelerator shows high scalability for different scales of FPGAs with a stable performance improvement; (2) it performs 1.95 faster than the one in PipeZK; (3) and it achieves 27.98 , 1.74 speedup and 6.9 , 6 energy efficiency improvement than AMD Ryzen 9 5900X single core and 12 cores respectively when integrated into the well‐known ZKP open‐source project, Bellman.

Minimizing Global Buffer Access in a Deep Learning Accelerator Using a Local Register File with a Rearranged Computational Sequence.

We propose a method for minimizing global buffer access within a deep learning accelerator for convolution operations by maximizing the data reuse through a local register file, thereby substituting the local register file access for the power-hungry global buffer access. To fully exploit the merits of data reuse, this study proposes a rearrangement of the computational sequence in a deep learning accelerator. Once input data are read from the global buffer, repeatedly reading the same data is performed only through the local register file, saving significant power consumption. Furthermore, different from prior works that equip local register files in each computation unit, the proposed method enables sharing a local register file along the column of the 2D computation array, saving resources and controlling overhead. The proposed accelerator is implemented on an off-the-shelf field-programmable gate array to verify the functionality and resource utilization. Then, the performance improvement of the proposed method is demonstrated relative to popular deep learning accelerators. Our evaluation indicates that the proposed deep learning accelerator reduces the number of global-buffer accesses to nearly 86.8%, consequently saving up to 72.3% of the power consumption for the input data memory access with a minor increase in resource usage compared to a conventional deep learning accelerator.

A Hierarchical Network with User Memory Matrix for Long Sequence Recommendation

In many recommendation scenarios, the interactions between users and items are divided into a series of sessions according to the time interval. The traditional Recurrent Neural Network has some shortcomings, such as limited memory ability, inflexible access to memory data, and obvious deficiency in feature capture for long sequences. To deal with the mentioned issues, we propose a hierarchical network with user memory matrix, named HNUM2, which utilizes the memory network to store users' long-term and short-term interests. The memory network is more flexible to access memory data, which can solve the problem of insufficient capture of long sequence features. The proposed model is a hierarchical recommendation algorithm, which consists of two layers. The first layer is the session-level GRU model, which obtains the sequence characteristics of the current session to predict the next item. The second layer is the user-level memory network model which exploits the attention mechanism and incorporates the write module and read module. The experimental results on two public available datasets show that HNUM2 has achieved significant performance improvement comparing to the state-of-the-art methods.

Single-chip multi-processing architecture for spaceborne SAR imaging and intelligent processing

The satellite-borne SAR image intelligent processing system needs to process on-orbit real-time imaging and various tasks of applications, for which reason designing a dedicated high-efficient single-chip multi-processor is of prioritized necessity that can simultaneously satisfy requirements of real-time and low power consumption. Aiming at on-chip data organization and memory access structure, two typical models of SAR(synthetic aperture radar) imaging CSA (chirp scaling) and neural network VGG-11 are analyzed, and then a collaborative computing model for the intelligent processing on remote sensing is extracted. A strip Tile data processing scheme and a dedicated multi-processing architecture is not only proposed, but a data organization and a caching strategy of Tile space synchronization splicing is also presented. In addition, the designed data caching structure among the processing units greatly reduces off-chip access memory bandwidth while supporting parallel pipeline execution of multi-task model. The chip adopts 28 nm CMOS technology featuring with merely 1.83 W of the overall power consumption, whose throughput and energy efficiency reaches 9.89TOPS and 5.4 TOPS/W, respectively. The present architecture can improve real-time performance of the on-orbit remote sensing intelligent processing platform while reducing the complexity of system designing, which is highly adaptive to differentiated expansions according to different models of algorithm.

Evaluation of De-Embedding Methods for Impedance Characterization of Magnetic Nanoparticles

Data storage and memory access has been, over the years, a technological issue due to the big volume of digital information. High-frequency magneto-transport in ferromagnetic nanoparticles may be used to deal with nanoscale storage devices, which requires an adequate electrical and magnetic characterization in the frequency domain. In this work, we fabricate ferromagnetic nanoparticles on top of platinum electrodes using focused-electron-beam induced +deposition (FEBID) method. Electrical characterization was performed using a Vector Network Analyzer (VNA). After these measurements, three de-embedding methods were applied: Open De-Embedding (OPD), Open-Short De-Embedding (OSD) and the Thru-Short-Open De-Embedding (TSOD) methods. We could observe that Thru-Short-Open De-Embedding provides good accuracy for evaluating the scattering parameters and to compensate the electrodes (fixture) signal due to the high frequencies behavior of ferromagnetic nanoparticles. The results will be useful to characterize ferromagnetic nanodisks used in data storage devices.

Efficient graphical-processor-unit parallelization algorithm for computing Eigen values

Several leading-edge applications such as pathology detection, biometric identification, and face recognition are based mainly on blob and line detection. To address this problem, Eigen value computing has been commonly employed due to its accuracy and robustness. However, Eigen value computing requires a raised computational processing, intensive memory data access, and data overlapping, which involve higher execution times. To overcome these limitations, we propose in this paper a new parallel strategy to implement Eigen value computing using a graphics processing unit (GPU). Our contributions are (1) to optimize instruction scheduling to reduce the computation time, (2) to efficiently partition processing into blocks to increase the occupancy of streaming multiprocessors, (3) to provide efficient input data splitting on shared memory to benefit from its lower access time, and (4) to propose new data management of shared memory to avoid access memory conflict and reduce memory bank accesses. Experimental results show that our proposed GPU parallel strategy for Eigen value computing achieves speedups of 27 compared with a multithreaded implementation, of 16 compared with a predefined function in the OpenCV library, and of eight compared with a predefined function in the Cublas library, all of which are performed into a quad core multi-central-processing unit platform. Next, our parallel strategy is evaluated through an Eigen value-based method for retinal thick vessel segmentation, which is essential for detecting ocular pathologies. Eigen value computing is executed in 0.017 s when using Structured Analysis of the Retina database images. Accordingly, we achieved real-time thick retinal vessel segmentation with an average execution time of about 0.039 s.

Optimization of System Framework for Secure Communication and Data Virtualization on Cloud Computing.

The consequent deployment of vital infrastructure to provide the secure communication and virtualization not only rectifies the challenges and difficulties but also benefits with saving the process of digitization. With a bang in evolution of cloud computing the uniqueness of every organization affects the virtualization of its information communication technology and applications. The answer for such organizations is to stipulate the precise cloud computing model equivalent to the deliberated and operational goals. This research article emphasized at budding and optimizing a framework for virtual infrastructure and cloud computing model to support the organization stakeholders. Moreover it also improves the cloud security with a protected virtual infrastructure and communication model with better performance. On account of data storage security necessities and distinctiveness of cloud computing environment, a method for secured data & storage based on dynamic allocation and access control mechanism is presented. Dynamic resource allocation is applied for resource utilization that focused on data virtualization and memory virtualization, for protection and access control a modified KPABE algorithm is integrated. The combination of these methods resulting on the optimize resource use, centralizing of storage. The implementation and comparison of results revealed that the proposed modified KP-ABE has performed effectively than the other security standards and especially for resource utilization. The proposed method serves for efficient data storage, access control solutions and computation in cloud environment.

HydraList

Increased capacity of main memory has led to the rise of in-memory databases. With disk access eliminated, efficiency of index structures has become critical for performance in these systems. An ideal index structure should exhibit high performance for a wide variety of workloads, be scalable, and efficient in handling large data sets. Unfortunately, our evaluation shows that most state-of-the-art index structures fail to meet these three goals. For an index to be performant with large data sets, it should ideally have time complexity independent of the key set size. To ensure scalability, critical sections should be minimized and synchronization mechanisms carefully designed to reduce cache coherence traffic. Moreover, complex memory hierarchy in servers makes data placement and memory access patterns important for high performance across all workload types. In this paper, we present HydraList, a new concurrent, scalable, and high performance in-memory index structure for massive multi-core machines. The key insight behind our design of HydraList is that an index structure can be divided into two components (search and data layers) which can be updated independently leading to lower synchronization overhead. By isolating the search layer, we are able to replicate it across NUMA nodes and reduce cache misses and remote memory accesses. As a result, our evaluation shows that HydraList outperforms other index structures especially in a variety of workloads and key types.

CuTensor-Tubal: Efficient Primitives for Tubal-Rank Tensor Learning Operations on GPUs

Tensors are the cornerstone data structures in high-performance computing, big data analysis and machine learning. However, tensor computations are compute-intensive and the running time increases rapidly with the tensor size. Therefore, designing high-performance primitives on parallel architectures such as GPUs is critical for the efficiency of ever growing data processing demands. Existing GPU basic linear algebra subroutines (BLAS) libraries (e.g., NVIDIA cuBLAS) do not provide tensor primitives. Researchers have to implement and optimize their own tensor algorithms in a case-by-case manner, which is inefficient and error-prone. In this paper, we develop the cuTensor-tubal library of seven key primitives for the tubal-rank tensor model on GPUs: t-FFT, inverse t-FFT, t-product, t-SVD, t-QR, t-inverse, and t-normalization. cuTensor-tubal adopts a frequency domain computation scheme to expose the separability in the frequency domain, then maps the tube-wise and slice-wise parallelisms onto the single instruction multiple thread (SIMT) GPU architecture. To achieve good performance, we optimize the data transfer, memory accesses, and design the batched and streamed parallelization schemes for tensor operations with data-independent and data-dependent computation patterns, respectively. In the evaluations of t-product, t-SVD, t-QR, t-inverse and t-normalization, cuTensor-tubal achieves maximum $16.91 \times, 27.03 \times, 38.97 \times, 22.36 \times, 15.43 \times$ 16 . 91 × , 27 . 03 × , 38 . 97 × , 22 . 36 × , 15 . 43 × speedups respectively over the CPU implementations running on dual 10-core Xeon CPUs. Two applications, namely, t-SVD-based video compression and low-tubal-rank tensor completion, are tested using our library and achieve maximum $9.80 \times$ 9 . 80 × and $269.26 \times$ 269 . 26 × speedups over multi-core CPU implementations.

CASCADE

A Coarse-Grained Reconfigurable Array (CGRA) is a promising high-performance low-power accelerator for compute-intensive loop kernels. While the mapping of the computations on the CGRA is a well-studied problem, bringing the data into the array at a high throughput remains a challenge. A conventional CGRA design involves on-array computations to generate memory addresses for data access undermining the attainable throughput. A decoupled access-execute architecture, on the other hand, isolates the memory access from the actual computations resulting in a significantly higher throughput. We propose a novel decoupled access-execute CGRA design called CASCADE with full architecture and compiler support for high-throughput data streaming from an on-chip multi-bank memory. CASCADE offloads the address computations for the multi-bank data memory access to a custom designed programmable hardware. An end-to-end fully-automated compiler synchronizes the conflict-free movement of data between the memory banks and the CGRA. Experimental evaluations show on average 3× performance benefit and 2.2× performance per watt improvement for CASCADE compared to an iso-area conventional CGRA with a bigger processing array in lieu of a dedicated hardware memory address generation logic.

GAMERA: A Three-dimensional Finite-volume MHD Solver for Non-orthogonal Curvilinear Geometries

Abstract Efficient simulation of plasmas in various contexts often involves the use of meshes that conform to the intrinsic geometry of the system under consideration. We present here a description of a new magnetohydrodynamic code, GAMERA (Grid Agnostic MHD for Extended Research Applications), designed to combine geometric flexibility with high-order spatial reconstruction and constrained transport to maintain the divergence-free magnetic field. GAMERA carries on the legacy of its predecessor, the LFM (Lyon–Fedder–Mobarry), a research code whose use in space physics has spanned three decades. At the time of its initial development, the LFM code had a number of novel features: eighth-order centered spatial differencing, the Partial Donor Cell Method limiter for shock capturing, a non-orthogonal staggered mesh with constrained transport, and conservative averaging-reconstruction for axis singularities. The capability to handle multiple ion species was also added later. GAMERA preserves the core numerical philosophy of LFM while also incorporating numerous algorithmic and computational improvements. The upgrades in the numerical schemes include accurate grid metric calculations using high-order Gaussian quadrature techniques, high-order upwind reconstruction, non-clipping options for interface values, and improved treatment of axis singularities. The improvements in the code implementation include the use of data structures and memory access patterns conducive to aligned vector operations and the implementation of hybrid parallelism, using MPI and OMP. GAMERA is designed to be a portable and easy-to-use code that implements multidimensional MHD simulations in arbitrary non-orthogonal curvilinear geometries on modern supercomputer architectures.

Critical Memory Data Access Monitor Based on Dynamic Strategy Learning

Critical Memory Data Access Monitor Based on Dynamic Strategy Learning

A methodology correlating code optimizations with data memory accesses, execution time and energy consumption

The advent of data proliferation and electronic devices gets low execution time and energy consumption software in the spotlight. The key to optimizing software is the correct choice, order as well as parameters of optimization transformations that has remained an open problem in compilation research for decades for various reasons. First, most of the transformations are interdependent and thus addressing them separately is not effective. Second, it is very hard to couple the transformation parameters to the processor architecture (e.g., cache size) and algorithm characteristics (e.g., data reuse); therefore, compiler designers and researchers either do not take them into account at all or do it partly. Third, the exploration space, i.e., the set of all optimization configurations that have to be explored, is huge and thus searching is impractical. In this paper, the above problems are addressed for data-dominant affine loop kernels, delivering significant contributions. A novel methodology is presented reducing the exploration space of six code optimizations by many orders of magnitude. The objective can be execution time (ET), energy consumption (E) or the number of L1, L2 and main memory accesses. The exploration space is reduced in two phases: firstly, by applying a novel register blocking algorithm and a novel loop tiling algorithm and secondly, by computing the maximum and minimum ET/E values for each optimization set. The proposed methodology has been evaluated for both embedded and general-purpose CPUs and for seven well-known algorithms, achieving high memory access, speedup and energy consumption gain values (from 1.17 up to 40) over gcc compiler, hand-written optimized code and Polly. The exploration space from which the near-optimum parameters are selected is reduced from 17 up to 30 orders of magnitude.

Performance Modeling for CNN Inference Accelerators on FPGA

The recently reported successes of convolutional neural networks (CNNs) in many areas have generated wide interest in the development of field-programmable gate array (FPGA)-based accelerators. To achieve high performance and energy efficiency, an FPGA-based accelerator must fully utilize the limited computation resources and minimize the data communication and memory access, both of which are impacted and constrained by a variety of design parameters, e.g., the degree and dimension of parallelism, the size of on-chip buffers, the bandwidth of the external memory, and many more. The large design space of the accelerator makes it impractical to search for the optimal design in the implementation phase. To address this problem, a performance model is described to estimate the performance and resource utilization of an FPGA implementation. By this means, the performance bottleneck and design bound can be identified and the optimal design option can be explored early in the design phase. The proposed performance model is validated using a variety of CNN algorithms comparing the results with on-board test results on two different FPGAs.

GPU-based adaptive compacting neighborhood tabu search for hardware/software partitioning

Hardware/software (HW/SW) partitioning is an essential step in hardware/software co-design as it determines the functions to be implemented in the hardware and software. HW/SW partitioning problems are NP hard. The increasing complexities of modern embedded systems worsen the problem, thus motivating the search for solutions by heuristics. The tabu search algorithm is an effective method for HW/SW partitioning. However, the process is time consuming. The existing tabu search methods for HW/SW partitioning focus on sequential implementations that involve a trade-off between solution time and solution quality. This trade-off sacrifices the solution quality. This paper presents a GPU-based adaptive compacting neighborhood tabu search for HW/SW partitioning. First, we present an adaptive strategy that enhances the search intensification to improve the solution quality. The massive parallelism of GPU architecture can reduce the solution time of the proposed strategy. Next, to ensure that the algorithm execute efficiently on the GPU, we further propose several implementation strategies such as the representation of task graph on the GPU, the mapping between the GPU thread and candidate, and data layout and memory access optimization. Finally, we realize the proposed method in a computing unified device architecture, namely CUDA, and verify the effectiveness according to the related benchmark with different communication-to-computation ratios and real-time constraint requirements. The results show that the proposed method can yield a better solution quality compared to the existing methods. By comparing with the naive GPU implementation of adaptive compacting neighborhood tabu search, the proposed implementation strategies on the GPU significantly reduced the solution time. In addition, we verified the advantage of the GPU-based method for very large HW/SW partitioning.

The VLSI Architecture of a Highly Efficient Deblocking Filter for HEVC Systems

This paper presents the VLSI architecture and hardware implementation of a highly efficient deblocking filter (DBF) for High Efficiency Video Coding systems. In order to reduce the number of data accesses and thus to enhance the timing efficiency, novel data structures and memory access schemes for image pixels are proposed. Furthermore, a novel edge-fetching order is presented to strike a balance between the processing throughput and complexity. Based on the proposed structure and access pattern, a six-stage pipelined two-line DBF engine with low-latency data access sequence is designed, aiming to achieve high processing throughput while at the same time maintaining low complexity. The detailed storage structure and data access scheme are illustrated and VLSI architecture for the DBF engine is depicted in this paper. In addition, the proposed DBF is implemented using TSMC 90-nm standard cell library. The experimental results based on postlayout estimations show that the proposed design can achieve 60 frames/s for a frame resolution of $4096\times 2048$ pixels (ultra high definition resolution) assuming an operating frequency of 100 MHz. Moreover, this design occupies an area complexity of 466.5 kGE with a power consumption of 26.26 mW. In comparison with prior designs targeting similar system specification and throughput, the proposed design results in a significantly reduced area complexity.

An efficient wavefront parallel algorithm for structured three dimensional LU-SGS

Parallel computing is a useful technology for scientific and engineering algorithms/applications. LU-SGS (lower-upper Symmetric-Gauss–Seidel method) is an efficient and robust scheme for CFD (Computational fluid dynamics) and has strong data dependence in its computation. In this paper, we present an efficient wavefront parallel algorithm for 3D (three dimensional) LU-SGS with structured meshes. The corresponding data structure and memory access method with better data locality and communication optimization is designed. The performances of the presented parallel algorithm are reported with different problem sizes. Some discussion and performance issues are also reported. The results show that the overall performance speedup of one Intel E5540 CPU (4 CPU cores) ranges from 2.23 to 2.95 compared with one E5540 core. The parallel efficiency of 1024, 128 processes are up to 35.68%, 72.69% compared with 32 processes on a distributed memory cluster system. The CFD simulation of M6 wing model shows the effect of the presented parallel algorithm.

Single-Scale Fusion: An Effective Approach to Merging Images.

Due to its robustness and effectiveness, multi-scale fusion (MSF) based on the Laplacian pyramid decomposition has emerged as a popular technique that has shown utility in many applications. Guided by several intuitive measures (weight maps) the MSF process is versatile and straightforward to be implemented. However, the number of pyramid levels increases with the image size, which implies sophisticated data management and memory accesses, as well as additional computations. Here, we introduce a simplified formulation that reduces MSF to only a single level process. Starting from the MSF decomposition, we explain both mathematically and intuitively (visually) a way to simplify the classical MSF approach with minimal loss of information. The resulting single-scale fusion (SSF) solution is a close approximation of the MSF process that eliminates important redundant computations. It also provides insights regarding why MSF is so effective. While our simplified expression is derived in the context of high dynamic range imaging, we show its generality on several well-known fusion-based applications, such as image compositing, extended depth of field, medical imaging, and blending thermal (infrared) images with visible light. Besides visual validation, quantitative evaluations demonstrate that our SSF strategy is able to yield results that are highly competitive with traditional MSF approaches.