Memory Reuse Research Articles

Area-Delay-Power Efficient VLSI Architecture 2D FIR Filter using Modified Multipliers and Adders

In this paper, a low power, area, and delay 2D Finite Impulse Response (FIR) filter architecture is derived from an analysis of a memory-efficient design. The completely direct-form 2D FIR filter is where the idea of parallel processing is first presented. As a result, the FIR filter may make better use of its memory by reusing its contents. With a block size of L and a filter length of N, a non-separable 2D FIR filter structure is developed and implemented. The FIR filter's arithmetic module makes use of high-speed, low-power multipliers and Carry Look Ahead (CLA) adders, with the output calculated by a pipelined adder unit. Verilog HDL code is used to represent the proposed architecture, and the CADENCE environment's NC Simulator and RTL Compiler synthesis tool are used to verify the design. Existing memory-efficient 2D FIR filter hardware architectures are compared to the produced area, power, and delay reports. Using Modified CLA (MCLA) adders and pipelining, we were able to cut down on power consumption by 44% and delay by 20%. Index Terms: Memory reuse, 2D-FIR filter, Low Power Multiplier, Parallel Prefix Adder, Carry Look Ahead adder, and pipelining.

Method for scalable and performant GPU-accelerated simulation of multiphase compressible flow

Multiphase compressible flows are often characterized by a broad range of space and time scales, entailing large grids and small time steps. Simulations of these flows on CPU-based clusters can thus take several wall-clock days. Offloading the compute kernels to GPUs appears attractive but is memory-bound for many finite-volume and -difference methods, damping speedups. Even when realized, GPU-based kernels lead to more intrusive communication and I/O times owing to lower computation costs. We present a strategy for GPU acceleration of multiphase compressible flow solvers that addresses these challenges and obtains large speedups at scale. We use OpenACC for directive-based offloading of all compute kernels while maintaining low-level control when needed. An established Fortran preprocessor and metaprogramming tool, Fypp, enables otherwise hidden compile-time optimizations. This strategy exposes compile-time optimizations and high memory reuse while retaining readable, maintainable, and compact code. Remote direct memory access realized via CUDA-aware MPI and GPUDirect reduces halo-exchange communication time. We implement this approach in the open-source solver MFC [1]. Metaprogramming results in an 8-times speedup of the most expensive kernels compared to a statically compiled program, reaching 46% of peak FLOPs on modern NVIDIA GPUs and high arithmetic intensity (about 10 FLOPs/byte). In representative simulations, a single NVIDIA A100 GPU is 7-times faster compared to an Intel Xeon Cascade Lake (6248) CPU die, or about 300-times faster compared to a single such CPU core. At the same time, near-ideal (97%) weak scaling is observed for at least 13824 GPUs on OLCF Summit. A strong scaling efficiency of 84% is retained for an 8-times increase in GPU count. Collective I/O, implemented via MPI3, helps ensure the negligible contribution of data transfers (<1% of the wall time for a typical, large simulation). Large many-GPU simulations of compressible (solid-)liquid-gas flows demonstrate the practical utility of this strategy.

Efficient memory reuse methodology for CNN-based real-time image processing in mobile-embedded systems

Efficient memory reuse methodology for CNN-based real-time image processing in mobile-embedded systems

Implementation of distributed arithmetic-based symmetrical 2-D block finite impulse response filter architectures.

Background: This paper presents an efficient two-dimensional (2-D) finite impulse response (FIR) filter using block processing for two different symmetries. Architectures for a general filter (without symmetry) and two symmetrical filters (diagonal and quadrantal symmetry) are implemented. The proposed architectures need fewer multipliers because of the symmetry of the filter coefficients. Methods: A distributed arithmetic (DA)- based multiplication method is used in the proposed architecture. A dual-port memory-based lookup table (DP-MLUT) is used in the multiplication instead of lookup-table (LUT) to reduce the area and power of the FIR filter. The filter's throughput is increased by using block processing. Memory reuse and memory sharing methods are introduced, which reduces the need for many registers and hence the circuit complexity. The architectures are written in Verilog Hardware Description Language and synthesized using Genus Synthesis tool-19.1 in 45nm technology with a generic library of Cadence vendor constraints. The synthesis tool generates the area, delay, and power reports. Power consumption of architectures is calculated with an image size of 64 X 64 and at 20 MHz frequency. Results: Compared to existing architectures, the synthesis results show improvements in power, area, area delay product (ADP), and power delay product (PDP). The proposed MLUT-based 2-D block Quadrantal Symmetry Filter (QSF) for length 8 with block size 4 consumes 58.94% less power, occupies 59.5% less area, 48.44% less ADP and 47.78% less PDP compared to best existing methods. Conclusions: A novel DA-based 2-D block FIR filter architecture with various symmetries is realized. Symmetry is incorporated into the filter coefficients to minimize the number of multipliers. The LUT size is optimized by odd multiples or even multiples storage techniques. Also, the overall area of the architecture is decreased by DP-LUT-based multipliers. The proposed filter architecture is area-power-efficient. It is best suited for applications that have fixed coefficients.

A Resource Allocation Algorithm for a History-Aware Frame Graph

We consider the problem of memory consumption by a real-time GPU-accelerated graphical application. A history of a resource is defined for a particular frame to be the final contents of such a resource at the end of the previous frame. When organizing a graphical application using a frame rendering graph approach, it makes sense to implement automatic serving of resource history read requests of nodes. In absence of history resource requests, allocating resources for a fixed frame graph is the classic problem of dynamic storage allocation (DSA). In this paper, we formulate a generalization of DSA that enables memory reuse for resources with history requests and provide a practical approximate algorithm for solving it.

Ethical approach to translation memory reuse: discussions from copyright and business ethics perspectives

ABSTRACT Under existing copyright provisions, translation memories are protected as “composite works” comprised of source texts and their translations, or as “databases” when created with a large volume of the source text and target text pairs. Freelance translators should exercise copyright to translations in line with the “sweat of the brow” principle. However, the translation industry maintains practices that dispossess translators of translation memories and reuse them without translators’ approval. When examined based on Kant's categorical imperative, this utilitarian practice is deemed unethical since it violates individual translators’ rights. This study offers proposals for promoting the ethical reuse of translation memories. First, translations by freelancers should be excluded from works-for-hire to allow the global protection of language assets and the rights of freelancers. Second, a collective licensing regime for translation memories should be introduced to promote distributive justice and the free circulation of knowledge.

Neural Acceleration of Graph Based Utility Functions for Sparse Matrices

Many graph-based algorithms in high performance computing (HPC) use approximate solutions due to having algorithms that are computationally expensive or serial in nature. Neural acceleration, i.e., the process of speeding up approximation computation elements via artificial neural networks, is relatively new and has not focused on HPC graph-based algorithms. In this paper, we propose a starting point for applying models for neural acceleration to graph-based HPC algorithms utilizing an understanding of the connectivity computational pattern, recursive neural networks, and graph neural networks. We demonstrate these techniques on the problem related to the utility functions of sparse matrix ordering and fill-in (i.e., zero elements becoming nonzero during factorization) calculations. The problem of sparse matrix ordering is commonly used for issues related to load balancing, improving memory reuse, or reducing computational and memory costs in direct sparse linear solver methods. These utility functions are ideal for demonstration as they comprise a number of different graph-based subproblems, and thus demonstrate the usefulness of our method over a wide range. We show that we can accurately approximate the best ordering and the nonzero count of the sparse factorization matrix while speeding up the calculation by as much as 30.3× over the traditional serial method.

Memory-Throughput Trade-off for CNN-Based Applications at the Edge

Many modern applications require execution of Convolutional Neural Networks (CNNs) on edge devices, such as mobile phones or embedded platforms. This can be challenging, as the state-of-the art CNNs are memory costly, whereas the memory budget of edge devices is highly limited. To address this challenge, a variety of CNN memory reduction methodologies have been proposed. Typically, the memory of a CNN is reduced using methodologies such as pruning and quantization. These methodologies reduce the number or precision of CNN parameters, thereby reducing the CNN memory cost. When more aggressive CNN memory reduction is required, the pruning and quantization methodologies can be combined with CNN memory reuse methodologies. The latter methodologies reuse device memory allocated for storage of CNN intermediate computational results, thereby further reducing the CNN memory cost. However, the existing memory reuse methodologies are unfit for CNN-based applications that exploit pipeline parallelism available within the CNNs or use multiple CNNs to perform their functionality. In this article, we therefore propose a novel CNN memory reuse methodology. In our methodology, we significantly extend and combine two existing CNN memory reuse methodologies to offer efficient memory reuse for a wide range of CNN-based applications.

Very large scale integration implementation of efficient finite impulse response filter architectures using novel distributed arithmetic for digital channelizer of software defined radio

AbstractAlways an efficient and low complexity reconfigurable filter architecture is required for the channel filters of digital channelizer in software‐defined radio (SDR). In this paper, a block‐based reconfigurable finite impulse response (FIR) filter is proposed using distributed arithmetic (DA) technique. The complexity of the conventional multiplier is replaced with the DA multiplication process, and the throughput of the entire filter is increased by block processing. Memory reuse is also achieved in the proposed direct form systolic FIR filter architecture due to parallel processing. The different bandwidth filters and the corresponding coefficients are stored in the look‐up tables (LUTs) concerning different channels of the digital channelizer. The proper channel selection is used to choose an appropriate filter and partial products are generated using offset binary coding (OBC). Next, the multiplication process is done by the proposed decomposed LUT‐based DA technique in the processing blocks of the filter. The proposed filter is coded by Verilog and synthesized by application specific integrated circuit (ASIC)‐based tools from Cadence in 45 nm CMOS technology. The performance parameters such as area, delay, power consumption, area‐delay product (ADP), and power delay product (PDP) are evaluated and compared with state‐of‐the‐art works. The ADP and PDP values are saved by 44.1% and 30% by the proposed OBC‐DA‐based filter architecture than the conventional DA‐based filter architecture, respectively.

A Low-Power Analog Processor-in-Memory-Based Convolutional Neural Network for Biosensor Applications.

This paper presents an on-chip implementation of an analog processor-in-memory (PIM)-based convolutional neural network (CNN) in a biosensor. The operator was designed with low power to implement CNN as an on-chip device on the biosensor, which consists of plates of 32 × 32 material. In this paper, 10T SRAM-based analog PIM, which performs multiple and average (MAV) operations with multiplication and accumulation (MAC), is used as a filter to implement CNN at low power. PIM proceeds with MAV operations, with feature extraction as a filter, using an analog method. To prepare the input feature, an input matrix is formed by scanning a 32 × 32 biosensor based on a digital controller operating at 32 MHz frequency. Memory reuse techniques were applied to the analog SRAM filter, which is the core of low power implementation, and in order to accurately grasp the MAC operational efficiency and classification, we modeled and trained numerous input features based on biosignal data, confirming the classification. When the learned weight data was input, 19 mW of power was consumed during analog-based MAC operation. The implementation showed an energy efficiency of 5.38 TOPS/W and was differentiated through the implementation of 8 bits of high resolution in the 180 nm CMOS process.

Implementation of Low Power Generic 2D FIR Filter Bank Architecture Using Memory-based Multipliers

In this paper, a generic filter bank architecture for 2D FIR filter is proposed using block processing, symmetry in the filter coefficients, and memory-based multipliers. The different symmetry filters are considered as sub-filters of the filter bank to decrease the number of multipliers and the desired filter can be selected using control logic to reduce the power consumption. The block processing is incorporated to increase the throughput of the filter. Due to this block processing, memory sharing and memory reuse are achieved to optimize the architecture in terms of area, memory, and power. In each filter, the conventional multipliers are replaced with Distribute Arithmetic (DA) based memory multipliers to decrease the delay, power, and area of each sub-filter. The proposed design is coded by Verilog HDL and synthesized using Cadence Genus tools in 45 nm technology. The physical design is carried out using Cadence Innovus tools. The proposed design results are compared with state-of-the-art works.

A 58.6/91.3 pJ/b Dual-Mode Belief-Propagation Decoder for LDPC and Polar Codes in the 5G Communications Standard

This letter presents the first belief propagation (BP) decoder IC implementation for the two forward error correction (FEC) codes in the 5G communication standard. The LDPC mode supports 5G BG2 with 128 lifting size, while the polar mode supports code length <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$N = 1024$ </tex-math></inline-formula> . The 40-nm CMOS chip features BP module sharing, memory reuse, check node unit design, forwarding and layer pipelining, and dataflow rearrangement. Compared to two single-mode decoders, this dual-mode decoder saved 37% in the overall die area, 32% in the computation circuit area, and 41% in the memory area. The chip delivers throughputs of 2.38 and 1.85 Gb/s from 0.9-V Vdd with energy efficiencies of 58.6 and 91.3 pJ/b in the LDPC mode and the polar mode, respectively.

FPGA-Based Implementation of Ship Detection for Satellite On-Board Processing

At present, the raw echo data captured by spaceborne SAR is usually downlinked to the ground stations for imaging and analysis. If the SAR data can be processed on-board, the processing results could be transmitted to users directly through broadcast distribution, which can greatly reduce the delay time for emergency applications. However, on-board processing implementation faces many challenges due to the space radiation environment and limited resources of the satellite. In this paper, an FPGA-based implementation of ship detection for on-board processing is proposed, which has high efficiency in logic and memory resources. First, the hardware-oriented clustering approach is proposed. Image processing operations with similar regularity of data access can be mapped in one processing engine, which can reduce the employment of logic resources. Second, the concept of the data-buffering cycle (DBC) is proposed. The DBC provides a method to analyze the intermediate data and optimize the memory reuse. Finally, based on the above optimization methods, the FPGA-based implementation for a ship detection algorithm is presented. Compared with the traditional methods, the experimental results show the efficacy of our proposed method with lower consumption in logic and memory resources.

Mining high utility pattern with negative items in dynamic databases

High utility pattern mining with negative items (HUPMN) has more practical applications because it can process data with negative utility values. But the existing HUPMN algorithms assume that the database is static and it would be very expensive to use these algorithms directly to deal with dynamic databases. To cope with this challenge, an high utility pattern algorithm for mining negative items from an incremental database is proposed for the first time, and an incremental index list structure is designed, which uses index values to quickly access and update the information stored in the list. In addition, A memory reuse strategy is also applied to reduce memory usage. Finally, a HUPMN algorithm based on sliding window was proposed, which can quickly update item information when the window was sliding. Extensive experimental evaluations have been carried out on a variety of data sets, and the results show that the proposed algorithm exhibits excellent performance in terms of running time and memory usage.

Machine Learning–enabled Scalable Performance Prediction of Scientific Codes

Hardware architectures become increasingly complex as the compute capabilities grow to exascale. We present the Analytical Memory Model with Pipelines (AMMP) of the Performance Prediction Toolkit (PPT). PPT-AMMP takes high-level source code and hardware architecture parameters as input and predicts runtime of that code on the target hardware platform, which is defined in the input parameters. PPT-AMMP transforms the code to an (architecture-independent) intermediate representation, then (i) analyzes the basic block structure of the code, (ii) processes architecture-independent virtual memory access patterns that it uses to build memory reuse distance distribution models for each basic block, and (iii) runs detailed basic-block level simulations to determine hardware pipeline usage. PPT-AMMP uses machine learning and regression techniques to build the prediction models based on small instances of the input code, then integrates into a higher-order discrete-event simulation model of PPT running on Simian PDES engine. We validate PPT-AMMP on four standard computational physics benchmarks and present a use case of hardware parameter sensitivity analysis to identify bottleneck hardware resources on different code inputs. We further extend PPT-AMMP to predict the performance of a scientific application code, namely, the radiation transport mini-app SNAP. To this end, we analyze multi-variate regression models that accurately predict the reuse profiles and the basic block counts. We validate predicted SNAP runtimes against actual measured times.

Compiler-assisted Operator Template Library for DNN Accelerators

Despite many dedicated accelerators are gaining popularity for their performance and energy efficiency in the deep neural network (DNN) domain, high-level programming support for these accelerators remains thin. In contrast to existing researches targeting the whole DNNs, we choose to dive into details and review this problem from a finer-grained level, operators. Due to performance concerns, operator programmers may have to take hand-written assembly as their first choice, which is error-prone and involves many programming chores. To alleviate this problem, we propose TOpLib, a compiler-assisted template library. By providing a unified user-view abstraction, TOpLib allows programmers to express computational kernels with high-level tensor primitives, which will be automatically lowered into low-level intrinsic primitives via expression templates. Moreover, considering memory management is performance-critical and the optimization strategy of expression template is limited to enumeration based rewriting rules, we implement TOpLib with a compiler-assisted approach. We address the memory reuse challenges into the compiler, which allows TOpLib to make full use of on-chip buffers and result in better performance. Experiments over 55 typical DNN operators demonstrate that TOpLib can generate scalable code with performance faster than or on par with hand-written assembly versions.

Bridging the Gap Between Memory and Communication Efficiency on Distributed Deep Learning Systems

Large-scale distributed deep learning is of great importance in various applications. For data-parallel distributed training systems, limited hardware resources (e.g., GPU memory and interconnection bandwidth) often become a performance bottleneck, and it is necessary to consider the full utilization of multiple resources simultaneously, especially for extreme-scale deep neural networks. Although two different types of strategies, based on memory management and sparse communication, have been proposed to reduce the usage of resources, a naive combination of these two optimizations is impractical, since they cannot successfully coexist with each other. We therefore consider the idea of collaborative optimization in terms of both system memory and bandwidth resources, and propose a layer-centric memory-efficient distributed sparse communication mechanism called LaySA. Firstly, to tackle the memory ballooning issue caused by sparse communication, the existing memory reuse strategy is refined, and the data object of the memory optimization is augmented and redefined. Secondly, a mirror weight update mechanism is proposed to address the contradiction between memory management and sparse communication optimization for weight gradients. Our scheme, which involves the deep integration and collaborative execution of these two types of strategies, can fill the gap in relation to multiple resource optimization in distributed GPU-based training systems. Our experimental results show that the proposed collaborative optimization can significantly alleviate the memory pressure on the computing nodes, and improve both the resource utilization and the throughput of distributed training systems. Compared with baseline systems using only a single strategy, LaySA can help to reduce the system memory usage by up to 80.5%, and the overall training time of the neural network models on a single GPU is reduced by about 12.25%. Furthermore, LaySA can scale up the batch size of the datasets by an extremely large factor during distributed training, and the overall throughput is increased by more than 150%, meaning that our approach outperforms current systems that use memory or communication optimization mechanisms alone.

The Layer-Wise Training Convolutional Neural Networks Using Local Loss for Sensor-Based Human Activity Recognition

Recently, deep learning, which are able to extract automatically features from data, has achieved state-of-the-art performance across a variety of sensor based human activity recognition (HAR) tasks. However, the existing deep neural networks are usually trained with a global loss, and all hidden layer weights have to be always kept in memory before the forward and backward pass has completed. The backward locking phenomenon prevents the reuse of memory, which is a crucial limitation for wearable activity recognition. In the paper, we proposed a layer-wise convolutional neural networks (CNN) with local loss for the use of HAR task. To our knowledge, this paper is the first that uses local loss based CNN for HAR in ubiquitous and wearable computing arena. We performed experiments on five public HAR datasets including UCI HAR dataset, OPPOTUNITY dataset, UniMib-SHAR dataset, PAMAP dataset, and WISDM dataset. The results show that local loss works better than global loss for tested baseline architectures. At no extra cost, the local loss can approach the state-of-the-arts on a variety of HAR datasets, even though the number of parameters was smaller. We believe that the layer-wise CNN with local loss can be used to update the existing deep HAR methods.

Layup

Although GPUs have emerged as the mainstream for the acceleration of convolutional neural network (CNN) training processes, they usually have limited physical memory, meaning that it is hard to train large-scale CNN models. Many methods for memory optimization have been proposed to decrease the memory consumption of CNNs and to mitigate the increasing scale of these networks; however, this optimization comes at the cost of an obvious drop in time performance. We propose a new memory optimization strategy named Layup that realizes both better memory efficiency and better time performance. First, a fast layer-type-specific method for memory optimization is presented, based on the new finding that a single memory optimization often shows dramatic differences in time performance for different types of layers. Second, a new memory reuse method is presented in which greater attention is paid to multi-type intermediate data such as convolutional workspaces and cuDNN handle data. Experiments show that Layup can significantly increase the scale of extra-deep network models on a single GPU with lower performance loss. It even can train ResNet with 2,504 layers using 12GB memory, outperforming the state-of-the-art work of SuperNeurons with 1,920 layers (batch size = 16).

A reliable PUF in a dual function SRAM

The Internet of Things (IoTs) employs resource-constrained sensor nodes for sensing and processing data that require robust, lightweight cryptographic primitives. The SRAM Physical Unclonable Function (SRAM-PUF) is a potential candidate for secure key generation. An SRAM-PUF is able to generate random and unique cryptographic keys based on start-up values by exploiting intrinsic manufacturing process variations. The reuse of the available on-chip SRAM memory in a system as a PUF might achieve useful cost efficiency. However, as CMOS technology scales down, aging-induced Negative Bias Temperature Instability (NBTI) becomes more pronounced resulting in asymmetric degradation of memory bit cells after prolonged storage of the same bit values. This causes unreliable start-up values for an SRAM-PUF. In this paper, the on-chip memory in the ARM architecture has been used as a case study to investigate reliability in an SRAM-PUF. We show that the bit probability in a 32-bit ARM instruction cache has a predictable pattern and hence predictable aging. Therefore, we propose using an instruction cache as a PUF to save silicon area. Furthermore, we propose a bit selection technique to mitigate the NBTI effect. We show that this technique can reduce the predicted bit error in an SRAM-PUF from 14.18% to 5.58% over 5 years. Consequently, as the bit error reduces, the area overhead of the error-correction circuitry is about 6 × smaller compared to that without a bit selection technique.