Off-chip Memory Access Research Articles

CRANarch: A feasible processor micro-architecture for Cloud Radio Access Network

Cloud Radio Access Network (C-RAN) becomes a promising infrastructure, which can improve hardware resource utilization of traditional Radio Access Network (RAN). For C-RAN, data centers are essential hardware platform, and these data centers are universally equipped with general commercial multi-core processors which are not designed for wireless communication protocols dedicatedly. In this paper, firstly, we evaluate performance bottlenecks of general multi-core processors with current dominant micro-architecture via typical wireless communication protocol. The results show that the dominant micro-architecture mismatches to demands of wireless communication protocols, which makes dominant general multi-core processor is inefficient when used in this application field. Secondly, we analyze the essential micro-architectural level reasons for the inefficiency in detail. A typical characteristic of wireless communication protocols is there are lots of random memory accesses in a large memory address space, which results in high miss ratio of on-chip cache hierarchies. The frequent on-chip cache misses result in amounts of off-chip memory access operations, which incurs longer memory access latency as a dominant factor to degrade overall performance. Based on the results, we identify key micro-architectural characteristics that meet demands of wireless communication protocols. Finally, we propose a feasible micro-architecture of multi-core processor for C-RAN, called as CRANarch, the performance results show its improved hardware utilization efficiency and power efficiency when used in C-RAN data centers.

SC2

Low utilization of on-chip cache capacity limits performance and wastes energy because of the long latency, limited bandwidth, and energy consumption associated with off-chip memory accesses. Value replication is an important source of low capacity utilization. While prior cache compression techniques manage to code frequent values densely, they trade off a high compression ratio for low decompression latency, thus missing opportunities to utilize capacity more effectively. This paper presents, for the first time, a detailed designspace exploration of caches that utilize statistical compression. We show that more aggressive approaches like Huffman coding, which have been neglected in the past due to the high processing overhead for (de)compression, are suitable techniques for caches and memory. Based on our key observation that value locality varies little over time and across applications, we first demonstrate that the overhead of statistics acquisition for code generation is low because new encodings are needed rarely, making it possible to off-load it to software routines. We then show that the high compression ratio obtained by Huffman-coding makes it possible to utilize the performance benefits of 4X larger last-level caches with about 50% lower power consumption than such larger caches

Victim retention for reducing cache misses in tiled chip multiprocessors

This paper presents CMP-VR (Chip-Multiprocessor with Victim Retention), an approach to improve cache performance by reducing the number of off-chip memory accesses. The objective of this approach is to retain the chosen victim cache blocks on the chip for the longest possible time. It may be possible that some sets of the CMPs last level cache (LLC) are heavily used, while certain others are not. In CMP-VR, some number of ways from every set are used as reserved storage. It allows a victim block from a heavily used set to be stored into the reserve space of another set. In this way the load of heavily used sets are distributed among the underused sets. This logically increases the associativity of the heavily used sets without increasing the actual associativity and size of the cache. Experimental evaluation using full-system simulation shows that CMP-VR has less off-chip miss-rate as compared to baseline Tiled CMP. Results are presented for different cache sizes and associativity for CMP-VR and baseline configuration. The best improvements obtained are 45.5% and 14% in terms of miss rate and cycles per instruction (CPI) respectively for a 4MB, 4-way set associative LLC. Reduction in CPI and miss rate together guarantees performance improvement.

Optimizing two-dimensional DMA transfers for scratchpad Based MPSoCs platforms

Reducing the effects of off-chip memory access latency is a key factor in exploiting efficiently embedded multi-core platforms. We consider architectures that admit a multi-core computation fabric, having its own fast and small memory to which the data blocks to be processed are fetched from external memory using a DMA (direct memory access) engine, employing a double- or multiple-buffering scheme to avoid processor idling. In this paper we focus on application programs that process two-dimensional data arrays and we determine automatically the size and shape of the portions of the data array which are subject to a single DMA call, based on hardware and applications parameters. When the computation on different array elements are completely independent, the asymmetry of memory structure leads always to prefer one-dimensional horizontal pieces of memory, while when the computation of a data element shares some data with its neighbors, there is a pressure for more “square” shapes to reduce the amount of redundant data transfers. We provide an analytic model for this optimization problem and validate our results by running a mean filter application on the Cell simulator.

MultiMaKe

The increasing demand for low-power and high-performance multimedia embedded systems has motivated the need for effective solutions to satisfy application bandwidth and latency requirements under a tight power budget. As technology scales, it is imperative that applications are optimized to take full advantage of the underlying resources and meet both power and performance requirements. We propose MultiMaKe, an application mapping design flow capable of discovering and enabling parallelism opportunities via code transformations, efficiently distributing the computational load across resources, and minimizing unnecessary data transfers. Our approach decomposes the application's tasks into smaller units of computations called kernels, which are distributed and pipelined across the different processing resources. We exploit the ideas of inter-kernel data reuse to minimize unnecessary data transfers between kernels, early execution edges to drive performance, and kernel pipelining to increase system throughput. Our experimental results on JPEG and JPEG2000 show up to 97% off-chip memory access reduction, and up to 80% execution time reduction over standard mapping and task-level pipelining approaches.

Fast and deterministic hash table lookup using discriminative bloom filters

Hash tables are widely used in network applications, as they can achieve O(1) query, insert, and delete operations at moderate loads. However, at high loads, collisions are prevalent in the table, which increases the access time and induces non-deterministic performance. Slow rates and non-determinism can considerably hurt the performance and scalability of hash tables in the multi-threaded parallel systems such as ASIC/FPGA and multi-core. So it is critical to keep the hash operations faster and more deterministic.This paper presents a novel fast collision-free hashing scheme using Discriminative Bloom Filters (DBFs) to achieve fast and deterministic hash table lookup. DBF is a compact summary stored in on-chip memory. It is composed of an array of parallel Bloom filters organized by the discriminator. Each element lookup performs parallel membership checks on the on-chip DBF to produce a possible discriminator value. Then, the element plus the discriminator value is hashed to a possible bucket in an off-chip hash table for validating the match. This DBF-based scheme requires one off-chip memory access per lookup as well as less off-chip memory usage. Experiments show that our scheme achieves up to 8.5-fold reduction in the number of off-chip memory accesses per lookup than previous schemes.

Forward-Projection Architecture for Fast Iterative Image Reconstruction in X-Ray CT

Iterative image reconstruction can dramatically improve the image quality in X-ray computed tomography (CT), but the computation involves iterative steps of 3D forward- and back-projection, which impedes routine clinical use. To accelerate forward-projection, we analyze the CT geometry to identify the intrinsic parallelism and data access sequence for a highly parallel hardware architecture. To improve the efficiency of this architecture, we propose a water-filling buffer to remove pipeline stalls, and an out-of-order sectored processing to reduce the off-chip memory access by up to three orders of magnitude. We make a floating-point to fixed-point conversion based on numerical simulations and demonstrate comparable image quality at a much lower implementation cost. As a proof of concept, a 5-stage fully pipelined, 55-way parallel separable-footprint forward-projector is prototyped on a Xilinx Virtex-5 FPGA for a throughput of 925.8 million voxel projections/s at 200 MHz clock frequency, 4.6 times higher than an optimized 16-threaded program running on an 8-core 2.8-GHz CPU. A similar architecture can be applied to back-projection for a complete iterative image reconstruction system. The proposed algorithm and architecture can also be applied to hardware platforms such as graphics processing unit and digital signal processor to achieve significant accelerations.

Scalable matrix decompositions with multiple cores on FPGAs

Hardware accelerators are getting increasingly important in heterogeneous systems for many applications, including those that employ matrix decompositions. In recent years, a class of tiled matrix decomposition algorithms has been proposed for out-of-memory computations and multi-core architectures including GPU-based heterogeneous systems. However, on FPGAs these scalable solutions for large matrices are rarely found. In this paper we use the latest tiled decomposition algorithms from high performance linear algebra for off-chip memory access and loop mapping on multiple processing cores for on-chip computation to perform scalable and high performance QR and LU matrix decompositions on FPGAs.

블룸 필터를 사용한 길이에 대한 2차원 이진검색 패킷 분류 알고리즘

패킷 분류는 인터넷 라우터가 수행하는 가장 중요한 기능 중 하나로써 들어오는 모든 패킷을 선 속도로 처리하기를 요구한다. 영역분할을 사용한 사분트라이 구조에 길이 별 이진 검색을 적용한 알고리즘은 2차원 필드를 동시에 검색하면서 검색영역을 반으로 줄여나갈 수 있으므로 매우 효율적인 구조이다. 하지만 트라이의 레벨에 노드가 없는 경우에도 해시 테이블에 접근하는 문제점이 존재한다. 따라서 본 논문에서는 해시 메모리로의 불필요한 접근을 줄이기 위해서 영역분할을 사용한 사분 트라이의 길이별 이진 검색에 블룸 필터를 적용하는 패킷분류 구조를 제안한다. 현재 사용되는 ACL, FW, IPC 룰 타입의 1000, 5000, 10000개의 룰 셋으로 실험한 결과, 블룸 필터를 적용함으로써 검색 성능이 21~33%까지 향상되는 결과를 얻었다. As one of the most challenging tasks in designing the Internet routers, packet classification is required to achieve the wire-speed processing for every incoming packet. Packet classification algorithm which applies binary search on trie levels to the area-based quad-trie is an efficient algorithm. However, it has a problem of unnecessary access to a hash table, even when there is no node in the corresponding level of the trie. In order to avoid the unnecessary off-chip memory access, we proposed an algorithm using Bloom filters along with the binary search on levels to multiple disjoint tries. For ACL, FW, IPC sets with about 1000, 5000, and 10000 rules, performance evaluation result shows that the search performance is improved by 21 to 33 percent by adding Bloom filters.

Date Flow Optimization of Dynamically Coarse Grain Reconfigurable Architecture for Multimedia Applications

This paper proposes a novel sub-architecture to optimize the data flow of REMUS-II (REconfigurable MUltimedia System 2), a dynamically coarse grain reconfigurable architecture. REMUS-II consists of a µPU (Micro-Processor Unit) and two RPUs (Reconfigurable Processor Unit), which are used to speeds up control-intensive tasks and data-intensive tasks respectively. The parallel computing capability and flexibility of REMUS-II makes itself an excellent candidate to process multimedia applications, which require a large amount of memory accesses. In this paper, we specifically optimize the data flow to deal with those performance-hazard and energy-hungry memory accessing in order to meet the bandwidth requirement of parallel computing. The RPU internal memory could work in multiple modes, like 2D-access mode and transformation mode, according to different multimedia access patterns. This novel design can improve the performance up to 26% compared to traditional on-chip memory. Meanwhile, the block buffer is implemented to optimize the off-chip data flow through reducing off-chip memory accesses, which reducing up to 43% compared to direct DDR access. Based on RTL simulation, REMUS-II can achieve 1080p@30fps of H.264 High Profile@ Level 4 and High Level MPEG2 at 200MHz clock frequency. The REMUS-II is implemented into 23.7mm2 silicon on TSMC 65nm logic process with a 400MHz maximum working frequency.

An adaptive true motion estimation algorithm for frame rate conversion of high definition video and its hardware implementations

Frame Rate Up-Conversion (FRUC) is required for displaying low frame rate video signals on high frame rate flat panel displays. In this paper, we propose an adaptive true Motion Estimation (ME) algorithm for FRUC of High Definition (HD) video. The proposed ME algorithm produces similar quality results with less number of calculations or higher quality results with similar number of calculations compared to 3D Recursive Search true ME algorithm by adaptively using optimized sets of candidate search locations and several computational complexity reduction techniques. We also propose two different complexity hardware architectures for implementing this ME algorithm. The second hardware uses an efficient data re-use technique for reducing the number of off-chip memory accesses. We implemented both hardware architectures in Verilog HDL. The hardware implementations are capable of processing 158 and 168 720p HD frames per second on average respectively. Therefore, they can be used in consumer electronics products that require real-time FRUC of HD video.

Multidimensional DFT IP Generator for FPGA Platforms

Multidimensional (MD) discrete Fourier transform (DFT) is a key kernel algorithm in many signal processing applications. In this paper we describe an MD-DFT intellectual property (IP) generator and a bandwidth-efficient MD DFT IP for high performance implementations of 2-D and 3-D DFT on field-programmable gate array (FPGA) platforms. The proposed architecture is generated automatically and is based on a decomposition algorithm that takes into account FPGA resources and the characteristics of off-chip memory access, namely, the burst access pattern of the synchronous dynamic RAM (SDRAM). The IP generator has been integrated into an in-house FPGA development platform, AlgoFLEX, for easy verification and fast integration. The corresponding 2-D and 3-D DFT architectures have been ported onto the BEE3 board and their performance measured and analyzed. The results shows that the architecture can maintain the maximum memory bandwidth throughout the whole procedure while avoiding matrix transpose operations used in most other MD DFT implementations. To further enhance the performance, the proposed architecture is being ported onto the newly released Xilinx ML605 board. The simulation results show that 2 K × 2 K images with complex 64-bit precision can be processed in less than 27 ms.

Fine-grained DVFS using on-chip regulators

Limit studies on Dynamic Voltage and Frequency Scaling (DVFS) provide apparently contradictory conclusions. On the one hand early limit studies report that DVFS is effective at large timescales (on the order of million(s) of cycles) with large scaling overheads (on the order of tens of microseconds), and they conclude that there is no need for small overhead DVFS at small timescales. Recent work on the other hand—motivated by the surge of on-chip voltage regulator research—explores the potential of fine-grained DVFS and reports substantial energy savings at timescales of hundreds of cycles (while assuming no scaling overhead). This article unifies these apparently contradictory conclusions through a DVFS limit study that simultaneously explores timescale and scaling speed. We find that coarse-grained DVFS is unaffected by timescale and scaling speed, however, fine-grained DVFS may lead to substantial energy savings for memory-intensive workloads. Inspired by these insights, we subsequently propose a fine-grained microarchitecture-driven DVFS mechanism that scales down voltage and frequency upon individual off-chip memory accesses using on-chip regulators. Fine-grained DVFS reduces energy consumption by 12% on average and up to 23% over a collection of memory-intensive workloads for an aggressively clock-gated processor, while incurring an average 0.08% performance degradation (and at most 0.14%). We also demonstrate that the proposed fine-grained DVFS mechanism is orthogonal to existing coarse-grained DVFS policies, and further reduces energy by 6% on average and up to 11% for memory-intensive applications with limited performance impact (at most 0.7%).

An embedded compression algorithm integrated with Motion JPEG2000 system for reduction of off-chip video memory bandwidth

In Motion JPEG2000, huge bandwidth requirement of off-chip memory access is the bottleneck in required system performance. For the alleviation of this bandwidth requirement, a low complexity and lossless embedded compression algorithm is devised. And for both random accessibility and low latency, we propose simple and efficient entropy coding algorithm. We achieved significant memory bandwidth reductions (75%), without requiring any modification or performance degradation in JPEG2000 standard algorithm.

Compiler-directed memory management for heterogeneous MPSoCs

Advances in semiconductor technique enable multiple processor cores to be integrated into a single chip. Heterogeneous multiprocessor system-on-a-chip (MPSoC) becomes important platforms to accelerate applications. However, compilation techniques for memory management on MPSoCs still lag behind. This paper presents an automatic memory management framework to orchestrate the data movement between local memory and off-chip memory. In our framework, data alignment, hierarchically data distribution, communication generation, loop tiling, and loop splitting are employed. Moreover, a communication optimization approach is proposed to improve data reuse. These techniques can reduce off-chip memory access and exploit data locality. Experimental results on Cell BE show that our data management framework can generate efficient code for the program.

Data locality and parallelism optimization using a constraint-based approach

Embedded applications are becoming increasingly complex and processing ever-increasing datasets. In the context of data-intensive embedded applications, there have been two complementary approaches to enhancing application behavior, namely, data locality optimizations and improving loop-level parallelism. Data locality needs to be enhanced to maximize the number of data accesses satisfied from the higher levels of the memory hierarchy. On the other hand, compiler-based code parallelization schemes require a fresh look for chip multiprocessors as interprocessor communication is much cheaper than off-chip memory accesses. Therefore, a compiler needs to minimize the number of off-chip memory accesses. This can be achieved by considering multiple loop nests simultaneously. Although compilers address these two problems, there is an inherent difficulty in optimizing both data locality and parallelism simultaneously. Therefore, an integrated approach that combines these two can generate much better results than each individual approach. Based on these observations, this paper proposes a constraint network (CN)-based formulation for data locality optimization and code parallelization. The paper also presents experimental evidence, demonstrating the success of the proposed approach, and compares our results with those obtained through previously proposed approaches. The experiments from our implementation indicate that the proposed approach is very effective in enhancing data locality and parallelization.

Tuple Pruning Using Bloom Filters for Packet Classification

Tuple pruning for packet classification provides fast search and a low implementation complexity. The tuple pruning algorithm reduces the search space to a subset of tuples determined by individual field lookups that cause off-chip memory accesses. The authors propose a tuple-pruning algorithm that reduces the search space through Bloom filter queries, which do not require off-chip memory accesses.

Parallel-pipelined architecture of H.264 deblocking filter with adaptive dynamic power

In H.264, the computational complexity and memory access of deblocking filter are variable and depend on the video contents. In this paper, a pipelined VLSI architecture of deblocking filter with adaptive dynamic power is proposed. It avoids redundant computations and memory access by precluding the blocks which can be skipped. And the vertical and horizontal edges are simultaneously processed in an advanced scan order to speed up the decoder. As a result, the dynamic power of the proposed architecture can be reduced (up to about 89%) adaptively for different videos. And the off-chip memory access is improved compared to the previous designs. Moreover, the processing capability of the proposed architecture is very appropriate for real-time deblocking of high-definition television (HDTV, 1 920× 1 080 pixel/frame, 30 frame/s video signals) video operation at 38 MHz, which significantly outperforms the previous designs from 1.25 times to 4.8 times.

Hierarchical packet classification using a Bloom filter and rule-priority tries

Packet classification techniques have received significant attention in the network literature over the past 10 years, due to its fundamental role in the Internet routers. In recent years, Bloom filter, which is an efficient data structure for membership queries, becomes popular in the network applications. Though Bloom filter allows an error called “false positives,” the efficiency and the space saving overweigh this drawback when the false positive rate is properly controlled. In this paper, we proposed a packet classification algorithm based on a hierarchical approach. While the same data structure is used both for the source and the destination prefix fields in most of other hierarchical packet classification algorithms, our proposed hierarchical packet classification algorithm uses a Bloom filter for the source prefix field and a trie structure for the destination prefix field. The Bloom filter is primarily employed to pre-filter the sub-strings of the source address which have no match for the source prefixes of a given rule set. For the sub-strings with a positive result from the Bloom filter, rule-priority tries constructed based on a destination prefix field determine the highest priority rule matching the input packet for entire rule fields. Since the Bloom filter requires a small amount of memory, it is implemented with an on-chip memory or a fast cache, and hence the off-chip memory accesses are not occurred in the first stage of the hierarchical approach in the proposed algorithm. The proposed packet classification algorithm also provides incremental update. To compare the performance of the proposed packet classification algorithm with other related algorithms, extensive simulations for various algorithms are performed. The simulation result shows that the proposed algorithm renders a better performance in terms of average and worst-case search performance and memory requirement.

High-performance cone beam reconstruction using CUDA compatible GPUs

Compute unified device architecture (CUDA) is a software development platform that allows us to run C-like programs on the nVIDIA graphics processing unit (GPU). This paper presents an acceleration method for cone beam reconstruction using CUDA compatible GPUs. The proposed method accelerates the Feldkamp, Davis, and Kress (FDK) algorithm using three techniques: (1) off-chip memory access reduction for saving the memory bandwidth; (2) loop unrolling for hiding the memory latency; and (3) multithreading for exploiting multiple GPUs. We describe how these techniques can be incorporated into the reconstruction code. We also show an analytical model to understand the reconstruction performance on multi-GPU environments. Experimental results show that the proposed method runs at 83% of the theoretical memory bandwidth, achieving a throughput of 64.3 projections per second (pps) for reconstruction of 512 3-voxel volume from 360 512 2-pixel projections. This performance is 41% higher than the previous CUDA-based method and is 24 times faster than a CPU-based method optimized by vector intrinsics. Some detailed analyses are also presented to understand how effectively the acceleration techniques increase the reconstruction performance of a naive method. We also demonstrate out-of-core reconstruction for large-scale datasets, up to 1024 3-voxel volume.