Reduce Memory Access Research Articles

NUMA-Aware Thread Scheduling for Big Data Transfers over Terabits Network Infrastructure

The evergrowing trend of big data has led scientists to share and transfer the simulation and analytical data across the geodistributed research and computing facilities. However, the existing data transfer frameworks used for data sharing lack the capability to adopt the attributes of the underlying parallel file systems (PFS). LADS (Layout-Aware Data Scheduling) is an end-to-end data transfer tool optimized for terabit network using a layout-aware data scheduling via PFS. However, it does not consider the NUMA (Nonuniform Memory Access) architecture. In this paper, we propose a NUMA-aware thread and resource scheduling for optimized data transfer in terabit network. First, we propose distributed RMA buffers to reduce memory controller contention in CPU sockets and then schedule the threads based on CPU socket and NUMA nodes inside CPU socket to reduce memory access latency. We design and implement the proposed resource and thread scheduling in the existing LADS framework. Experimental results showed from 21.7% to 44% improvement with memory-level optimizations in the LADS framework as compared to the baseline without any optimization.

A Fast MPEG’s CDVS Implementation for GPU Featured in Mobile Devices

The Moving Picture Experts Group’s Compact Descriptors for Visual Search (MPEG’s CDVS) intends to standardize technologies in order to enable an interoperable, efficient, and cross-platform solution for internet-scale visual search applications and services. Among the key technologies within CDVS, we recall the format of visual descriptors, the descriptor extraction process, and the algorithms for indexing and matching. Unfortunately, these steps require precision and computation accuracy. Moreover, they are very time-consuming, as they need running times in the order of seconds when implemented on the central processing unit (CPU) of modern mobile devices. In this paper, to reduce computation times and maintain precision and accuracy, we re-design, for many-cores embedded graphical processor units (GPUs), all main local descriptor extraction pipeline phases of the MPEG’s CDVS standard. To reach this goal, we introduce new techniques to adapt the standard algorithm to parallel processing. Furthermore, to reduce memory accesses and efficiently distribute the kernel workload, we use new approaches to store and retrieve CDVS information on proper GPU data structures. We present a complete experimental analysis on a large and standard test set. Our experiments show that our GPU-based approach is remarkably faster than the CPU-based reference implementation of the standard, and it maintains a comparable precision in terms of true and false positive rates.

Energy-Aware Data Allocation With Hybrid Memory for Mobile Cloud Systems

Resource scheduling is one of the most important issues in mobile cloud computing due to the constraints in memory, CPU, and bandwidth. High energy consumption and low performance of memory accesses have become overwhelming obstacles for chip multiprocessor (CMP) systems used in cloud systems. In order to address the daunting “memory wall” problem, hybrid on-chip memory architecture has been widely investigated recently. Due to its advantages in size, real-time predictability, power, and software controllability, scratchpad memory (SPM) is a promising technique to replace the hardware cache and bridge the processor–memory gap for CMP systems. In this paper, we present a novel hybrid on-chip SPM that consists of a static random access memory (RAM), a magnetic RAM (MRAM), and a zero-capacitor RAM for CMP systems by fully taking advantages of the benefits of each type of memory. To reduce memory access latency, energy consumption, and the number of write operations to MRAM, we also propose a novel multidimensional dynamic programming data allocation (MDPDA) algorithm to strategically allocate data blocks to each memory. Experimental results show that the proposed MDPDA algorithm can efficiently reduce the memory access cost and extend the lifetime of MRAM.

Scalable Bandwidth Shaping Scheme via Adaptively Managed Parallel Heaps in Manycore-Based Network Processors

Scalability of network processor-based routers heavily depends on limitations imposed by memory accesses and associated power consumption. Bandwidth shaping of a flow is a key function, which requires a token bucket per output queue and abuses memory bandwidth. As the number of output queues increases, managing token buckets becomes prohibitively expensive and limits scalability. In this work, we propose a scalable software-based token bucket management scheme that can reduce memory accesses and power consumption significantly. To satisfy real-time and low-cost constraints, we propose novel parallel heap data structures running on a manycore-based network processor. By using cache locking, the performance of heap processing is enhanced significantly and is more predictable. In addition, we quantitatively analyze the performance and memory footprint of the proposed software scheme using stochastic modeling and the Lyapunov central limit theorem. Finally, the proposed scheme provides an adaptive method to limit the size of heaps in the case of oversubscribed queues, which can successfully isolate the queues showing unideal behavior. The proposed scheme reduces memory accesses by up to three orders of magnitude for one million queues sharing a 100Gbps interface of the router while maintaining stability under stressful scenarios.

Lossless image compression algorithm and hardware architecture for bandwidth reduction of external memory

In high definition (HD) video coders, huge memory access bandwidth is the major throughput bottleneck. Lossless embedded compression is an efficient solution to alleviate the bandwidth burden, in which image are compressed before writing into local memory and decompressed after retrieving from local memory. This study proposes a hardware-oriented lossless image compression algorithm, supporting block and line random access flexibly for adapting diverse hardware video codec architectures. The major contributions are characterised as follows. First, block or pixel-level adaptive prediction is proposed to fully utilise the image spatial correlation by employing adaptive mode decision. Second, multiple-range semi-fixed (SF) variable length coding (VLC) is employed to describe the prediction residue, and adaptive block size selection is employed for SF VLC to fully utilise the statistical redundancy. In addition, Huffman VLC is further employed to represent the control syntax elements. Third, four-stage pipeline hardware architecture is proposed to implement the proposed algorithm. Simulation results show that the proposed algorithm achieves competitive rate compression performance compared with reference algorithms. The proposed hardware architecture is verified supporting real-time processing for quad-HD videos at the frequency of 166 MHz. The proposed work achieves reducing memory access bandwidth by ~55.2%, which is useful for hardwired video coding.

LA-LLC: Inter-Core Locality-Aware Last-Level Cache to Exploit Many-to-Many Traffic in GPGPUs

The reply network is a severe performance bottleneck in General Purpose Graphic Processing Units (GPGPUs), as the communication path from memory controllers (MC) to cores is often congested. In this paper, we find that instead of relying on the congested communication path between MCs and cores, the unused core-to-core communication path can be leveraged to transfer data blocks between cores. We propose the inter-core Locality-Aware Last-Level Cache (LA-LLC), which requires only few bits per cache block and enables a core to fetch shared data from another core's private cache instead of the LLC. Leveraging inter-core communication, LA-LLC transforms few-to-many traffic to many-to-many traffic, thereby mitigating the reply network bottleneck. For a set of applications exhibiting varying degrees of inter-core locality, LA-LLC reduces memory access latency and increases performance by 21.1 percent on average and up to 68 percent, with negligible hardware cost.

TU-AB-BRC-10: Modeling of Radiotherapy Linac Source Terms Using ARCHER Monte Carlo Code: Performance Comparison of GPU and MIC Computing Accelerators

Purpose: (1) To perform phase space (PS) based source modeling for Tomotherapy and Varian TrueBeam 6 MV Linacs, (2) to examine the accuracy and performance of the ARCHER Monte Carlo code on a heterogeneous computing platform with Many Integrated Core coprocessors (MIC, aka Xeon Phi) and GPUs, and (3) to explore the software micro-optimization methods. Methods: The patient-specific source of Tomotherapy and Varian TrueBeam Linacs was modeled using the PS approach. For the helical Tomotherapy case, the PS data were calculated in our previous study (Su et al. 2014 41(7) Medical Physics). For the single-view Varian TrueBeam case, we analytically derived them from the raw patient-independent PS data in IAEA's database, partial geometry information of the jaw and MLC as well as the fluence map. The phantom was generated from DICOM images. The Monte Carlo simulation was performed by ARCHER-MIC and GPU codes, which were benchmarked against a modified parallel DPM code. Software micro-optimization was systematically conducted, and was focused on SIMD vectorization of tight for-loops and data prefetch, with the ultimate goal of increasing 512-bit register utilization and reducing memory access latency. Results: Dose calculation was performed for two clinical cases, a Tomotherapy-based prostate cancer treatment and a TrueBeam-based left breast treatment. ARCHER was verified against the DPM code. The statistical uncertainty of the dose to the PTV was less than 1%. Using double-precision, the total wall time of the multithreaded CPU code on a X5650 CPU was 339 seconds for the Tomotherapy case and 131 seconds for the TrueBeam, while on 3 5110P MICs it was reduced to 79 and 59 seconds, respectively. The single-precision GPU code on a K40 GPU took 45 seconds for the Tomotherapy dose calculation. Conclusion: We have extended ARCHER, the MIC and GPU-based Monte Carlo dose engine to Tomotherapy and Truebeam dose calculations.

Memory-efficient table look-up optimized algorithm for context-based adaptive variable length decoding in H.264/advanced video coding

Table look-up operation plays a very important role during the decoding processing of context-based adaptive variable length decoding (CAVLD) in H.264/advanced video coding (AVC). However, frequent table look-up operation can result in big table memory access, and then lead to high table power consumption. Aiming to solve the problem of big table memory access of current methods, and then reduce high power consumption, a memory-efficient table look-up optimized algorithm is presented for CAVLD. The contribution of this paper lies that index search technology is introduced to reduce big memory access for table look-up, and then reduce high table power consumption. Specifically, in our schemes, we use index search technology to reduce memory access by reducing the searching and matching operations for code_word on the basis of taking advantage of the internal relationship among length of zero in code_prefix, value of code_suffix and code_lengh, thus saving the power consumption of table look-up. The experimental results show that our proposed table look-up algorithm based on index search can lower about 60% memory access consumption compared with table look-up by sequential search scheme, and then save much power consumption for CAVLD in H.264/AVC.

Instruction scheduling heuristic for an efficient FFT in VLIW processors with balanced resource usage

The fast Fourier transform (FFT) is perhaps today’s most ubiquitous algorithm used with digital data; hence, it is still being studied extensively. Besides the benefit of reducing the arithmetic count in the FFT algorithm, memory references and scheme’s projection on processor’s architecture are critical for a fast and efficient implementation. One of the main bottlenecks is in the long latency memory accesses to butterflies’ legs and in the redundant references to twiddle factors. In this paper, we describe a new FFT implementation on high-end very long instruction word (VLIW) digital signal processors (DSP), which presents improved performance in terms of clock cycles due to the resulting low-level resource balance and to the reduced memory accesses of twiddle factors. The method introduces a tradeoff parameter between accuracy and speed. Additionally, we suggest a cache-efficient implementation methodology for the FFT, dependently on the provided VLIW hardware resources and cache structure. Experimental results on a TI VLIW DSP show that our method reduces the number of clock cycles by an average of 51 % (2 times acceleration) when compared to the most assembly-optimized and vendor-tuned FFT libraries. The FFT was generated using an instruction-level scheduling heuristic. It is a modulo-based register-sensitive scheduling algorithm, which is able to compute an aggressively efficient sequence of VLIW instructions for the FFT, maximizing the parallelism rate and minimizing clock cycles and register usage.

첨단운전자보조시스템용 이동객체검출을 위한 광학흐름추정기의 설계 및 구현

In this paper, the design and implementation results of the optical flow estimator (OFE) for moving object detection (MOD) in advanced driver assistance system (ADAS). In the proposed design, Brox's algorithm with global optimization is considered, which shows the high performance in the vehicle environment. In addition, Cholesky factorization is applied to solve Euler-Lagrange equation in Brox's algorithm. Also, shift register bank is incorporated to reduce memory access rate. The proposed optical flow estimator was designed with Verilog-HDL, and FPGA board was used for the real-time verification. Implementation results show that the proposed optical flow estimator includes the logic slices of 40.4K, 155 DSP48s, and block memory of 11,290Kbits.

Data Allocation for Hybrid Memory With Genetic Algorithm

The gradually widening speed disparity between CPU and memory has become an overwhelming bottleneck for the development of chip multiprocessor systems. In addition, increasing penalties caused by frequent on-chip memory accesses have raised critical challenges in delivering high memory access performance with tight power and latency budgets. To overcome the daunting memory wall and energy wall issues, this paper focuses on proposing a new heterogeneous scratchpad memory architecture, which is configured from SRAM, MRAM, and Z-RAM. Based on this architecture, we propose a genetic algorithm to perform data allocation to different memory units, therefore, reducing memory access cost in terms of power consumption and latency. Extensive and experiments are performed to show the merits of the heterogeneous scratchpad architecture over the traditional pure memory system and the effectiveness of the proposed algorithms.

Architecture and data migration methodology for L1 cache design with hybrid SRAM and volatile STT-RAM configuration

Spin-Transfer Torque RAM (STT-RAM) has the advantages of circuit density and ignorable leakage power. However, it suffers from the bad write latency and poor write power consumption. Therefore, it is difficult to replace entire SRAM with STT-RAM in the L1 cache, but we can relax the retention time of STT-RAM cell to improve its write performance and replace some of the SRAM capacity to reduce leakage power. In this paper, we propose a locality-aware approach for L1 cache design with hybrid SRAM and volatile STT-RAM configuration. Based on the principle of cache locality, data block is mapped to SRAM firstly to reduce write latency and write energy, and is moved to volatile STT-RAM to reduce leakage power consumption. After a time period when there is no access of a data block in the volatile STT-RAM, we then stop its refresh operations to further reduce power consumption. Experimental results show that in comparison with the SRAM only L1 cache configuration, our hybrid cache configuration and data migration methodology reduce energy consumption by about 15–20%, with only nearly to 5% of latency overhead. Also when comparing to the STT-RAM only L1 cache configuration, we reduce memory access latency nearly to 20% with close or even better energy consumption.

Hybrid Main Memory for High Bandwidth Multi-Core System

In the hybrid main memory sub-system which consists of DRAM and phase change RAM (PRAM), DRAM is often used as a cache. Such a configuration of DRAM cache and PRAM main memory has two problems. First, it requires high bandwidth of data movement between DRAM and PRAM due to the large granularity (i.e., DRAM row) of caching. Second, the memory bandwidth of PRAM is not fully utilized since it can be accessed only by the DRAM cache thereby limiting the memory bandwidth of hybrid main memory to that of DRAM cache. In this paper, we propose a novel architecture where both DRAM and PRAM can be accessed in parallel for higher memory performance. We analyze that the performance and fairness of hybrid memory sub-system is limited by the stall time at the data queue of memory controller. In order to resolve this problem, we propose caching data selectively in a way to reduce stall time thereby reducing memory access latency and improving fairness. Our experimental results show significant improvements in performance (by an average of 21 percent), fairness (by 2.1 times), and energy consumption (by 10 percent) compared with the best of the existing methods in a multi-core system which consists of multiple CPUs and GPU.

Multiple clone row DRAM

Several previous works have changed DRAM bank structure to reduce memory access latency and have shown performance improvement. However, changes in the area-optimized DRAM bank can incur large area-overhead. To solve this problem, we propose Multiple Clone Row DRAM (MCR-DRAM), which uses existing DRAM bank structure without any modification. Our key idea is Multiple Clone Row (MCR), in which multiple rows are simultaneously turned on or off to consist of a logically single row. MCR provides two advantages which enable our low-latency mechanisms (Early-Access, Early-Precharge and Fast-Refresh). First, MCR increases the speed of the sensing process by increasing the number of sensed-cells. Thus, it enables a READ/WRITE command to an MCR to be issued earlier than possible for a normal row (Early-Access). Second, DRAM cells in an MCR exhibit more frequent refreshes without additional REFRESH commands, thereby reducing the amount of charge leakage during the refresh interval for the identical cell. The reduced amount of charge leakage enables a PRECHARGE command to be served before the activated-cells are fully restored (Early-Precharge) and a REFRESH operation to be completed before the refreshed-cells are fully restored (Fast-Refresh). Even though MCR-DRAM sacrifices memory capacity for low-latency, it can be dynamically reconfigured from low-latency to full-capacity DRAM. MCR-DRAM improves both performance and energy efficiency for both single-core and multi-core systems.

A Vocabulary Forest Object Matching Processor With 2.07 M-Vector/s Throughput and 13.3 nJ/Vector Per-Vector Energy for Full-HD 60 fps Video Object Recognition

Approximate nearest neighbor searching has been studied as the keypoint matching algorithm for object recognition systems, and its hardware realization has reduced the external memory access which is the main bottleneck in object recognition process. However, external memory access reduction alone cannot satisfy the ever-increasing memory bandwidth requirement due to the rapid increase of the image resolution and frame rate of many recent applications such as advanced driver assistance system. In this paper, vocabulary forest (VF) processor is proposed that achieves both high accuracy and high speed by integrating on-chip database (DB) to remove external memory access. The area-efficient reusable-vocabulary tree architecture is proposed to reduce area, and the propagate-and-compute-array architecture is proposed to enhance the processing speed of the VF. The proposed VF processor can speed up the object matching stage by 16.4x compared with the state-of-the-art matching processor [Hong et al., Symp. VLSIC, 2013] for high resolution (Full-HD) and real-time (60 fps) video object recognition. It is fabricated using 65 nm CMOS technology and integrated into an object recognition SoC. The proposed VF chip achieves 2.07 M-vector/s throughput and 13.3 nJ/vector per-vector energy with 95.7% matching accuracy for 100 objects.

Data Allocation with Minimum Cost under Guaranteed Probability for Multiple Types of Memories

As the advance of memory technologies, multiple types of memories such as different kinds of non-volatile memory (NVM), SRAM, DRAM, etc. provide a flexible configuration considering performance, energy and cost. For improving the performance of systems with multiple types of memories, data allocation is one of the most important tasks. The previous studies on data allocation problem assume the worst (fixed) case of data-access frequencies. However, the data allocation produced by employing worst case usually leads to an inferior performance for most of time. In this paper, we model this problem by probabilities and design efficient algorithms that can give optimal-cost data allocation with a guaranteed probability. We propose DAGP algorithm produces a set of feasible data allocation solutions which generates the minimum access time or cost guaranteed by a given probability. We also propose a polynomial-time algorithm, MCS algorithm, to solve this problem. The experiments show that our technique can significantly reduce the access cost compared with the technique considering worst case scenario. For example, comparing with the optimal result generated by employing the worst cases, DAGP can reduce memory access cost by 9.92 % on average when guaranteed probability is set to be 0.9. Moreover, for 90 percents of cases, memory access time is reduced by 12.47 % on average. Comparing with greedy algorithm, DAGP and MCS can reduce memory access cost by 78.92 % and 44.69 % on average when guaranteed probability is set to be 0.9.

Implementation of Multi-GPU Based Lattice Boltzmann Method for Flow Through Porous Media

AbstractThe lattice Boltzmann method (LBM) can gain a great amount of performance benefit by taking advantage of graphics processing unit (GPU) computing, and thus, the GPU, or multi-GPU based LBM can be considered as a promising and competent candidate in the study of large-scale fluid flows. However, the multi-GPU based lattice Boltzmann algorithm has not been studied extensively, especially for simulations of flow in complex geometries. In this paper, through coupling with the message passing interface (MPI) technique, we present an implementation of multi-GPU based LBM for fluid flow through porous media as well as some optimization strategies based on the data structure and layout, which can apparently reduce memory access and completely hide the communication time consumption. Then the performance of the algorithm is tested on a one-node cluster equipped with four Tesla C1060 GPU cards where up to 1732 MFLUPS is achieved for the Poiseuille flow and a nearly linear speedup with the number of GPUs is also observed.

Fast bitwise pattern-matching algorithm for DNA sequences on modern hardware

We introduce a fast bitwise exact pattern-matching algorithm, which speeds up short-length pattern searches on large-sized DNA databases. Our contributions are two-fold. First, we introduce a novel exact matching algorithm designed specifically for modern processor architectures. Second, we conduct a detailed comparative performance analysis of bitwise exact matching algorithms by utilizing hardware counters. Our algorithmic technique is based on condensed bitwise operators and multifunction variables, which minimize register spills and instruction counts during searches. In addition, the technique aims to efficiently utilize CPU branch predictors and to ensure smooth instruction flow through the processor pipeline. Analyzing letter occurrence probability estimations for DNA databases, we develop a skip mechanism to reduce memory accesses. For comparison, we exploit the complete \textit{Mus musculus} sequence, a commonly used DNA sequence that is larger than 2 GB. Compared to five state-of-the-art pattern-matching algorithms, experimental results show that our technique outperforms the best algorithm even for the worst-case DNA pattern for our technique.

Selective Cache Line Replication Scheme in Shared Last Level Cache

In current multi-core systems with the shared last level cache (LLC) physically distributed across all the cores, both initial data placement and subsequent placement of data close to the requesting core can contribute to reducing memory access latency and power consumption. This paper extends a replication scheme that balances between access latency and cache capacity in shared NUCA designs by selectively replicating frequently used cache lines close to the requesting cores. Our scheme reduces completion time by 15% and improves energy consumption by 27% when compared to the Static-NUCA (S-NUCA) management scheme, when simulated on an eight core system.

An Overview of H.264 Hardware Encoder Architectures Including Low-Power Features

H.264 is the most popular video coding standard with high potent coding performance. For its efficiency, the H.264 is expected to encode real-time and/or high-definition video. However, the H.264 standard also requires highly complex and long lasting computation. To overcome these difficulties, many efforts have been deployed to increase encoding speed. Besides, with the revolution of portable devices, multimedia chips for mobile environments are more and more developed. Thus, power-oriented design for H.264 video encoders is currently a tremendous challenge. This paper discusses these trends and presents an overview of the state of the art on power features for different H.264 hardware encoding architectures. We also propose the VENGME’s design, a particular hardware architecture of H.264 encoder that enables applying low-power techniques and developing power-aware ability. This low power encoder is a four-stage architecture with memory access reduction, in which, each module has been optimized. The actual total power consumption, estimated at Register-Transfer-Level (RTL), is only 19.1 mW.