Texture Cache Research Articles

Low occupancy high performance elemental products in assembly free FEM on GPU

Assembly free FEM bypasses the assembly step and solves the system of linear equations at the element level using Conjugate Gradient (CG) type iterative solver. The smaller dense Matrix-vector Products (MvPs) are encapsulated within the CG solver and are computed either at element level or degree of freedom (DoF) level. Both these strategies exploit the computing power of GPU effectively, but the performance is lagging due to the uncoalesced global memory access on GPU. This paper proposes an improved MvP strategy in assembly free FEM, which improves the performance by coalesced global memory access using on-chip faster shared memory and using the texture cache memory on GPU. Since GPU has limited shared memory (in few KBs), the proposed technique suffers from a problem known as low occupancy. Despite the low occupancy issue, the proposed strategy outperforms both element based and DoF based MvP strategies on GPU. Numerical experiments compared with element level and DoF level strategies on GPU and found that, GPU instance of proposed MvP outperforms both strategies approximately by factor of 7 and 1.5 respectively.

Cache-aware volume rendering methods with dynamic data reorganization

We present two cache-aware methods for accelerating volume rendering on graphics processing units (GPUs). The proposed methods extend a previous method such that they improve the worst frame rate typically observed at oblique viewing angles. The key technique for realizing this extension is a dynamic reorganization of the volume data that improves the data locality based on the expected data access pattern. The proposed methods are: (1) a basic method which requires three times the amount of memory to store the volume and (2) an adaptive method which utilizes on-the-fly data reorganization to achieve similar performance without increasing the memory consumption. Experimental results indicate that the GPU texture cache hit rate increased from 58.8 to 71.8% for oblique viewing angles using a 1 GB data set, compared to the previous method. Furthermore, the maximum speedup of the proposed adaptive method was 3.6 $$\times$$ that of the previous method.

Design of texture cache for embedded mobile graphics processing unit

Design of texture cache for embedded mobile graphics processing unit

A cache-friendly sampling strategy for texture-based volume rendering on GPU

The texture-based volume rendering is a memory-intensive algorithm. Its performance relies heavily on the performance of the texture cache. However, most existing texture-based volume rendering methods blindly map computational resources to texture memory and result in incoherent memory access patterns, causing low cache hit rates in certain cases. The distance between samples taken by threads of an atomic scheduling unit (e.g. a warp of 32 threads in CUDA) of the GPU is a crucial factor that affects the texture cache performance. Based on this fact, we present a new sampling strategy, called Warp Marching, for the ray-casting algorithm of texture-based volume rendering. The effects of different sample organizations and different thread-pixel mappings in the ray-casting algorithm are thoroughly analyzed. Also, a pipeline manner color blending approach is introduced and the power of warp-level GPU operations is leveraged to improve the efficiency of parallel executions on the GPU. In addition, the rendering performance of the Warp Marching is view-independent, and it outperforms existing empty space skipping techniques in scenarios that need to render large dynamic volumes in a low resolution image. Through a series of micro-benchmarking and real-life data experiments, we rigorously analyze our sampling strategies and demonstrate significant performance enhancements over existing sampling methods.

Dissecting GPU Memory Hierarchy Through Microbenchmarking

Memory access efficiency is a key factor in fully utilizing the computational power of graphics processing units (GPUs). However, many details of the GPU memory hierarchy are not released by GPU vendors. In this paper, we propose a novel fine-grained microbenchmarking approach and apply it to three generations of NVIDIA GPUs, namely Fermi, Kepler and Maxwell, to expose the previously unknown characteristics of their memory hierarchies. Specifically, we investigate the structures of different GPU cache systems, such as the data cache, the texture cache and the translation look-aside buffer (TLB). We also investigate the throughput and access latency of GPU global memory and shared memory. Our microbenchmark results offer a better understanding of the mysterious GPU memory hierarchy, which will facilitate the software optimization and modelling of GPU architectures. To the best of our knowledge, this is the first study to reveal the cache properties of Kepler and Maxwell GPUs, and the superiority of Maxwell in shared memory performance under bank conflict.

IP Address Lookup by Using GPU

We present a novel parallel IP address lookup architecture based on graphics processing unit (GPU) via compute unified device architecture (CUDA). Our architecture consists of two functions: 1) host function and 2) device function. The host function is executed by a CPU to construct and update the data structure of IP address lookup executed by the device function in a GPU. Both host and device functions are executed simultaneously to fully utilize computational resources. To shorten the lookup time, a trie-based data structure optimized for CUDA is developed. The trie-based data structure uses multi-bit stride to shorten the trie depth and also improves the efficiency of texture cache in GPUs. The experimental results show that a low-end G92 GPU can achieve a throughput of more than 1.3 billion packets per second for IPv4 routing tables with more than 350K prefixes while a high-end GT200 GPU can further double the performance. By employing dual data structures, the implementation can support several hundred thousand updates per second.

Code reordering using local random extraction and insertion (LREI) operator for GPGPU-based track-before-detect systems

Track-before-detect (TBD) algorithms are used for tracking systems, where the object's signal is below the noise floor (low-SNR objects). A lot of computations and memory transfers for real-time signal processing are necessary. GPGPU in parallel processing devices for TBD algorithms is well suited. Finding optimal or suboptimal code, due to lack of documentation for low-level programming of GPGPUs is not possible. High-level code optimization is necessary and the evolutionary approach, based on the single parent and single child is considered, that is local search approach. Brute force search technique is not feasible, because there are N! code variants, where N is the number of motion vectors components. The proposed evolutionary operator--LREI (local random extraction and insertion) allows source code reordering for the reduction of computation time due to better organization of memory transfer and the texture cache content. The starting point, based on the sorting and the minimal execution time metric is proposed. The unbiased random and biased sorting techniques are compared using experimental approach. Tests shows significant improvements of the computation speed, about 8 % over the conventional code for CUDA code. The time period of optimization for the sample code is about 1 h (1,000 iterations) for the considered recursive spatio-temporal TBD algorithm.

Boosting mobile GPU performance with a decoupled access/execute fragment processor

Smartphones represent one of the fastest growing markets, providing significant hardware/software improvements every few months. However, supporting these capabilities reduces the operating time per battery charge. The CPU/GPU component is only left with a shrinking fraction of the power budget, since most of the energy is consumed by the screen and the antenna. In this paper, we focus on improving the energy efficiency of the GPU since graphical applications consist an important part of the existing market. Moreover, the trend towards better screens will inevitably lead to a higher demand for improved graphics rendering. We show that the main bottleneck for these applications is the texture cache and that traditional techniques for hiding memory latency (prefetching, multithreading) do not work well or come at a high energy cost. We thus propose the migration of GPU designs towards the decoupled access-execute concept. Furthermore, we significantly reduce bandwidth usage in the decoupled architecture by exploiting inter-core data sharing. Using commercial Android applications, we show that the end design can achieve 93% of the performance of a heavily multithreaded GPU while providing energy savings of 34%.

Texture Caches

This column examines the texture cache, an essential component of modern GPUs that plays an important role in achieving real-time performance when generating realistic images. GPUs have many components and the texture cache is only one of them. But it has a real impact on the performance of the GPU if rasterization and memory tiling are set up correctly.

From CUDA to OpenCL: Towards a performance-portable solution for multi-platform GPU programming

In this work, we evaluate OpenCL as a programming tool for developing performance-portable applications for GPGPU. While the Khronos group developed OpenCL with programming portability in mind, performance is not necessarily portable. OpenCL has required performance-impacting initializations that do not exist in other languages such as CUDA. Understanding these implications allows us to provide a single library with decent performance on a variety of platforms. We choose triangular solver (TRSM) and matrix multiplication (GEMM) as representative level 3 BLAS routines to implement in OpenCL. We profile TRSM to get the time distribution of the OpenCL runtime system. We then provide tuned GEMM kernels for both the NVIDIA Tesla C2050 and ATI Radeon 5870, the latest GPUs offered by both companies. We explore the benefits of using the texture cache, the performance ramifications of copying data into images, discrepancies in the OpenCL and CUDA compilers' optimizations, and other issues that affect the performance. Experimental results show that nearly 50% of peak performance can be obtained in GEMM on both GPUs in OpenCL. We also show that the performance of these kernels is not highly portable. Finally, we propose the use of auto-tuning to better explore these kernels' parameter space using search harness.

A Dual-Shader 3-D Graphics Processor With Fast 4-D Vector Inner Product Units and Power-Aware Texture Cache

This paper presents a fully programmable 3-D graphics processor using unified shaders for mobile environment. In the system level, we adopted dual-core, dual-issue VLIW, and multithreading methods to utilize instruction, data, and task level parallelism in the graphics applications. In the shader core level, a novel IEEE-754 compliant 4-D vector inner product arithmetic unit and a configurable texture cache are proposed. Using these methods, the proposed processor achieves 143 Mvertices/s and 2.3 Gtexels/s consuming the power of 367 mW. The evaluation shows significant performance and power-delay product benefits. For real graphics applications, test results indicate 2.07 times improvement in performance and 34% reduction in power-delay product compared to previous mobile 3-D graphics processors. The proposed 3-D graphics processor is implemented in 4.5× 4.52 mm using 0.18 μm CMOS technology.

A fast GEMM implementation on the cypress GPU

We present benchmark results of optimized dense matrix multiplication kernels for Cypress GPU. We write general matrix multiply (GEMM) kernels for single (SP), double (DP) and double-double (DDP) precision. Our SGEMM and DGEMM kernels show ~ 2 Top/s and ~ 470 Glop/s, respectively. These results for SP and DP correspond to 73% and 87% of the theoretical performance of the GPU, respectively. Currently, our SGEMM and DGEMM kernels are fastest with one GPU chip to our knowledge. Furthermore, the performance of our matrix multiply kernel in DDP is 31 Gop/s. This performance in DDP is more than 200 times faster than the performance results in DDP on single core of a recent CPU (with mpack version 0.6.5). We describe our GEMM kernels with main focus on the SGEMM implementation since all GEMM kernels share common programming and optimization techniques. While a conventional wisdom of GPU programming recommends us to heavily use shared memory on GPUs, we show that texture cache is very effective on the Cypress architecture.

Performance Evaluation and Optimization on GPU

GPU provides higher peak performance with hundreds of cores than CPU counterpart. However, it is a big challenge to take full advantage of their computing power. In order to understand performance bottlenecks of applications on many-core GPU and then optimize parallel programs on GPU architectures, we propose a performance evaluating model based on memory wall and then classify applications into AbM (Application bound-in Memory) and AbC (Application bound-in Computing). Furthermore, we optimize kernels characterized with low memory bandwidth including matrix multiplication and FFT (Fast Fourier Transform) by employing texture cache on NVIDIA GTX280 using CUDA (Compute Unified Device Architecture). Experimental results show that texture cache is helpful for AbM with better data locality, so it is critical to utilize GPU memory hierarchy efficiently for performance improvement.

Fast Image-Based Ambient Occlusion IBAO

The quality of computer rendering and perception of realism greatly depend on the shading method used to implement the interaction of light with the surfaces of objects in a scene. Ambient occlusion (AO) enhances the realistic impression of rendered objects and scenes. Properties that make Screen Space Ambient Occlusion (SSAO) interesting for real-time graphics are scene complexity independence, and support for fully dynamic scenes. However, there are also important issues with current approaches: poor texture cache use, introduction of noise, and performance swings. In this paper, a straightforward solution is presented. Instead of a traditional, geometry-based sampling method, a novel, image-based sampling method is developed, coupled with a revised heuristic function for computing occlusion. Proposed algorithm harnessing GPU power improves texture cache use and reduces aliasing artifacts. Two implementations are developed, traditional and novel, and their comparison reveals improved performance and quality of the proposed algorithm.

Parallelizing Network Coding Using CUDA

Network coding has emerged as a promising technique to improve network throughput and bandwidth. However, due to high computational complexity, its practicability has remained to be a challenge. At the same time, applications accelerated by GPU are confined to GPU acting as a coprocessor to consume dataset transferred from CPU. Therefore, an aggressive parallel network coding is customized for GPU using CUDA (Compute Unified Device Architecture), in which dataset are partitioned for exploiting both thread-level parallelism and data-level parallelism, and collaboration between GPU and CPU is introduced to decoding with texture cache so that GPU can act as not only data consumer but also data producer. Moreover, random linear network coding is parallelizing on CUDA-enabled GPU to validate proposed techniques. Experimental results demonstrate that it is effective to parallelize network coding on GPU-accelerated system using proposed techniques.

Fast rendering of dynamic clouds

We propose an efficient framework for dynamic cloud rendering. Based on the proposed simplified lighting model, the shadow relation table and metaball lighting texture database are constructed in the preprocessing, and features such as self-shadowing and light scattering can be computed quickly using table lookup at the run time. Clouds are rendered using textured billboards and alpha blending, which is further speeded up by incorporating the octree hierarchy and the hierarchical texture caching.

Miss-rate reduction in texture cache by adaptive cache indexing

An adaptive cache indexing is proposed to reduce the memory bandwidth required for fetching texture image data by reducing the cache miss-rate through adaptive selection of index bits. By designing a texture mapping hardware including a texture cache, the number of cache miss and total cycles are examined for texture mapping from cycle-accurate HDL simulations on some test scenes. The number of cache miss and total cycles were reduced by 23% and 8.9% respectively by the adaptive cache indexing.

Perceptually-driven decision theory for interactive realistic rendering

In this paper we introduce a new approach to realistic rendering at interactive rates on commodity graphics hardware. The approach uses efficient perceptual metrics within a decision theoretic framework to optimally order rendering operations, producing images of the highest visual quality within system constraints. We demonstrate the usefulness of this approach for various applications such as diffuse texture caching, environment map prioritization and radiosity mesh simplification. Although here we address the problem of realistic rendering at interactive rates, the perceptually-based decision theoretic methodology we introduce can be usefully applied in many areas of computer graphics.

A reconfigurable multilevel parallel texture cache memory with 75-GB/s parallel cache replacement bandwidth

Recently, the level of realism in PC graphics applications has been approaching that of high-end graphics workstations, necessitating a more sophisticated texture data cache memory to overcome the finite bandwidth of the AGP or PCI bus. This paper proposes a multilevel parallel texture cache memory to reduce the required data bandwidth on the AGP or PCI bus and to accelerate the operations of parallel graphics pipelines in PC graphics cards. The proposed cache memory is fabricated by 0.16-/spl mu/m DRAM-based SOC technology. It is composed of four components: an 8-MB DRAM L2 cache, 8-way parallel SRAM L1 caches, pipelined texture data filters, and a serial-to-parallel loader. For high-speed parallel L1 cache data replacement, the internal bus bandwidth has been maximized up to 75 GB/s with a newly proposed hidden double data transfer scheme. In addition, the cache memory has a reconfigurable architecture in its line size for optimal caching performance in various graphics applications from three-dimensional (3-D) games to high-quality 3-D movies.

Multi-level texture caching for 3D graphics hardware

Traditional graphics hardware architectures implement what we call the push architecture for texture mapping. Local memory is dedicated to the accelerator for fast local retrieval of texture during rasterization, and the application is responsible for managing this memory. The push architecture has a bandwidth advantage, but disadvantages of limited texture capacity, escalation of accelerator memory requirements (and therefore cost), and poor memory utilization. The push architecture also requires the programmer to solve the bin- packing problem of managing accelerator memory each frame. More recently graphics hardware on PC-class machines has moved to an implementation of what we call the pull architecture. Texture is stored in system memory and downloaded by the accelerator as needed. The pull architecture has advantages of texture capacity, stems the escalation of accelerator memory requirements, and has good memory utilization. It also frees the programmer from accelerator texture memory management. However, the pull architecture suffers escalating requirements for bandwidth from main memory to the accelerator. In this paper we propose multi-level texture caching to provide the accelerator with the bandwidth advantages of the push architecture combined with the capacity advantages of the pull architecture. We have studied the feasibility of 2-level caching and found the following: (1) significant re-use of texture between frames; (2) L2 caching requires significantly less memory than the push architecture; (3) L2 caching requires significantly less bandwidth from host memory than the pull architecture; (4) L2 caching enables implementation of smaller L1 caches that would otherwise bandwidth-limit accelerators on the workloads in this paper. Results suggest that an L2 cache achieves the original advantage of the pull architecture --- stemming the growth of local texture memory --- while at the same time stemming the current explosion in demand for texture bandwidth between host memory and the accelerator.