Foveated Rendering Research Articles

Individualized foveated rendering with eye-tracking head-mounted display

Foveated rendering (FR) technology is designed to improve the efficiency of graphical rendering processes. In rendering, individualized approaches can help to balance users’ experiences of visual quality and saving computational resource. However, previous studies have not rigorously examined it related with the FR techniques. To address this issue, we developed an individualized FR (IFR) method using different central vision sizes and peripheral vision resolutions across individuals in virtual reality. In three user studies with 88 participants who were divided into groups designated as “large central area (LCA)” and “small central area (SCA),” the effects of IFR were compared with those of using the full-resolution condition and the average FR condition. The results indicate that the LCA group experienced higher visual quality under the IFR and full-resolution conditions than under the average FR condition. In contrast, the SCA group exhibited comparable levels of dependent measures between the IFR and average FR conditions, but both were lower than those of the full-resolution condition. We also evaluated the computational benefits of the proposed IFR method, and the results demonstrated the effectiveness of our approach in saving resources across the two groups. Although lower-bound adjustments may be required for some users, our overall results suggest that IFR is a malleable technology for enhancing rendering efficiency in virtual reality.

A Cloud-Edge Collaborative Gaming Framework Using AI-Powered Foveated Rendering and Super Resolution

Cloud gaming (CG) has gradually gained popularity. By leveling shared computing resources on the cloud, CG technology allows those without expensive hardware to enjoy AAA games using a low-end device. However, the bandwidth requirement for streaming game video is high, which can cause backbone network congestion for large-scale deployment and expensive bandwidth bills. To address this challenge, the authors proposed an innovative edge-assisted computing architecture that collaboratively uses AI-powered foveated rendering (FR) and super-resolution (SR). Using FR, the cloud server can stream gaming video in lower resolution, significantly reducing the transmitted data volume. The edge server will then upscale the video using a game-specific SR model, recovering the quality of the video, especially for the areas players pay the most attention. The authors built a prototype system called FRSR and did thorough, objective comparative experiments to demonstrate that this architecture can reduce bandwidth usage by 39.47% compared with classic CG implementation for similar perceived quality.

29‐1: Common Triangle Optimizations and Coarse Pixel Shading for Foveated Rendering Acceleration

AbstractImproved foveated rendering performance for head‐mounted displays (HMDs) requires re‐architecting a multi‐view render pipeline. This pipeline must manage common triangle culling/clipping operations across views while reordering triangle vs. view processing and varying the rasterization sampling rate. This paper explores render pipelines for HMD rendering improvement.

Kernel Foveated Rendering

Foveated rendering coupled with eye-tracking has the potential to dramatically accelerate interactive 3D graphics with minimal loss of perceptual detail. In this paper, we parameterize foveated rendering by embedding polynomial kernel functions in the classic log-polar mapping. Our GPU-driven technique uses closed-form, parameterized foveation that mimics the distribution of photoreceptors in the human retina. We present a simple two-pass kernel foveated rendering (KFR) pipeline that maps well onto modern GPUs. In the first pass, we compute the kernel log-polar transformation and render to a reduced-resolution buffer. In the second pass, we carry out the inverse-log-polar transformation with anti-aliasing to map the reduced-resolution rendering to the full-resolution screen. We have carried out pilot and formal user studies to empirically identify the KFR parameters. We observe a 2.8X -- 3.2X speedup in rendering on 4K UHD (2160p) displays with minimal perceptual loss of detail. The relevance of eye-tracking-guided kernel foveated rendering can only increase as the anticipated rise of display resolution makes it ever more difficult to resolve the mutually conflicting goals of interactive rendering and perceptual realism.