Real-time foveal light fields for spatial imaging devices

Abstract

Recent advances in display technology are introducing new devices that provide multiple viewers with a true 3D experience. These spatial imaging devices display either volumetric models or four dimensional image-based light-field models. The light field or plenoptic function gives radiance along the lines passing through a scene. Due to its higher dimensionality it is expensive to model, capture, generate, store, retrieve and display. Geometric models are not suitable for spatial imaging, as rendering many views of complex geometries requires more effort than extracting and displaying carefully arranged images of a scene. We thus focus in this paper on image-based representations of the light field for spatial imaging devices. We present a multiresolution representation for the light field that is independent from the viewers' positions. Our representation allows lazy level-of-detail (LOD) extraction for adaptive frame-rate control. It also supports continuous LOD rendering to avoid popping artifacts. The system relies on a two-level light-field cache that always keeps a minimum working set of images in graphics memory. This set is large enough to render the model from any vantage point. Our approach uses multi-threading and delayed image retrieval from disk to main memory, and from main memory to graphics memory. We also propose a predictive caching algorithm that only retrieves and maintains radiance data likely to be viewed by the user in coming frames. Finally, we implement foveal vision for added efficiency in light-field rendering. Our system is targeted at autostereoscopic devices like lenticular, holographic and parallax-barrier displays. (8 pages)