Enabling Five-Degree-of-Freedom From Low-Dimensional Light Fields With Extended Light Ray Acquisition Field-of-View

Abstract

Conventional captured-image-based virtual reality (VR) systems hardly support free movement of the viewpoint, greatly reducing the user’s sense of reality. A light field (LF), which generates a novel view at a free viewpoint through a combination of light rays, is a suitable approach for increasing the degree of freedom (DoF) in captured-image-based VR. Previous LF-based VR systems have increased LF dimensions or used various acquisition layouts to obtain as many light rays as possible and thereby expand the renderable view range. However, these attempts are costly and time-consuming. This paper enables 5-DoF outcomes from a low-dimensional LF by increasing the field-of-view during light ray acquisition. The method starts with the simple idea that if a three-dimensional (3D) LF is constructed with 360° images that can capture all directions, the renderable range can be widened in a cost-efficient manner. However, using light rays with a large incident angle in a 3D LF can cause visual artifacts. To cope with this problem, this paper initially defines and analyzes the visual artifacts. While these errors were simply ignored in the past, they become particularly noticeable when using light rays with large incident angles. Thus, a careful analysis of the errors and their impacts is required. Additionally, this paper proposes two schemes to deal with visual artifacts based on the error analysis: a horizontal LF and vertical depth (HLF-VD) view generation scheme that complements the structural limitation of 3D LFs and can address all types of errors, and a hybrid 3D LF connection scheme that effectively reduces connection errors by controlling the connection position of neighboring 3D LFs. The proposed system is implemented and verified in various real-world scenarios, and the experimental results show that the user’s movement is naturally reflected at a level similar to that of high-dimensional LF, with minimal visual artifacts.