Realistic physical camera motion for light field visualization

Abstract

Light field displays depict the real world by emitting light rays corresponding to the 3D space of the scene or object that is to be represented. Therefore, such displays are suitable for utilization in a multitude of use cases, including the cinema. Cinematography is the art encompassing motion picture photography. It describes a great number of methods for working with physical cameras. For example, a camera can be mounted on a car in a typical chase scene, attached to a spring, suspended or mounted on a colliding object and many more. Although these camera rigs may produce exceptional visuals, achieving similar results by using light field cameras will be extremely challenging. In order to capture light fields, two sets of possible solutions can occur. These include capturing light fields by means of an actual light field camera or via an array of cameras, where the latter may be planar or curved. Using one way or the other is rather demanding, as each method has its own limitations and difficulties. When using a light field camera, the captured light field should map to that of the light field display, on which it shall be visualized. Whereas capturing a scene with a camera array, can unfortunately lead to self-capture. Moreover, the portability of the camera arrays is evidently problematic, due to the sheer size and weight. In addition to these challenges, light field rendering on its own is far from being trivial. While rendering to conventional 2D displays from camera arrays may use image-based rendering, where many views of the scene can be set up from pre-captured images, light fields are represented by 5D plenoptic functions that are not easy to capture with conventional camera arrays. Moreover, image-based rendering techniques often fail to produce convincing results for light field displays. For some use cases, they can be reduced into 4D for horizontal- parallax-only light fields, since our eyes are horizontally separated and horizontal motion is more frequent than vertical. In practice, the creation of a light field scene from a set of images requires injecting each 2D image into a 4D light field representation. In this paper, we visualize different simulations for realistic physical camera motions on a real light field display. In order to overcome the aforementioned problems, virtual cameras were used to simulate a set of different physical camera motions used in cinematography for light field displays. In applications, physics simulation libraries include algorithms for the dynamics of soft as well as rigid bodies. Moreover, collision detection is also accounted for. Many tools have been devised to simulate physics, among which is the Bullet Physics library. In our work, we used the Bullet Physics library in order to generate realistic physical camera motions as well as physical environment simulations for light field displays. The limitations and challenges imposed by light field displays when simulating physical camera motions are discussed, along with the results and the produced outputs.