Chapter 9 - Light field processing for media applications

Abstract

Light fields describe the amount and direction of light rays flowing through every point in space. They allow reproducing different perspectives of a captured scene or a digitized object. This is particularly beneficial for photo-realistic virtual reality scenarios. Unfortunately, practical applications are still rare, as capturing and processing of information divided up in rich angular and spatial details contained in a light field is difficult. To overcome this situation, this chapter will describe algorithms and software solutions for both sparse and dense light fields. At first, a complete pipeline from capturing and processing to rendering of light fields is described. Light field capture is performed using a custom-built camera array. A node-based processing solution in a professional post-production environment permits to steer the light field reconstruction process for best achievable quality. The result can then be integrated into a VR environment with the help of a game engine. In the second part of this chapter, possibilities of neural networks to increase achievable rendering quality or reduce processing and capturing efforts are explored in detail.