Spherical light field rendering in application for analysis by synthesis

Abstract

This paper presents a novel approach for object classification and pose estimation which employs spherical light field rendering to generate virtual views based on the synthesis parameters determined and successively refined in a two-stage analysis by synthesis process. Compared with previous object recognition techniques, the presented approach provides a significant improvement in terms of object classification quality and computational efficiency. Our Graphics Processing Unit based light field renderer exploits per-pixel depth information available with modern Time-Of-Flight sensors like the Photonic Mixer Device camera for high-quality image synthesis in real-time. The renderer uses combined per-pixel RGB and depth values to minimise ghosting artefacts to a non-noticeable amount and employs a spherical parameterisation to ensure full 6 d.f. for virtual view synthesis. Synthetic views are used in our two-stage analysis by synthesis technique which implements a pre-classification and pre-pose-estimation to reduce parameter search space, thus, providing improved object classification quality at less computation time.