Towards Approximation of Human's Perceptive Space on Photos, Videos and 3D Scenes

Abstract

We initiate the study of processing photo pictures and video frames as well as synthetic scenes in three-dimensional (3D) space, which would allow to get images more similar to what human's vision sees, compared to images created by existing technologies. It is known that human's vision is nonlinear and creates sensually perceived image of 3D space not equal to well-known linear perspective. Because of that we offer inspired by the biology of human's vision, a three-stage scheme for processing images and video. The first stage is a deep image analysis by means of neural networks and other machine learning algorithms to reconstruct a 3D model of the observed scene. The second stage is mapping 3D scene in Euclidean space to 3D scene in human's perceptive space. The third stage is the synthesis of near-optimal 2D projections in perceptual space. We show that such processing requires simulation of visual image processing by the human's brain. The concept can be applied to improve the quality of pictures and movies, to create more believable virtual reality or augmented reality, to improve using computer vision in human-robot interaction, to create in the future eye prostheses for blinds etc.