Image Virtual Viewpoint Generation Method under Hole Pixel Information Update

Abstract

A virtual viewpoint generation method is proposed to address the problem of low fidelity in the generation of virtual viewpoints for images with overlapping pixel points. Virtual viewpoint generation factors such as overlaps, holes, cracks, and artifacts are analyzed and preprocessed. When the background of the hole is a simple texture, pheromone information around the hole is used as the support, a pixel at the edge of the hole is detected, and the hole is predicted at the same time, so that the hole area is filled in blocks. When the hole background has a relatively complex texture, the depth information of the hole pixels is updated with the inverse 3D transformation method, and the updated area pheromone is projected onto the auxiliary plane and compared with the known plane pixel auxiliary parameters. The hole filling is performed according to the symmetry of the pixel position of the auxiliary reference viewpoint plane to obtain the virtual viewpoint after optimization. The proposed method was validated using image quality metrics and objective evaluation metrics such as PSNR. The experimental results show that the proposed method could generate virtual viewpoints with high fidelity, excellent quality, and a short image-processing time, which effectively enhanced the virtual viewpoint generation performance.