Depth Assisted Adaptive Workload Balancing for Parallel View Synthesis

Abstract

Depth image-based rendering has been adopted by MPEG as the recommended view synthesis technique for free viewpoint TV applications. In this paper, a workload balancing algorithm is proposed for parallel view synthesis on multicore platforms. First, view synthesis workload is defined as the function of the number of hole-pixels in the warped images. Then, a novel depth assisted prediction method is proposed to predict the number of hole-pixels in the current frame by exploiting the depth differences between the neighboring frames, which reflects the movement of objects in video content. Feeding the predicted workload to the proposed cost function, each input frame is partitioned adaptively to balance the synthesis workload among the cores. The proposed workload prediction method outperforms the existing approaches both in terms of frame average prediction error and standard deviation in prediction error. Applying the proposed workload balancing method, the parallel view synthesis system provides higher acceleration ratio and better synchronization performance among the cores compared with other parallel processing systems without sacrificing the subjective and objective quality. It is also robust to different platforms, which shows high potential in being applied to mobile oriented applications.