Layered user dependent multi-view video streaming

Abstract

Multi-view video consists of multiple video sequences captured simultaneously by multiple closely spaced cameras from different angles. It allows the users to change their viewpoints by playing different video sequence. However, the transmission of multi-view video is much larger than conventional media and increases with increasing number of views, which brings much more increment in the bandwidth requirement. In our previous works, we have proposed a live-encoding scheme call UDMVT [1][2] to reduce the transmission bitrate for multi-view video transmission. In UDMVT, the periodic feedback containing the position and switching speed of user is used to predict a triangle area. Frames in this triangle area are possible to be displayed in next period of time. Only these specific frames are transmitted instead of all the frames to reduce the transmission bitrate. Even though, UDMVT hasn't effectively solve the “out of triangle” problem in which the user may switch to the outside of the triangle during the playback in the triangle area. In this paper, we propose a layered scheme called Layered User dependent Multi-view video Streaming (LUMS) to solve the “out of triangle” problem for UDMVT. In LUMS each triangle is layered into several layers according to the minimum and maximum switching speed. The corresponding layers are transmitted according to the current switching speed each time received from feedback. If the “out of triangle” occurs, the higher layers will be retransmitted. All the frames in high layers are encoded into B-frame predicted from the basic and lower layers, so the retransmission bitrate is low. Performance evaluation proves that LUMS reduce transmission bitrate by 44.9% compared to UDMVT.