Inter-view Prediction Structure Research Articles

Efficient Inter-View Prediction Structure for Multi-View High Efficiency Video Coding

Efficient Inter-View Prediction Structure for Multi-View High Efficiency Video Coding

MVP2P: Layer-dependency-aware live MVC video streaming over peer-to-peer networks

Multiview video supports observing a scene from different viewpoints. The Joint Video Team (JVT) developed H.264/MVC to enhance the compression efficiency for multiview video, however, MVC encoded multiview video (MVC video) still requires high bitrates for transmission. This paper investigates live MVC video streaming over Peer-to-Peer (P2P) networks. The goal is to minimize the server bandwidth costs whist ensuring high streaming quality to peers. MVC employs intra-view and inter-view prediction structures, which leads to a complicated layer dependency relationship. As the peers’ outbound bandwidth is shared while supplying all the MVC video layers, the bandwidth allocation to one MVC layer affects the available outbound bandwidth of the other layers. To optimize the utilization of the peers’ outbound bandwidth for providing video layers, a maximum flow based model is proposed which considers the MVC video layer dependency and the layer supplying relationship between peers. Based on the model, a layer dependency aware live MVC video streaming method over a BitTorrent-like P2P network is proposed, named MVP2P. The key components of MVP2P include a chunk scheduling strategy and a peer selection strategy for receiving peers, and a bandwidth scheduling algorithm for supplying peers. To evaluate the efficiency of the proposed solution, MVP2P is compared with existing methods considering the constraints of peer bandwidth, peer numbers, view switching rates, and peer churns. The test results show that MVP2P significantly outperforms the existing methods.

Enhanced view random access ability for multiview video coding

Apart from the efficient compression, reducing the complexity of the view random access is one of the most important requirements that should be considered in multiview video coding. In order to obtain an efficient compression, both temporal and inter-view correlations are exploited in the multiview video coding schemes, introducing higher complexity in the temporal and view random access. We propose an inter-view prediction structure that aims to lower the cost of randomly accessing any picture at any position and instant, with respect to the multiview reference model JMVM and other recent relevant works. The proposed scheme is mainly based on the use of two base views (I-views) in the structure with selected positions instead of a single reference view as in the standard structures. This will, therefore, provide a direct inter-view prediction for all the remaining views and will ensure a low-delay view random access ability while maintaining a very competitive bit-rate performance with a similar video quality measured in peak signal-to-noise ratio. In addition to a new evaluation method of the random access ability, the obtained results show a significant improvement in the view random accessibility with respect to other reported works.

Subjective evaluation of Super Multi-View compressed contents on high-end light-field 3D displays

Super Multi-View (SMV) video content is composed of tens or hundreds of views that provide a light-field representation of a scene. This representation allows a glass-free visualization and eliminates many causes of discomfort existing in current available 3D video technologies. Efficient video compression of SMV content is a key factor for enabling future 3D video services. This paper first compares several coding configurations for SMV content and several inter-view prediction structures are also tested and compared. The experiments mainly suggest that large differences in coding efficiency can be observed from one configuration to another. Several ratios for the number of coded and synthesized views are compared, both objectively and subjectively. It is reported that view synthesis significantly affects the coding scheme. The amount of views to skip highly depends on the sequence and on the quality of the associated depth maps. Reported ranges of bitrates required to obtain a good quality for the tested SMV content are realistic and coherent with future 4K/8K needs. The reliability of the PSNR metric for SMV content is also studied. Objective and subjective results show that PSNR is able to reflect increase or decrease in subjective quality even in the presence of synthesized views. However, depending on the ratio of coded and synthesized views, the order of magnitude of the effective quality variation is biased by PSNR. Results indicate that PSNR is less tolerant to view synthesis artifacts than human viewers. Finally, preliminary observations are initiated. First, the light-field conversion step does not seem to alter the objective results for compression. Secondly, the motion parallax does not seem to be impacted by specific compression artifacts. The perception of the motion parallax is only altered by variations of the typical compression artifacts along the viewing angle, in cases where the subjective image quality is already low. To the best of our knowledge, this paper is the first to carry out subjective experiments and to report results of SMV compression for light-field 3D displays. It provides first results showing that improvement of compression efficiency is required, as well as depth estimation and view synthesis algorithms improvement, but that the use of SMV appears realistic according to next generation compression technology requirements.

Optimizing Multiview Video Plus Depth Prediction Structures for Interactive Multiview Video Streaming

Several multiview video coding standards have been developed to efficiently compress images from different camera views capturing the same scene by exploiting the spatial, the temporal and the interview correlations. However, the compressed texture and depth data have typically many interview coding dependencies, which may not suit interactive multiview video streaming (IMVS) systems, where the user requests only one view at a time. In this context, this paper proposes an algorithm for the effective selection of the interview prediction structures (PSs) and associated texture and depth quantization parameters (QPs) for IMVS under relevant constraints. These PSs and QPs are selected such that the visual distortion is minimized, given some storage and point-to-point transmission rate constraints, and a user interaction behavior model. Simulation results show that the novel algorithm has near-optimal compression efficiency with low computational complexity, so that it offers an effective encoding solution for IMVS applications.

Multi-View Video Coding Based on Video Correction

In order to enables the video to be displayed on a multitude of different terminals, Multi-view video coding (MVC) demands high compression rates as well as view scalability. A new the inter-view prediction structure with flexibility, MVC compatibility and view scalability is proposed. The proposed scheme first correct views. It then uses the position of the cameras and the relation of inter-views to divide all views into base view and enhancement views. Finally, the views quire by the viewers use scalable Multi-view Video Coding (SMVC). The main bitstream is the same as a H.264/AVC mono-sequence bitstream for H.264/AVC compatibility. The auxiliary bitstream come from enhancement views coding. We proposed SMVC scheme is tested with two Multi-view sequences to determine its flexibly view scalability, high coding efficiency and random access performance.

Multiview video coding with an improved prediction structure for faster random access

As any existing video codec, multiview video coding requires a highly efficient compression as well as a faster random access to any given view. Such efficiency for the coding approach can be achieved by exploiting both the temporal and the inter-view correlation. The temporal and view random access are improved by the increase of intracoded pictures (I pictures), which requires more bit rate. In this research study, we propose and evaluate an improved inter-view prediction structure mainly based on reducing the complexity of view random access at any instance T n and improving the inter-view correlation for P-view. These improvements can be guaranteed by the appropriate choice of the position of I-view, the type of the different views, and the number of each frame type to use. Compared to the prediction scheme, which is based on the hierarchical B pictures structure and used for joint multiview video model, experimental results have shown that the proposed prediction structure provides better bit-rate performance of up to 6.81% with a similar compression quality measured in peak signal-to-noise ratio and significantly improves view random access by up to 25%.

Optimized MVC Prediction Structures for Interactive Multiview Video Streaming

The Multiview Video Coding (MVC) standard efficiently compresses multiview video by considering spatial, temporal and interview correlations. This letter studies the impact of the MVC interview prediction structure on both the transmission and the overall coding rates for an interactive multiview video streaming system, considering both unicast and multicast scenarios, with the user interactive behavior represented by some view-popularity model. We propose a method to identify the optimal prediction structure minimizing the visual distortion, given some storage and link capacities constraints. Simulation results confirm that the optimal prediction structure results from a non-trivial tradeoff between the system constraints, the transmission model and the views' popularity.

Optimizing inter-view prediction structures for multi-view video coding using simulated annealing

New video applications, such as 3D video and free viewpoint video, require efficient compression of multi-view video. In addition to temporal redundancy, exploiting the inter-view redundancy is crucial to improve the performance of multi-view video coding. In this paper, we present a novel method to construct the optimal inter-view prediction structure for multi-view video coding using simulated annealing. In the proposed model, the design of the prediction structure is converted to the arrangement of coding order. Then, a simulated annealing algorithm is employed to minimize the total cost for obtaining the best coding order. This method is applicable to arbitrary irregular camera arrangements. As experiment results reveal, the annealing process converges to satisfactory results rapidly and the generated optimal prediction structure outperforms the reference prediction structure of the joint multi-view video model (JMVM) by 0.1–0.8 dB PSNR gains.

Global Disparity Compensation for Multi-view Video Coding

While single view video coding uses the temporal prediction scheme, multi-view video coding (MVC) applies both temporal and inter-view prediction schemes. Thus, the key problem of MVC is how to reduce the inter-view redundancy efficiently, because various existing video coding schemes have already provided solutions to reduce the temporal correlation. In this paper, we propose a global disparity compensation scheme which increases the inter-view correlation and a new inter-view prediction structure based on the global disparity compensation. By experiment, we demonstrate that the proposed global disparity compensation scheme is less sensitive to change of the search range. In addition, the new Inter-view prediction structure achieved about <TEX>$0.1{\sim}0.3dB$</TEX> quality improvement compared to the reference software.

Optimising inter-view prediction structure for multiview video coding with minimum spanning tree

A new systematic method to construct a near-optimal inter-view prediction structure for multiview video coding is proposed based on the directed minimum spanning tree of inter-view prediction cost graph. The method is applicable to arbitrary irregular and dynamic changing camera arrangements. Results show that the generated structure achieves better compression efficiency than the reference structure of the joint multiview video model.

Efficient Prediction Structures for Multiview Video Coding

<para xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> An experimental analysis of multiview video coding (MVC) for various temporal and inter-view prediction structures is presented. The compression method is based on the multiple reference picture technique in the H.264/AVC video coding standard. The idea is to exploit the statistical dependencies from both temporal and inter-view reference pictures for motion-compensated prediction. The effectiveness of this approach is demonstrated by an experimental analysis of temporal versus inter-view prediction in terms of the Lagrange cost function. The results show that prediction with temporal reference pictures is highly efficient, but for 20% of a picture's blocks on average prediction with reference pictures from adjacent views is more efficient. Hierarchical B pictures are used as basic structure for temporal prediction. Their advantages are combined with inter-view prediction for different temporal hierarchy levels, starting from simulcast coding with no inter-view prediction up to full level inter-view prediction. When using inter-view prediction at key picture temporal levels, average gains of 1.4-dB peak signal-to-noise ratio (PSNR) are reported, while additionally using inter-view prediction at nonkey picture temporal levels, average gains of 1.6-dB PSNR are reported. For some cases, gains of more than 3 dB, corresponding to bit-rate savings of up to 50%, are obtained. </para>

View Scalable Multiview Video Coding Using 3-D Warping With Depth Map

Multiview video coding demands high compression rates as well as view scalability, which enables the video to be displayed on a multitude of different terminals. In order to achieve view scalability, it is necessary to limit the inter-view prediction structure. In this paper, we propose a new multiview video coding scheme that can improve the compression efficiency under such a limited inter-view prediction structure. All views are divided into two groups in the proposed scheme: base view and enhancement views. The proposed scheme first estimates a view-dependent geometry of the base view. It then uses a video encoder to encode the video of base view. The view-dependent geometry is also encoded by the video encoder. The scheme then generates prediction images of enhancement views from the decoded video and the view-dependent geometry by using image-based rendering techniques, and it makes residual signals for each enhancement view. Finally, it encodes residual signals by the conventional video encoder as if they were regular video signals. We implement one encoder that employs this scheme by using a depth map as the view-dependent geometry and 3-D warping as the view generation method. In order to increase the coding efficiency, we adopt the following three modifications: (1) object-based interpolation on 3-D warping; (2) depth estimation with consideration of rate-distortion costs; and (3) quarter-pel accuracy depth representation. Experiments show that the proposed scheme offers about 30% higher compression efficiency than the conventional scheme, even though one depth map video is added to the original multiview video.