Scalable Multi-view Video Coding Research Articles

Hierarchical modulation for client-driven selective streaming of multi-view video over AWGN channels

We consider a client-driven selective streaming system for multi-view video, which is supported by scalable multi-view video coding (SMVC). The system is used to reduce bit-rates for efficient transmission of multi-view video. The transmit source is partitioned into three layers: a base and two enhancement layers. The base layer contains all views encoded by MVC at a low-quality, while each enhancement layer contains high-quality part of the selected left and right views, respectively. The base layer is more important than the enhancement layers to create 3D perception, albeit of low-quality, and should be more protected than the enhancement layers. In this paper, two-level and three-level hierarchical 64QAM are used to provide unequal error protection (UEP) for the client-driven selective streaming system. First we find the suitable hierarchical values, which is the ratio of the distances in hierarchical 64QAM, for the client-driven selective streaming system. Furthermore, we analyze numerically how the hierarchical 64QAM impacts peak signal-to-noise ratio (PSNR) performance of fast and slow-motion sequences. Simulation results show that the three-level hierarchical 64QAM outperforms both conventional 64QAM with Gray mapping and the two-level hierarchical 64QAM in terms of achieving the target average PSNR performance at a low SNR environment.

Scalable Multi-view Video Coding based on HEVC

In this paper, we propose an integrated spatial and view scalable video codec based on high efficiency video coding (HEVC). The proposed video codec is developed based on similarity and uniqueness between the scalable extension and 3D multi-view extension of HEVC. To improve compression efficiency using the proposed scalable multi-view video codec, inter-layer and interview predictions are jointly employed by using high-level syntaxes that are defined to identify view and layer information. For the inter-view and inter-layer predictions, a decoded picture buffer (DPB) management algorithm is also proposed. The inter-view and inter-layer motion predictions are integrated into a consolidated prediction by harmonizing with the temporal motion prediction of HEVC. We found that the proposed scalable multi-view codec achieves bitrate reduction of 36.1%, 31.6% and 15.8% on the top of ×2, ×1.5 parallel scalable codec and parallel multi-view codec, respectively.

GPB 메카니즘을 활용한 스케일러블 다시점 비디오 부호화를 위한 효율적인 DPB 설계 기법

본 논문에서는 효율적인 스케일러블 다시점 비디오 부호화를 수행하기 위해 필요한 참조화면 재순서화, 마킹 프로세스, 참조픽처 리스트 구성을 포함하는 DPB (Decoded Picture Buffer) 작동을 위한 새로운 설계 방법을 제안한다. 다양한 실험 결과를 통해서 제안된 방법에 의해 스케일러블 다시점 비디오 부호화에 있어서 개선된 압축효율을 얻을 수 있고 BD-Rate 및 BD-PSNR 측면에서 향상된 비디오 화질을 얻을 수 있음을 확인한다. In this paper, we propose a novel design scheme for the operation of Decoded Picture Buffer (DPB) including reference picture re-ordering, marking process, and reference picture list construction to perform an efficient scalable multi-view video coding. Extensive simulations show that the proposed method can provide improved compression efficiency and improved video quality measured in terms of BD-Rate and BD-PSNR for the scalable multi-view video coding.

Hoda Roodaki

With the growing demand for 3D video, efforts are underway to incorporate it in the next generation of broadcast and streaming applications and standards. Scalability is one possible solution to reduce the amount of data in multi-view/3D video in heterogeneous environments. But using Scalable Multi-view Video Coding (SMVC) for multi-view/3D video still has many unresolved challenges. In this thesis, we propose an adaptive framework to use SMVC in various 3D video applications effectively. For this issue, first, the proper scalable modality should be selected according to the application at hand, its related features and requirements. To the best of our knowledge, no work has systematically defined new and proper scalable modalities specifically for multi-view 3D video, so far. Hence, at the first step of the proposed framework, we will suggest a methodology to extract the proper scalable modalities for multi-view/3D video. In addition, while SMVC can help support heterogeneous receivers, the question becomes: how to scale the 3D video content in a given type of scalability and a specified application in order to achieve the highest performance and satisfy the receivers? constraint as much as possible? In other words, the proper mechanism to assign SMVC data to various layers should be clearly determined. This issue is considered as the second step of our proposed framework. This method uses the inter-layer and intralayer disparity concepts. Note that specific features of any given scalable modality should be used to define these concepts in that specific scalable modality. Simulation results indicate that the proposed method achieves relatively better compression rate for each layer, with much less overhead. At the next step of our proposed framework, we propose an analytical view-level rate model for multiview video coding. Our rate model takes into account both previous theoretical results as well as new results specifically obtained for multi-view video and confirmed by comprehensive practical experiments. Simulation results show that our model can predict the rate of each view with relatively high precision and a low estimation error of 12% on average for tested sequences. In addition, the evaluation of the overall visual quality of scalable multi-view video requires a new objective perceptual quality measure specifically designed for scalable multi-view/3D video. Although several subjective and objective quality assessment methods have been proposed for multi-view/3D sequences, no comparable attempt has been made for quality assessment of scalable multi-view/3D video so far. Hence, in this framework, we propose a new methodology to build suitable objective quality assessment metrics for different scalable modalities in multi-view/3D video. Our proposed methodology considers the importance of each layer and its content as a quality of experience factor in the overall quality. Furthermore, in addition to the quality of each layer, the concept of inter-layer and intra-layer disparity is considered as an effective feature to evaluate overall perceived quality more accurately. Our simulation results indicate that the correlation coefficient between our extracted objective quality evaluation metric and subjective quality assessment is 0.8 on average for tested video sequences. At the last step of our proposed framework, we present a novel method for rate-distortion optimization in scalable multi-view video that tries to minimize the perceptual distortion of decoded video under the conditions that the sum of bits generated from different views is constrained within a given bit budget. Since the constraint-based optimization problem is usually computational intensive, our proposed approach considers the concept of intra-layer and inter-layer disparity to reduce this computational complexity. Experimental results show that the proposed approach uses on average 24% and 42% less bitrate than the H.264/AVC rate-distortion optimization for base and base plus enhancement layers, respectively. Although the thesis is in Farsi (Persian), the following English papers capture most of its essence: H. Roodaki, M.R. Hashemi, and S. Shirmohammadi, ?A New Methodology to Derive Objective Quality Assessment Metrics for Scalable Multi-view 3D Video Coding?, ACM Transactions on Multimedia Computing, Communications, and Applications, Vol. 8, No. 3S, Article 44, September 2012, 25 pages. DOI: 10.1145/2348816.2348823 H. Roodaki, M.R. Hashemi, and S. Shirmohammadi, ?Rate-Distortion Optimization for Scalable Multi-View Video Coding?, Proc. IEEE International Conference on Multimedia and Expo, Chengdu, China, July 14-18 2014, 6 pages. DOI: 10.1109/ICME.2014.6890275 H. Roodaki, Z. Iravani, M.R. Hashemi, S. Shirmohammadi, and M. Gabbouj, ?A New Rate Distortion Model for Multi-View/3D Video Coding?, Proc. IEEE International Workshop on Hot Topics in 3D, in Proc. IEEE International Conference on Multimedia and Expo, July 15-19 2013, San Jose, USA, 6 pages. DOI: 10.1109/ICMEW.2013.6618338 H. Roodaki, M.R. Hashemi, and S. Shirmohammadi, ?New Scalable Modalities in Multi-view 3D Video?, Proc. ACM Workshop on Mobile Video, Oslo, Norway, February 27 2013, pp. 25-30. DOI: 10.1145/2457413.2457420

Multi-View Video Coding Based on Video Correction

In order to enables the video to be displayed on a multitude of different terminals, Multi-view video coding (MVC) demands high compression rates as well as view scalability. A new the inter-view prediction structure with flexibility, MVC compatibility and view scalability is proposed. The proposed scheme first correct views. It then uses the position of the cameras and the relation of inter-views to divide all views into base view and enhancement views. Finally, the views quire by the viewers use scalable Multi-view Video Coding (SMVC). The main bitstream is the same as a H.264/AVC mono-sequence bitstream for H.264/AVC compatibility. The auxiliary bitstream come from enhancement views coding. We proposed SMVC scheme is tested with two Multi-view sequences to determine its flexibly view scalability, high coding efficiency and random access performance.

스케일러블 다시점 비디오 부호화를 위한 효율적인 움직임 예측구조와 DPB 설계

본 논문에서는 SVC와 MVC의 부호화 구조를 결합하여 구현된 스케일러블 다시점 비디오 부호화를 위한 효율적인 움직임 추정 기법과 DPB 설계 메카니즘에 대해 제안한다. 제안된 움직임 추정 기법에서는 부호화 과정에서 필요한 예측 부호화의 성능 향상을 위해서 서로 다른 시점 (view)의 픽처 정보를 참조픽처의 후보로서 활용한다. 제안된 움직임 예측 구조에 의해서 압축된 비디오 데이터의 크기를 감소시켜 압축 효율을 증대시킬 수 있다. 또한, 스케일러블 다시점 비디오 부호화를 수행할 때 SVC와 MVC의 DPB (Decoded Picture Buffer)를 통합한 통합형 DPB 설계 메카니즘에 대해 제안한다. 다양한 실험을 통해서 제안된 예측 구조를 적용함으로써 스케일러블 다시점 비디오 부호화에서의 압축 효율의 향상을 얻어낼 수 있음을 확인하였다. In this paper, we propose an efficient motion prediction architecture and DPB design mechanism for scalable multi-view video coding which is implemented by integrating SVC and MVC coding algorithms. In the proposed motion prediction architecture, we employ pictures associated with other views as a candidate for reference picture for improved motion prediction performance. By the proposed prediction architecture, we could enormously reduce the size of compressed video data. When performing scalable multi-view video coding, an integrated DPB design mechanism is also proposed. It is shown by various simulations that the proposed motion prediction architecture for scalable multi-view video coding can result in reduced data size in the compressed bitstream.

An efficient region of interest support in scalable multi-view video coding

In this paper, a scalable multi-view video coding (SMVC) scheme is proposed to efficiently realize region of interest (ROI) support in SMVC. In addition, in order to further improve the coding efficiency, we develop a novel bit allocation model which can implement reasonable bit allocation among views. Extensive experiments are conducted on the typical multi-view video (MVV) sequences, and the efficiency of the presented codec is demonstrated. Meanwhile, the addressed SMVC architecture can easily provide temporal, spatial, SNR, view, and content scalabilities.

Methods and Tools for Wavelet-Based Scalable Multiview Video Coding

A wavelet-based multiview video coding scheme is presented in this paper. It uses a 4-D wavelet transform, which is composed of a 1-D temporal wavelet transform, namely motion compensated temporal filtering, a 1-D view-directional wavelet transform, namely disparity compensated view filtering and a 2-D spatial wavelet transform. Since the presented framework can make use of the inherent scalability properties of the wavelet transforms involved, it allows full scalability of the coded bitstream in the temporal, view, spatial, and quality dimensions. Coding performance close to the H.264/advanced video coding based standard multiview video codec is shown. Enhancements of the view transform, in order to better account for brightness and color variations across views are introduced. Additionally, the use of a signal adaptive anisotropic wavelet packet (WP) transform as a generalization of WP transforms for the spatial decomposition is proposed. Both enhancements lead to a decrease of bit rate of up to 11% compared with the baseline version of the codec.