Multi-view Video Codec Research Articles

Auto-stereoskopik 3BTV için Öznel Deneyim Kalitesi Modellerinin

Measuring Quality of Experience (QoE) of stereoscopic 3D video is a hot research topic. Subjective models are considered as the most reliable and facilitate development of objective models. However, to collect user opinion scores takes long time. Therefore, new subjective assessment models should be proposed providing not only time-efficiency but also good accuracy and reliability. In this study, two novel subjective QoE models are proposed as alternative to the conventional Double Stimulus Continuous Quality Scale method. Also, a fair comparison method is proposed to evaluate performances of the three subjective methods using the same stimuli prepared with the most recent multi-view video codec on an auto-stereoscopic 3DTV. Correlations are calculated using two objective QoE measures using depth maps and structural similarities. The results demonstrate that the performances of the proposed models are comparable to each other and both models are superior to the conventional method.

Multi-view video codec using compressive sensing for wireless video sensor networks

In monitoring applications, different views are needed to be captured by multi-view video sensor nodes for understanding the scene clearly. These multi-view sequences have large volume of redundant data which affects the storage, transmission, bandwidth and lifetime of wireless video sensor nodes. A low complex coding technique is required for addressing these issues and for processing multi-view sensor data. Hence, in this paper, a framework on CS-based multi-view video codec using frame approximation technique (CMVC-FAT) is proposed. Quantisation with entropy coding based on frame skipping is adopted for achieving efficient video compression. For better prediction of skipped frame at receiver, a frame approximation technique (FAT) algorithm is proposed. Simulation results reveal that CMVC-FAT framework outperforms the existing method with achievement of 86.5% reduction in time and bits. Also, it shows 83.75% reduction in transmission energy compared with raw frame.

Multi-view Video CODEC Using Compressive Sensing for Wireless Video Sensor Networks

In monitoring applications, different views are needed to be captured by multi-view video sensor nodes for understanding the scene clearly. These multi-view sequences have large volume of redundant data which affects the storage, transmission, bandwidth and lifetime of wireless video sensor nodes. A low complex coding technique is required for addressing these issues and for processing multi-view sensor data. Hence, in this paper, a framework on CS-based multi-view video codec using frame approximation technique (CMVC-FAT) is proposed. Quantisation with entropy coding based on frame skipping is adopted for achieving efficient video compression. For better prediction of skipped frame at receiver, a frame approximation technique (FAT) algorithm is proposed. Simulation results reveal that CMVC-FAT framework outperforms the existing method with achievement of 86.5% reduction in time and bits. Also, it shows 83.75% reduction in transmission energy compared with raw frame.

Mixed-resolution HEVC based multiview video codec for low bitrate transmission

There has been increasing demand for multiview video transmission over band limited channel over past years and various techniques have been proposed to fulfil this need. In this paper, a High Efficiency Video Codec (HEVC) based spatial resolution scaling type of mixed resolution coding model, MRHEVC-MVC, for frame interleaved multiview videos is presented. However, enabling the HEVC to encode video with different frame resolutions is a challenge due to the coding tree partitioning used by the codec. This has been overcome by super-imposing the low resolution replica of each full resolution frame on their respective decoded picture buffer and setting the remaining space of the frame buffer to zero. The codec’s reference frames structure is designed to efficiently encode frame interleaved multiview videos using a HEVC based mixed resolution codec. The proposed MRHEVC-MVC codec has been tested against the standard multiview extension of high efficiency video codec (MV-HEVC) for “Balloon”, “Newspaper1”, “Undo_Dancer”, “Kendo” and ““Poznan_Street” standard multiview video sequences. Results show that the proposed codec gives significantly higher coding performance to that of the MV-HEVC codec at low bitrate both subjectively and objectively.

Toward enhancing the distributed video coder under a multiview video codec framework

The advance of video coding technology enables multiview video (MVV) or three-dimensional television (3-D TV) display for users with or without glasses. For mobile devices or wireless applications, a distributed video coder (DVC) can be utilized to shift the encoder complexity to decoder under the MVV coding framework, denoted as multiview distributed video coding (MDVC). We proposed to exploit both inter- and intraview video correlations to enhance side information (SI) and improve the MDVC performance: (1) based on the multiview motion estimation (MVME) framework, a categorized block matching prediction with fidelity weights (COMPETE) was proposed to yield a high quality SI frame for better DVC reconstructed images. (2) The block transform coefficient properties, i.e., DCs and ACs, were exploited to design the priority rate control for the turbo code, such that the DVC decoding can be carried out with fewest parity bits. In comparison, the proposed COMPETE method demonstrated lower time complexity, while presenting better reconstructed video quality. Simulations show that the proposed COMPETE can reduce the time complexity of MVME to 1.29 to 2.56 times smaller, as compared to previous hybrid MVME methods, while the image peak signal to noise ratios (PSNRs) of a decoded video can be improved 0.2 to 3.5 dB, as compared to H.264/AVC intracoding.

Scalable Multi-view Video Coding based on HEVC

In this paper, we propose an integrated spatial and view scalable video codec based on high efficiency video coding (HEVC). The proposed video codec is developed based on similarity and uniqueness between the scalable extension and 3D multi-view extension of HEVC. To improve compression efficiency using the proposed scalable multi-view video codec, inter-layer and interview predictions are jointly employed by using high-level syntaxes that are defined to identify view and layer information. For the inter-view and inter-layer predictions, a decoded picture buffer (DPB) management algorithm is also proposed. The inter-view and inter-layer motion predictions are integrated into a consolidated prediction by harmonizing with the temporal motion prediction of HEVC. We found that the proposed scalable multi-view codec achieves bitrate reduction of 36.1%, 31.6% and 15.8% on the top of ×2, ×1.5 parallel scalable codec and parallel multi-view codec, respectively.

A Novel High Efficiency Fractal Multiview Video Codec

Multiview video which is one of the main types of three-dimensional (3D) video signals, captured by a set of video cameras from various viewpoints, has attracted much interest recently. Data compression for multiview video has become a major issue. In this paper, a novel high efficiency fractal multiview video codec is proposed. Firstly, intraframe algorithm based on the H.264/AVC intraprediction modes and combining fractal and motion compensation (CFMC) algorithm in which range blocks are predicted by domain blocks in the previously decoded frame using translational motion with gray value transformation is proposed for compressing the anchor viewpoint video. Then temporal-spatial prediction structure and fast disparity estimation algorithm exploiting parallax distribution constraints are designed to compress the multiview video data. The proposed fractal multiview video codec can exploit temporal and spatial correlations adequately. Experimental results show that it can obtain about 0.36 dB increase in the decoding quality and 36.21% decrease in encoding bitrate compared with JMVC8.5, and the encoding time is saved by 95.71%. The rate-distortion comparisons with other multiview video coding methods also demonstrate the superiority of the proposed scheme.

Multiview Video Plus Depth Coding With Depth-Based Prediction Mode

In this paper, we propose a novel multiview video plus depth (MVD) codec that uses a depth-based prediction mode (DBPM) both for texture videos and the depth maps. We show that the proposed codec can provide up to 9.2%, 9.9%, and 6.7% bitrate savings over H.264/MVC (MVC) for coding MVD data, depth maps, and multiview videos, respectively. In addition, we present subjective test results that compare the perceptual video quality of stereo videos coded with the DBPM-enabled codec and MVC. We also show that a typical Lagrangian rate-distortion optimization is effective to successfully choose between the prediction modes available to the proposed encoder. We provide a complexity analysis for the proposed codec as well and show that the encoder complexity is comparable to MVC, while the decoder complexity is about twice of MVC. Finally, we analyze the effects of different encoder control parameters on the virtual view synthesis quality. Results show that video QP has the most influence on the synthesis quality, and there is a tradeoff between the depth map QP and resolution. Lower depth map resolutions yield more bitrate savings, whereas for similar bitrates higher resolution and higher QP depth maps achieve better view synthesis performance.

Perceptual Awareness Rate Control for Multi-View Video Encoder in Stereoscopic Display

Multi-view video coding technique has been widely applied to 3D stereoscopic display. This paper proposes a rate control algorithm for multi-view video codec which is a tradeoff method between the data compression ratio and picture quality of 3D video. This study utilizes the characteristics of visual perception for 3D video viewers to determine the interesting regions in all view videos. Interesting regions with different levels, which depend on the existence of macroblocks with high motion, objects with individual shape, or obvious texture, are determined in the proposed algorithm by considering the corresponding Just Noticeable Distortion function, luminance histogram bin variation and edge strength, respectively. Then the proposed algorithm assigns adequate quantization parameters to control the bit-rate for different levels of interesting regions such that the video quality of the interesting regions is preserved and the bit-rate in the non-interesting regions is reallocated to 3D video storage and transmission for stereoscopic display. The experiment results show that the proposed method successfully achieves high quality preservation and low bit-rate for 3D effects.

Methods and Tools for Wavelet-Based Scalable Multiview Video Coding

A wavelet-based multiview video coding scheme is presented in this paper. It uses a 4-D wavelet transform, which is composed of a 1-D temporal wavelet transform, namely motion compensated temporal filtering, a 1-D view-directional wavelet transform, namely disparity compensated view filtering and a 2-D spatial wavelet transform. Since the presented framework can make use of the inherent scalability properties of the wavelet transforms involved, it allows full scalability of the coded bitstream in the temporal, view, spatial, and quality dimensions. Coding performance close to the H.264/advanced video coding based standard multiview video codec is shown. Enhancements of the view transform, in order to better account for brightness and color variations across views are introduced. Additionally, the use of a signal adaptive anisotropic wavelet packet (WP) transform as a generalization of WP transforms for the spatial decomposition is proposed. Both enhancements lead to a decrease of bit rate of up to 11% compared with the baseline version of the codec.