Three-Dimensional High Efficiency Video Coding Research Articles

Fast Depth Map Coding Algorithm for 3D-HEVC Based on Gradient Boosting Machine

Three-Dimensional High-Efficiency Video Coding (3D-HEVC) has been extensively researched due to its efficient compression and deep image representation, but encoding complexity continues to pose a difficulty. This is mainly attributed to redundancy in the coding unit (CU) recursive partitioning process and rate–distortion (RD) cost calculation, resulting in a complex encoding process. Therefore, enhancing encoding efficiency and reducing redundant computations are key objectives for optimizing 3D-HEVC. This paper introduces a fast-encoding method for 3D-HEVC, comprising an adaptive CU partitioning algorithm and a rapid rate–distortion-optimization (RDO) algorithm. Based on the ALV features extracted from each coding unit, a Gradient Boosting Machine (GBM) model is constructed to obtain the corresponding CU thresholds. These thresholds are compared with the ALV to further decide whether to continue dividing the coding unit. The RDO algorithm is used to optimize the RD cost calculation process, selecting the optimal prediction mode as much as possible. The simulation results show that this method saves 52.49% of complexity while ensuring good video quality.

Fast 3D‐HEVC inter coding using data mining and machine learning

The Three-Dimensional High Efficiency Video Coding standard is a video compression standard developed based on the two-dimensional video coding standard HEVC and used to encode multi-view plus depth format video. This paper proposes an algorithm based on eXtreme Gradient Boosting to solve the problem of high inter-frame coding complexity in 3D-HEVC. Firstly, explore the correlation between the division depth of the inter-frame coding unit and the texture features in the map, as well as the correlation between the coding unit division structure between each map and each viewpoint. After that, based on the machine learning method, a fast selection mechanism for dividing the depth range of the inter-frame coding tree unit based on the eXtreme Gradient Boosting algorithm is constructed. Experimental results show that, compared with the reference software HTM-16.0, this method can save an average of 35.06% of the coding time, with negligible degradation in terms of coding performance. In addition, the proposed algorithm has achieved different degrees of improvement in coding performance compared with the related works.

Security of 3D-HEVC transmission based on fusion and watermarking techniques

This paper presents two robust hybrid watermarking techniques for efficiently securing the transmission of the Three-Dimensional High Efficiency Video Coding (3D-HEVC). The first watermarking technique is the homomorphic-transform-based Singular Value Decomposition (SVD) in Discrete Wavelet Transform (DWT) domain. The second technique is the three-level Discrete Stationary Wavelet Transform (DSWT) in Discrete Cosine Transform (DCT) domain. The objective of the two proposed hybrid watermarking techniques is to increase the immunity of the watermarked 3D-HEVC to attacks and achieve adequate perceptual quality. Also, we propose a wavelet-based fusion technique to combine multiple depth watermark frames into one fused depth watermark frame. Then, the resultant fused depth watermark is embedded in the 3D-HEVC color frames using the two proposed watermarking techniques to produce the watermarked 3D-HEVC streams. The proposed techniques reduce the required bit rate for transmitting the color-plus-depth 3D-HEVC data. The performances of the two proposed watermarking techniques are compared with those of the state-of-the-art watermarking techniques. The comparisons depend on both subjective visual results and objective results: the Peak Signal-to-Noise Ratios (PSNRs) of the watermarked frames and the Normalized Correlation (NC) of the extracted watermark frames. Extensive simulation experiments on different standard 3D video sequences have been conducted in the presence of attacks. The obtained results confirm that the proposed watermarking techniques achieve not only very good perceptual quality, appreciated PSNR values, and saving of the transmission bit rate, but also high correlation coefficients, and high robustness in the presence of attacks. Furthermore, the proposed techniques improve both the capacity of the embedded information and the robustness without affecting the perceptual quality of the original 3D-HEVC frames. Indeed, the extraction of the fused and multiple primary and secondary depth watermark frames is possible in the presence of different attacks.

CART-based fast CU size decision and mode decision algorithm for 3D-HEVC

Despite three-dimensional high efficiency video coding (3D-HEVC) has a good performance of 3D video coding and synthesized views, the recursive splitting process of the largest coding unit (LCU) and the best mode deciding process caused huge computational complexity. To reduce this computational burden, this paper presents a classification and regression tree-based (CART) fast coding level decision and mode decision algorithm for 3D depth video. The algorithm contained two parts: CART model training, fast coding level and mode decision. In the part of CART model training, we constructed a CART decision tree, where the complexity of depth map, the optimal depth level of co-located texture and the relativity of neighbor LCU were regarded as feature vectors, and the best coding unit depth level was regarded as class label. In the part of fast coding process, features were extracted to predict the depth level of each LCU of depth map; furthermore, some decision of coding mode could be skipped early. Experimental results show that proposed algorithm can save the times of coding process by 24.6% on average while maintaining almost the same rate distortion performance as the 3D-HEVC reference software.

Efficient multi-level security for robust 3D color-plus-depth HEVC

This paper presents two robust hybrid watermarking techniques for securing the Three-Dimensional High Efficiency Video Coding (3D-HEVC). The first watermarking technique is the homomorphic-transform-based Singular Value Decomposition (SVD) in Discrete Wavelet Transform (DWT) domain. The second watermarking technique is the three-level Discrete Stationary Wavelet Transform (DSWT) in Discrete Cosine Transform (DCT) domain. The objective of the two proposed hybrid watermarking techniques is to increase the immunity of the watermarked 3D-HEVC streams to attacks. Also, we propose a wavelet-based fusion technique to combine two depth watermark frames into one fused depth watermark frame. Then, the resultant fused depth watermark is encrypted using chaotic Baker map to increase the level of security. After that, the resultant chaotic encrypted fused depth watermark is embedded in the 3D-HEVC color frames using the proposed hybrid watermarking techniques to produce the watermarked 3D-HEVC streams. In addition to achieving multi-level security in the transmitted 3D-HEVC streams, the proposed hybrid techniques reduce the required bit rate for transmitting the color-plus-depth 3D-HEVC data over limited-bandwidth networks. The performance of the proposed hybrid techniques is compared with those of the state-of-the-art techniques. Extensive simulation results on standard 3D video sequences have been conducted in the presence of attacks. The obtained results confirm that the proposed hybrid fusion-encryption-watermarking techniques achieve not only a good perceptual quality with high Peak Signal-to-Noise Ratio (PSNR) values and less bit rate, but also high correlation coefficient values between the original and extracted watermarks in the presence of attacks. Furthermore, the proposed hybrid techniques improve the capacity of information embedding and the robustness without affecting the perceptual quality of the original 3D-HEVC frames. Indeed, the extraction of the encrypted, fused, primary, and secondary depth watermark frames is possible in the presence of attacks.

Block-level fast coding scheme for depth maps in three-dimensional high efficiency video coding

A block-level fast coding scheme for depth maps coding in 3D-high efficiency video coding (3D-HEVC) is presented. The proposed scheme uses the information of the correlated texture coding unit (CU) to accelerate the encoding depth map CU. The experimental analysis demonstrates that when the SKIP mode encodes the texture CU, the current depth map CU has a high probability of being encoded by SKIP or depth intraskip mode. Therefore, the evaluation of the remaining encoding modes can be skipped when any of these modes obtained a low rate-distortion cost. Experimental results obtained through 3D-HEVC test model 16.0 report that our scheme reduces the encoding time by 26.9% with encoding efficiency losses lower than 0.3%.

An efficient hardware solution for 3D-HEVC intra-prediction

Three-dimensional high-efficiency video coding (3D-HEVC) is the highest profile extension of HEVC for 3D video coding that supports the advanced 3D video format, multi-view video plus depth (MVD). Depth modelling modes (DMMs) are adopted into 3D-HEVC to preserve sharp edges and avoid ringing artefacts in synthesized views. However, computational complexity increases significantly with these modes. A novel hardware solution that reduces the complexity of DMMs for 3D-HEVC is presented in the present paper. This solution contains five modules working in parallel to find all DMMs defined in the new standard for all depth block sizes (from 4 × 4 to 32 × 32). A new technique for obtaining the most compatible wedgelet pattern with its own syntax is developed to minimize the quantity of calculations coming from DMM1; the aforementioned information (wedgelet pattern and syntax) is already calculated and stored in an external memory. This design is synthesized for a Xilinx Virtex 6 FPGA and can process up to 30 frames per second (FPS) while considering the encoding of 6 full HD views in real time. The proposed solution can reach a processing rate up to 30 FPS for 4 QHD views in real time when using 4K video encoding.

Reusable HEVC Design in 3D-HEVC

This paper proposes a reusable design for the merging process used in three-dimensional High Efficiency Video Coding (3D-HEVC), which can significantly reduce the implementation complexity by eliminating duplicated module redundancies. The majority of inter-prediction coding tools used in 3D-HEVC are utilized through a merge mode, whose extended merging process is based on built-in integration to completely wrap around the HEVC merging process. Consequently, the implementation complexity is unavoidably very high. To facilitate easy market implementation, the design of a legacy codec should be reused in an extended codec if possible. The proposed 3D-HEVC merging process is divided into the base merging process of reusing HEVC modules and reprocessing process of refining the existing processes that have been newly introduced or modified for 3D-HEVC. To create a reusable design, the causal and mutual dependencies between the newly added modules for 3D-HEVC and the reused HEVC modules are eliminated, and the ineffective methods are simplified. In an application of the proposed reusable design, the duplicated reimplementation of HEVC modules, which account for of the 3D-HEVC merging process, can be eliminated while maintaining the same coding efficiency. The proposed method has been adopted as a normative coding tool in the 3D-HEVC international standard.

Fast 3D-HEVC encoder algorithm for multiview video plus depth coding

The three-dimensional high efficiency video coding (3D-HEVC) is an extension of the HEVC standard for the compression of the multiview video plus depth (MVD) format. In 3D-HEVC, additional coding tools have been added to HEVC for improving the coding efficiency of the dependent video views and depth maps. Those tools achieve the highest possible coding efficiency, but also bring extremely high computational complexity which limits 3D-HEVC from real-time applications. In this paper, we propose a fast 3D-HEVC encoder algorithm based on the texture video and depth map correlation to reduce MVD coding computational complexity. Since the video texture and depth map represent the same scene at the same time instant (they have the same motion characteristics), it is not efficient to use all the prediction modes in 3D-HEVC mode decision procedure. The basic idea of the algorithm is to utilize the texture video and depth map property of coding information to predict the current CU prediction mode and early skip unnecessary variable-size mode decision. Experimental results demonstrate that the proposed fast 3D-HEVC encoder algorithm can save 75% computational complexity on average, while maintaining almost the same rate-distortion (RD) performance as the original 3D-HEVC encoder.

Efficient wedgelet pattern decision for depth modeling modes in three-dimensional high-efficiency video coding

The three-dimensional (3-D) video extension of high-efficiency video coding is an emerging coding standard for multiple-view-plus-depth that allows view synthesis for multiple displays with depth information. In order to avoid mixing between the foreground and background, the depth discontinuities defined at the object boundary should be retained. To solve this issue, a depth intramode, i.e., the depth-modeling mode (DMM), is introduced in 3-D-high-efficiency video coding as an edge predictor. The test model HTM 8.1 includes DMM1 and DMM3. However, the mode-decision strategy of DMM increases the complexity drastically. Therefore, we propose a fast DMM1 decision algorithm that estimates sharp edges by a subregional search method. The optimal wedgelet pattern of DMM1 is then searched only in the most probable region. Additionally, another fast method is raised to skip DMM3 when mismatch occurs between the depth prediction unit (PU) and its colocated texture PU. Simulation results show that the proposed algorithm has slightly better performance in terms of complexity reduction compared with the wedgelet-pattern-reducing algorithm from the literature while better maintaining the coding performance. In addition, the proposed algorithm has a performance similar to that of an existing DMM-skipping algorithm. Moreover, it could be integrated with that category of algorithms for additional time savings.

Hole-filling map-based coding unit size decision for dependent views in three-dimensional high-efficiency video coding

The three-dimensional high-efficiency video coding (3-D-HEVC) is an emerging compression standard for multiview video plus depth data. In addition to the quad-tree coding structure inherited from HEVC, some tools are integrated, which significantly improve the coding efficiency but also result in remarkably high computational complexity. We propose a fast coding unit (CU) size decision algorithm for both depth and texture components in dependent views, where hole-filling maps created through view synthesis are utilized. First, after coding the base view, warp it onto each dependent view via depth image based rendering, during which hole-filling maps are generated. Then for depth in dependent views, CU splitting can be early terminated considering the disoccluded information from hole-filling maps; for texture in dependent views, combining the disoccluded information and the interview correlations, the CU partitioning process can also be accelerated. Experimental results show that the proposed algorithm can achieve on average 54.3% time reduction, with a negligible Bjontegaard delta bitrate increase of 0.15% on synthesized views, and a 0.05% increase on all the coded plus synthesized views compared with the original encoding scheme in a 3-D-HEVC test model.

Adaptive merge list construction for 3D‐HEVC fast encoder

Adaptive merge list construction for the fast encoder of three-dimensional high efficiency video coding (3D-HEVC) scheme is proposed in this Letter. The encoder complexity of 3D-HEVC has been unavoidably very high in comparison with HEVC, because it should handle the multiple view images and their corresponding depth images. Proposed method is designed to reduce the encoder-side computational complexity of the 3D-HEVC merging process through a fast decision with early termination arranged by two conditions. Simulation results show that the proposed method can reduce overall encoding time by 9.7% without coding losses in comparison with the recent 3D-HEVC HTM-14.0 fast encoder that consisted of every encoder optimisation algorithm informatively adopted in both HEVC and 3D-HEVC standardisation activities.

DMMFast: a complexity reduction scheme for three-dimensional high-efficiency video coding intraframe depth map coding

We present a complexity reduction scheme for the depth map intraprediction of three-dimensional high-efficiency video coding (3-D-HEVC). The 3-D-HEVC introduces a new set of specific tools for depth map coding, inserting additional complexity to intraprediction, which results in new challenges in terms of complexity reduction. Therefore, we present the DMMFast (depth modeling modes fast prediction), a scheme composed of two new algorithms: the simplified edge detector (SED) and the gradient-based mode one filter (GMOF). The SED anticipates the blocks that are likely to be better predicted by the traditional intramodes, avoiding the evaluation of DMMs. The GMOF applies a gradient-based filter in the borders of the block and predicts the best positions to evaluate the DMM 1. Software evaluations showed that DMMFast is capable of achieving a time saving of 11.9% on depth map intraprediction, considering the random access mode, without affecting the quality of the synthesized views. Considering the all intraconfigurations, the proposed scheme is capable of achieving, on average, a time saving of 35% considering the whole encoder. Subjective quality assessment was also performed, showing that the proposed technique inserts minimal quality losses in the final encoded video.