Extension Of High Efficiency Video Coding Research Articles

Compression Performance Analysis of Experimental Holographic Data Coding Systems.

It is challenging to find a proper way to compress computer-generated holography (CGH) data owing to their huge data requirements and characteristics. This study proposes CGH data coding systems with high-efficiency video coding (HEVC), three-dimensional extensions of HEVC (3D-HEVC), and video-based point cloud compression (V-PCC) codecs. In the proposed system, we implemented a procedure for codec usage and format conversion and evaluated the objective and subjective results to analyze the performance of the three coding systems. We discuss the relative advantages and disadvantages of the three coding systems with respect to their coding efficiency and reconstruction results. Our analysis concluded that 3D-HEVC and V-PCC are potential solutions for compressing red, green, blue, and depth (RGBD)-sourced CGH data.

Scalable extension of HEVC with adaptive de-blocking filter for bandwidth-limited applications

A primary latest video coding standard H.265/SHVC (Scalable High Efficiency Video Coding), an extension of High Efficiency Video Coding (HEVC) is dealt in this paper. To transfer high quality video over any network that usually occupies large bandwidth, suffers bandwidth limitation. It can be controlled by trading-off between the quality and the coding efficiency. The trade-off between these two parameters is considered in this paper to keep the difference at the minimum level. A novel Neural Network based Adaptive De-blocking Filter (NNADF) technique is proposed to maintain good PSNR with minimum bit rate in SHVC encoder. The adaptive de-blocking filter (ADF) performs block matching for nonlinear predictions to all frames in the video. Moreover, the ADF performs frame de-noising hence good quality of the video is achieved and trains a neural network (NN) model. The NN model restrains their outputs to achieve bit rate reduction, thereby coding efficiency is achieved. The simulation results show that the proposed NNADF technique delivers an average increment of 0.62 dB in BD-PSNR and an average decrement of 24 % in BD-BR for combined quality and spatial ratios respectively compared with the previously proposed algorithm.

Adaptive coding unit size convolutional neural network for fast 3D-HEVC depth map intracoding

The advanced three-dimensional extension of high-efficiency video coding (3D-HEVC) is the latest coding standard for 3D video. The coding of the depth map for 3D-HEVC is very time-consuming. With the development of deep learning, it has become feasible to employ convolutional neural networks (CNNs) to predict the coding unit (CU) division of the depth map. However, there are three types of CU sizes: 64, 32, and 16, which makes it difficult to unify the model. The features of the depth map are very different from the texture map. In view of the aforementioned problems, we propose an adaptive CU size CNNs for fast 3D-HEVC depth map intracoding. We first employ spatial pyramid pooling to fully extract the features of the three types of CUs. Then, we apply the nonlocal self-attention mechanism to make it suitable for depth maps. Compared with the 3D-HEVC reference algorithm, the proposed network reduces the coding time by an average of 35.7%, while the quality degradation of the synthesized virtual view is negligible.

Fast intra- and inter-coding algorithms for the spatially scalable extension of H.265/HEVC

High Efficiency Video Coding (HEVC) is the latest international video coding standard. As an extension of HEVC, scalable HEVC (SHVC) achieves high compression efficiency at the cost of dramatically increased computational complexity. In this paper, a fast Coding Unit (CU) size decision algorithm is suggested for spatial SHVC intra-coding to early terminate the CU size decision process by jointly utilising the inter-layer and spatiotemporal correlations. Furthermore, in order to realise a fast Prediction Unit (PU) intra-mode decision, the mode dependency between the Base Layer (BL) and the Enhancement Layer (EL) is effectively used to remove unlikely coding modes. A fast PU inter-mode decision method is also developed for spatial SHVC inter-coding, in which the degree of motion activity and the PU mode dependency are exploited. Simulation results show that the proposed fast intra-coding algorithm produces a saving in encoding time of up to 72% and the proposed fast inter-coding scheme reduces the encoding time by up to 50% compared with the original SHVC encoder, and the loss in Rate-Distortion (RD) performance is negligible.

Performance Analysis of HEVC Scalable Extension for Depth Maps

High Efficiency Video Coding (HEVC) is actually the latest video standard that enables significant compression performance improvement in comparison with its predecessors. Scalable High Efficiency Video Coding (SHVC) and 3D High-Efficiency Video Coding (3D-HEVC) are extensions of HEVC hooked on recently emerged applications and issues. 3D-HEVC targets the monolayer compression of the specific multiview plus depth 3D video representation. It consists of multiple views and their associated depth maps that are required for synthesizing intermediate views in advanced free viewpoint applications. SHVC combines the compression efficiency and visual quality of HEVC, with the possibility of encoding different versions of a same video in a single bitstream. It is conceived to cope with the heterogeneity of networks and devices used in the transmission chain of texture videos. With the ongoing progess in equipments and technologies, the scalable multilayer coding is also solicited, not to say required, for depth maps. In comparison with 3D-HEVC, this paper shows how SHVC works for quality scalable depth maps compression over different base and enhancement layers quantization scenarios. Besides, this paper analyzes the SHVC efficiency for intermediate views that are synthesized from encoded depth maps. Thus we aim to provide support for upcoming works on depth map quality scalable encoding.

Shearlet Transform based Light Field Compression Under Low Bitrates.

Light field (LF) acquisition devices capture spatial and angular information of a scene. In contrast with traditional cameras, the additional angular information enables novel postprocessing applications, such as 3D scene reconstruction, the ability to refocus at different depth planes, and synthetic aperture. In this paper, we present a novel compression scheme for LF data captured using multiple traditional cameras. The input LF views were divided into two groups: key views and decimated views. The key views were compressed using the multi-view extension of high-efficiency video coding (MV-HEVC) scheme, and decimated views were predicted using the shearlet-transform-based prediction (STBP) scheme. Additionally, the residual information of predicted views was also encoded and sent along with the coded stream of key views. The proposed scheme was evaluated over a benchmark multi-camera based LF datasets, demonstrating that incorporating the residual information into the compression scheme increased the overall peak signal to noise ratio (PSNR) by 2 dB. The proposed compression scheme performed significantly better at low bit rates compared to anchor schemes, which have a better level of compression efficiency in high bit-rate scenarios. The sensitivity of the human vision system towards compression artifacts, specifically at low bit rates, favors the proposed compression scheme over anchor schemes.

SLEPX: An Efficient Lightweight Cipher for Visual Protection of Scalable HEVC Extension

This paper proposes a lightweight cipher scheme aimed at the scalable extension of the High Efficiency Video Coding (HEVC) codec, referred to as the Scalable HEVC (SHVC) standard. This stream cipher, Symmetric Cipher for Lightweight Encryption based on Permutation and EXlusive OR (SLEPX), applies Selective Encryption (SE) over suitable coding syntax elements in the SHVC layers. This is achieved minimal computational complexity and delay. The algorithm also conserves most SHVC functionalities, i.e. preservation of bit-length, decoder format-compliance, and error resilience. For comparative analysis, results were taken and compared with other state-of-art ciphers i.e. Exclusive-OR (XOR) and the Advanced Encryption Standard (AES). The performance of SLEPX is also compared with existing video SE solutions to confirm the efficiency of the adopted scheme. The experimental results demonstrate that SLEPX is as secure as AES in terms of visual protection, while computationally efficient comparable with a basic XOR cipher. Visual quality assessment, security analysis and extensive cryptanalysis (based on numerical values of selected binstrings) also showed the effectiveness of SLEPX's visual protection scheme for SHVC compared to previously-employed cryptographic techniques.

Hybrid stopping model-based fast PU and CU decision for 3D-HEVC texture coding

As an extension of High-Efficiency Video Coding (HEVC) standard, 3D-HEVC needs to encode multiple texture views and depth maps, which further increases the computational complexity. To reduce the complexity of dependent texture view coding, a fast prediction unit (PU) and coding unit (CU) decision method is proposed for 3D-HEVC based on hybrid stopping model. The inter-view correlation is used as a priori information to roughly predict the possible optimal PU and CU sizes. Then, by exploiting the encoded posterior information, the rate distortion cost correlation and the code block flag, the optimal PU and CU are further examined as being optimal or not. Experimental results show that the proposed fast PU and CU decision method achieves 52.7% encoding time saving on average with negligible loss of coding efficiency for 3D-HEVC-dependent texture view coding.

Fast Intra Coding of High Dynamic Range Videos in SHVC

Compared with the conventional standard dynamic range (SDR) content, high dynamic range (HDR) content supplies viewers with more immersive experience by offering a much higher range of luminance. Most of current consumer devices cannot afford to this emerging technology, and content providers decide to create both an HDR version and an SDR version of the same video. In this letter, scalable high efficiency video coding (HEVC) scalable extension of HEVC (SHVC) serves as the coding framework where the base layer (BL) is an 8-b SDR version and the enhancement layer (EL) is a 12-b HDR version. Recently, many fast coding algorithms for SDR videos are proposed, and there is an urgent demand for fast coding algorithms for EL HDR videos. With the coding information of the BL SDR videos, this letter proposes a fast algorithm to reduce the complexity of intra coding for EL HDR videos. First, depth information of neighboring coding tree units (CTUs) in the HDR version and the colocated CTU in the SDR version is used for early coding unit (CU) depth determination. Moreover, four classifiers are trained to predict the CTU depth range. Two classifiers are trained for CTUs in frames with a high average luma, and another two classifiers are used for CTUs in frames with a low average luma. Experimental results show that the proposed algorithm achieves 43% encoding time saving on average, with only a 0.54% Bjøntegaard delta bit rate (BDBR) increase compared to the original SHVC test model.

Tile-based 360-degree video streaming for mobile virtual reality in cyber physical system

Today, the demand for and interest in virtual reality (VR) is increasing, since we can now easily experience VR in many applications. However, the computational ability of mobile VR is limited compared to that of tethered VR. Since VR represents a 360-degree area, providing high quality only for the area viewed by the user saves considerable bandwidth. Therefore, we propose a new tile-based streaming method that transforms 360-degree videos into mobile VR using high efficiency video coding (HEVC) and the scalability extension of HEVC (SHVC). While the SHVC base layer (BL) represents the entire picture, the enhancement layer (EL) can transmit only the desired tiles by applying the proposed method. By transmitting the BL and EL using region of interest (ROI) tiles, the proposed method helps reduce not only the computational complexity on the decoder side but also the network bandwidth.

Coding unit complexity-based predictions of coding unit depth and prediction unit mode for efficient HEVC-to-SHVC transcoding with quality scalability

To support good video quality of experiences in heterogeneous environments, transcoding an existed HEVC (high efficiency video coding) video bitstream to a SHVC (scalability extension of HEVC) bitstream with quality scalability is highly required. A straightforward way is to first fully decode the input HEVC bitstream and then fully re-encode it with the SHVC encoder, which requires a tremendous computational complexity. To solve the problem, in this paper, a coding unit complexity (CUC)-based prediction method for predictions of CU (coding unit) depth and PU (prediction unit) mode for efficient HEVC-to-SHVC transcoding with quality scalability is proposed to significantly reduce the transcoding complexity. The proposed method contains two prediction techniques, including (i) early termination and (ii) adaptive confidence interval, and predicts the CU depth and PU mode relying on the decoded information from the input HEVC bitstream. Experimental results have shown that the proposed method significantly outperforms the traditional HEVC-to-SHVC method by 74.14% on average in reductions of encoding time for SHVC enhancement layer.

Computing Offloading Over mmWave for Mobile VR: Make 360 Video Streaming Alive

Compared with previous video streaming technologies, 360 video streaming is an emerging technology in commercial platforms that promise greater immersive video service. The use of 360 video streaming requires larger bandwidth and rapid responsiveness to users. In addition, mobile virtual-reality (VR) devices locally process video decoding, post-processing, and rendering. However, its performance is insufficient to stream high-resolution videos such as 4k–8k. Therefore, we propose an adaptive computing offloading scheme using millimeter wave (mmWave) communication. This offloading scheme helps mobile device sharing video decoding tasks to a powerful GPU-based PC. It allows the mobile VR device enhancing the capability of playing high-resolution videos. The mmWave 802.11ad wireless technology promises the use of high-bandwidth wireless communication to improve the capacity of multimedia systems. However, the current studies on video transmission over 802.11ad mmWave are specifically focused on supporting the outdoor environment. Furthermore, synchronization is an essential issue for ensuring the quality of service (QoS) of 360 video transmission. Thus, the proposed scheme concentrates on two parts: 1) 360 video streaming in an indoor environment, and offloading mechanism using mmWave 802.11ad 60-GHz wireless communication to offload video coding tasks and 2) adaptive video synchronization mechanisms, and high-efficiency video coding (HEVC) using scalable HEVC extension for 360 videos with equirectangular projection. The experiments prove that the proposed scheme provides high performance 360 video streaming in QoS-sensitive video streaming applications.

Segmental Prediction for Video Coding

The emerging standards for screen content coding (SCC) and 3D video coding known as High Efficiency Video Coding (HEVC)-SCC and 3D-HEVC, respectively, have been developed as extensions of HEVC. Compared with camera-captured video, screen content video and depth maps usually contain significantly fewer colors or sample intensities and much sharper edges in a frame. In this paper, a new coding method named segmental prediction is proposed to code screen content and depth maps more efficiently. First, samples in a prediction block are classified into several segments according to the sample intensities. Second, a particular predictor is calculated for each segment. Then an offset for a segment is derived at the encoder side and signaled to the decoder side. Finally, a single sample value is assigned to each segment in the reconstructed block. The experimental results show that the proposed approach can achieve a 2.2% average Bjontegaard Distortion-rate (BD-rate) saving under the HEVC-SCC common test conditions (CTCs) for sequences of the category YUV, text and graphics with motion, 1080p, with the all intra configuration. It can also achieve a 0.6% average BD-rate saving for synthesized views under the 3D-HEVC CTCs with the random access configuration.

Interactive Screen Video Streaming-Based Pervasive Mobile Workstyle

In this paper, we develop an interactive screen video streaming-based system to enable the ubiquitous mobile workstyle, which is referred to as personal computer to pervasive computing (PC2PC). The desktop screens of virtualized systems are compressed in the PC2PC servers and delivered to remote end users for stream decoding, rendering, and interactions. We have implemented a system from the scratch, where the emerging screen content coding extension of high-efficiency video coding is implemented to compress and stream the desktop screens of the virtualized system in real time. Three core asset channels, system, display, and inputs, are defined to enable systematic end-to-end communication. Compared with Red Hat SPICE virtual desktop infrastructure scheme, the proposed PC2PC could save network bandwidth consumption by a factor of 2, 7, and 4, respectively, in terms of typical video streaming, web browsing, and stationary office applications at the same visual quality. Meanwhile, we have also measured the delays of the system and presented preliminary results on the user experience aspect. A simple network estimation is applied to optimize the quality bandwidth adaptation for both single user and multiuser scenarios to consider the network dynamics.

An efficient edge detection algorithm for fast intra-coding for 3D video extension of HEVC

The high efficiency video coding (HEVC)-based 3D video coding (3D-HEVC) has been recently standardized by the Joint Collaborative Team on 3D video coding as a 3D extension of HEVC. The 3D-HEVC has been developed to improve the coding efficiency of Multi-view Video plus Depth. Intra-prediction is considered as an important technique in image and video compression, which aims to exploit spatial correlation within one picture. The use of the variable coding unit size and multiple intra-prediction modes makes the intra-coding of 3D-HEVC very efficient. However, the computational complexity is increased significantly. This paper presents a low complexity mode decision algorithm for 3D-HEVC intra-prediction based on local edge information.

Unified Blind Quality Assessment of Compressed Natural, Graphic, and Screen Content Images.

Digital images in the real world are created by a variety of means and have diverse properties. A photographical natural scene image (NSI) may exhibit substantially different characteristics from a computer graphic image (CGI) or a screen content image (SCI). This casts major challenges to objective image quality assessment, for which existing approaches lack effective mechanisms to capture such content type variations, and thus are difficult to generalize from one type to another. To tackle this problem, we first construct a cross-content-type (CCT) database, which contains 1,320 distorted NSIs, CGIs, and SCIs, compressed using the high efficiency video coding (HEVC) intra coding method and the screen content compression (SCC) extension of HEVC. We then carry out a subjective experiment on the database in a well-controlled laboratory environment. Moreover, we propose a unified content-type adaptive (UCA) blind image quality assessment model that is applicable across content types. A key step in UCA is to incorporate the variations of human perceptual characteristics in viewing different content types through a multi-scale weighting framework. This leads to superior performance on the constructed CCT database. UCA is training-free, implying strong generalizability. To verify this, we test UCA on other databases containing JPEG, MPEG-2, H.264, and HEVC compressed images/videos, and observe that it consistently achieves competitive performance.

Low-Complexity Enhancement Layer Compression for Scalable Lossless Video Coding Based on HEVC

Lossless compression is desired, especially for professional applications like medical imaging. However, scalability may be necessary to transmit a fast preview of pictures when the channel capacity is limited. Furthermore, there may be the need for random access to single frames of a losslessly reconstructed video. We present and evaluate a lossy-to-lossless scalable video coding system in which the scalability is achieved by using a lossy base layer (BL) in conjunction with lossless compression of the reconstruction error in the enhancement layer (EL). High Efficiency Video Coding (HEVC) is employed for encoding the BL. For compression of the EL, we propose a low-complexity scheme called sample-based weighted prediction for EL coding (SELC). Furthermore, EL compression using JPEG-LS and the scalable extension of HEVC (SHVC) are evaluated. The performance of scalable coding using each of these three methods is compared with lossless single-layer (SL) coding using the intra-main RExt configuration. Experimental results show that the scalable coding with SELC achieves average bitrate savings of 7.3% compared with the lossless SL coding, while average bitrate reductions of only 6.3% and 3.9% are obtained using JPEG-LS and SHVC, respectively. Concerning runtime, the additional encoder runtimes of SELC and JPEG-LS compared with the BL coding are negligible. In contrast to that, the runtime of the SHVC EL encoder is much higher and similar to the runtime of the BL encoder.

Real-time selective video encryption based on the chaos system in scalable HEVC extension

In this paper we propose a real-time selective video encryption solution in the scalable extension of High Efficiency Video Coding (HEVC) standard, referred to as SHVC. The proposed scheme encrypts a set of sensitive SHVC parameters with a minimum delay and complexity overheads. The encryption process is performed at the CABAC binstring level and fulfills both constant bitrate and format compliant video encryption requirements. In addition, it preserves all SHVC functionalities, including bitstream extraction for mid-network adaptation and error resilience.We compare the performance of three selective SHVC encryption schemes: the first scheme encrypts only the lowest SHVC layer, the second encrypts all layers and the last scheme encrypts only the highest layer. The performance of the proposed schemes is assessed over different video encryption criteria, at different scalability configurations and various High Definition (HD) video sequences. Experimental results showed that encrypt only the lowest layer or all layers enables a high security level, while encrypting only the highest layer leads to a perceptual encryption solution, by slightly decreasing the highest layer quality. Moreover, the processing complexity of the proposed solution is assessed in the context of a real-time SHVC decoder. The complexity overhead remains low and does not exceed 6% of the real-time decoding of SHVC video sequences.

Overview of Color Gamut Scalability

Displays’ new rendering capabilities combined with the ever-growing number of video applications have fueled the emergence of new video formats addressing wider color gamut and larger frame size. Thus, the need in scalable compression technology to provide backward compatibility with legacy devices and capitalize on the superior compression performance of High Efficiency Video Coding (HEVC) has increased significantly. This paper gives an overview of the work carried out in the Joint Collaborative Team on Video Coding of ITU-T Study Group 16 (VCEG) and ISO/IEC JTC1/SC29/WG11 motion picture experts group (MPEG) to define scalable extensions of HEVC (SHVC) targeting these market requirements. The color gamut scalability (CGS) tool of SHVC is specially designed to support efficient scalable coding with multiple layers in different color spaces. The genesis of the SHVC-CGS tool is presented and the performance of the various proposals is compared. Finally, the design of the recently adopted SHVC-CGS inter-layer prediction is detailed. The experimental results validate its efficiency in coding video with extended color gamut and high dynamic range.

Progressive Dictionary Learning With Hierarchical Predictive Structure for Low Bit-Rate Scalable Video Coding.

Dictionary learning has emerged as a promising alternative to the conventional hybrid coding framework. However, the rigid structure of sequential training and prediction degrades its performance in scalable video coding. This paper proposes a progressive dictionary learning framework with hierarchical predictive structure for scalable video coding, especially in low bitrate region. For pyramidal layers, sparse representation based on spatio-temporal dictionary is adopted to improve the coding efficiency of enhancement layers with a guarantee of reconstruction performance. The overcomplete dictionary is trained to adaptively capture local structures along motion trajectories as well as exploit the correlations between the neighboring layers of resolutions. Furthermore, progressive dictionary learning is developed to enable the scalability in temporal domain and restrict the error propagation in a closed-loop predictor. Under the hierarchical predictive structure, online learning is leveraged to guarantee the training and prediction performance with an improved convergence rate. To accommodate with the state-of-the-art scalable extension of H.264/AVC and latest High Efficiency Video Coding (HEVC), standardized codec cores are utilized to encode the base and enhancement layers. Experimental results show that the proposed method outperforms the latest scalable extension of HEVC and HEVC simulcast over extensive test sequences with various resolutions.