High Efficiency Video Coding Video Research Articles

Adaptive HEVC video steganograhpy based on PU partition modes

High EfficiencyVideoCoding (HEVC) −based steganography has gained attention as a prominent research focus. Especially, block structure based HEVC video steganography has received increasing attention due to commendable performance. However, current block structure- based steganography algorithms confront with challenges such as reduced coding efficiency and limited capacity. To avoid these problems, an adaptive video steganography algorithm based on Prediction Unit (PU) partition mode in I-frames is proposed. This is done through the analysis of the block division process and the visual distortion resulting from the modification of the PU partition mode in HEVC. The PU block structure is utilized as steganographic covers, and the Rate Distortion Optimization (RDO) technique is introduced to establish an adaptive distortion function for Syndrome-trellis code (STC). Further comparison is performed between the proposed method and the state-of-the-art steganography algorithms, confirming its advantages in embedding capacity, compression efficiency, visual quality, and resistance to video steganalysis.

Discrete Hyperchaotic S-Box Generation for Selective Video Frames Encryption

Cloud computing has become a significant technology with rapid popularization offering many services with highly effective computation and large-scale storage solutions. As multimedia files are growing explosively, securely storing them for authorized access is a major task. Concentrating on video file storage in a public cloud environment, the security and privacy of those outsourced videos are the major concerns. A selective frame encryption approach is proposed to encrypt the semantic elements of the selected frames of High Efficiency Video Coding (HEVC) compressed videos by storing them in the cloud. The semantic elements of HEVC videos are extracted and encrypted by a modified Advanced Encryption Standard (AES) operation in CFB mode with a Substitution-Box (S-Box). A dynamic, nonlinear 2D discrete hyperchaotic mapping-based S-box generation method is proposed to enhance security by providing good confusion, diffusion, and scrambling. The performance analysis of generated S-box and experimental results show that the proposed selective encryption highly resists attacks and encrypts with reduced time complexity.

An Anti-Steganalysis HEVC Video Steganography With High Performance Based on CNN and PU Partition Modes

The steganography research of videos leads to excellent communication methods for transmitting secret message, and high efficiency video coding(HEVC) video is one popular steganographic carrier. This article proposes a prediction unit(PU) based wide residual-net steganography(PWRN) for HEVC videos. The visual quality distortion of modifying PUs is theoretically analyzed, which illustrates that modifying PUs only has a little negative effect on visual quality. Therefore, the data hiding method in this article allows to modify all types of PUs except for <inline-formula><tex-math notation="LaTeX">$2N\times 2N$</tex-math></inline-formula> to each other according to the secret data. In this way, high embedding efficiency is achieved, and the PU distributions in stego-videos can be kept similar to those of cover-videos, which is essential for resisting steganalysis. Meanwhile, a super-resolution convolutional neural network(CNN) with wide residual-net filter(WRNF) is proposed to replace the in-loop filter in HEVC for reconstructing I-pictures, which results in more precisely predicted P-pictures, and it further leads to less bitrate cost and better visual quality of stego-videos. The experimental results show that the proposed PWRN successfully resists the latest PU-targeted steganalysis algorithms, and compared with the state-of-the-art work, PWRN has achieved the lowest bitrate cost and the highest visual quality under the same capacity.

HEVC Video Quality Enhancement Using Deep Learning with Super Interpolation and Laplacian Filter

Enhancing video quality is a vital area of research that has piqued the interest of academia and industry alike. The High-Efficiency Video Coding (HEVC) compressed videos suffer from significant quality loss, especially at low bit rates. As a result, the decoder must improve the image quality of HEVC videos. Deep Learning with Super Interpolation based Super Resolution (DLSI-SR) and Laplacian Filter (LF) method is proposed to eliminate artifacts. A Multi-Layered Deep Convolutional Neural Network (ML-DCNN) with LF has been introduced to reduce the artifacts quickly in compressed video and increase the subjective video quality. The quality of Low-Resolution (LR) frames can be effectively enhanced using Super Interpolation and Laplacian filter. The Edge orientation analysis is performed on various external training video frames during the training stage. The LR video frame is up-sampled and interpolated using the Super Interpolation algorithm during the upscaling phase. The interpolated frame is then submitted to edge detection using a Canny Edge detection algorithm for frame smoothing. Using local Laplacian filter, the reconstructed high quality video frame is sharpened with an edge preservation approach. The simulation results of the proposed DLSI-SR method shows higher performance in terms of PSNR and SSIM with less processing time in comparison with the existing approaches. The experimental results show that the proposed approach achieves a maximum PSNR of 42.5 dB (enhanced by 2.56 dB) and produced an SSIM improvement of 0.948 (enhanced by +0.022) as compared to the existing approaches.

HEVC video information hiding scheme based on adaptive double-layer embedding strategy

High Efficiency Video Coding (HEVC) is well-known as an internationally popular video coding standard, and HEVC-based steganography has received increasing attention. In this paper, a new adaptive HEVC video information hiding method based on Prediction Unit (PU) partition mode and double-layer embedding strategy is proposed. Double-layer embedding is a method to complete the first-layer embedding using the mapping rules of PU partition mode, and to perform the second-layer embedding after the first-layer embedding. The cost assignment function designed in this paper can accurately evaluate the second-layer data embedding distortion. The frame position, motion properties and block size of PU are taken into consideration for the second-layer data embedding, and the syndrome-trellis codes (STCs) are used to minimize the embedding distortion. Experimental results show that the proposed adaptive double-layer embedding algorithm has better embedding efficiency and less embedding distortion in most cases.

Improved HEVC video compression algorithm using low-complexity frame rate up conversion

The high-efficiency video coding (HEVC) standard can reduce the bit rate of compression video. However, the compression efficiency of HEVC is still challenging current internet bandwidth. This problem can be solved by reducing the number of video frames encoded by HEVC and reconstructing the reduced frames that rely on frame rate up conversion (FRUC). Unfortunately, most existing FRUC methods need to re-estimate the motion information, a step that will lead to a huge increase of video decoder complexity. An improved HEVC video compression algorithm using low-complexity FRUC is proposed based on reusing the motion vector (MV) predicted by HEVC encoding procedure to directly reconstruct the skipped frames. First, an adaptive frame-skipping scheme is used to maximize reducing the number of frames that will encode by HEVC. Second, decoder extracts the MV and residual from the received bit stream. Then an MV reliability checking and rectifying method is proposed according to adaptive residual energy analyzing. Finally, a motion-compensation method and efficient occluded area processing algorithm are proposed to reconstruct skipped frames. These methods all directly utilize the rectified MVs. Experimental results show that due to the reduced frames and decoder that does not need to re-estimate MV, the proposed method can not only significantly decrease the time of all codec procedure when comparing HEVC and other FRUC-based video compression methods but also achieve 9.7022% to 66.0030% reduction in Bjøntegaard delta bit rate at low bit rates when compared to HEVC standard. In addition, the compression of the proposed method is also better than other state-of-the-art methods.

A HEVC Video Steganalysis Against DCT/DST-Based Steganography

The development of video steganography has put forward a higher demand for video steganalysis. This paper presents a novel steganalysis against discrete cosine/sine transform (DCT/DST)-based steganography for high efficiency video coding (HEVC) videos. The new steganalysis employs special frames extraction (SFE) and accordion unfolding (AU) transformation to target the latest DCT/DST domain HEVC video steganography algorithms by merging temporal and spatial correlation. In this article, the distortion process of DCT/DST-based HEVC steganography is firstly analyzed. Then, based on the analysis, two kinds of distortion, the intra-frame distortion and the inter-frame distortion, are mainly caused by DCT/DST-based steganography. Finally, to effectively detect these distortions, an innovative method of HEVC steganalysis is proposed, which gives a combination feature of SFE and a temporal to spatial transformation, AU. The experiment results show that the proposed steganalysis performs better than other methods.

Performance Comparison of Emerging EVC and VVC Video Coding Standards with HEVC and AV1

<i>The recent dramatic increase in video content consumption requires efficient video coding standards, which is specifically true for ultrahigh-definition (UltraHD) resolutions, such as 4K and 8K (i.e., 3840&#x00D7;2160 or 7680&#x00D7;4320 resolutions in terms of luma samples, respectively). The well-known high-efficiency video coding (HEVC) [H.265/Moving Pictures Expert Group (MPEG)-H] standard was approved in 2013. Although HEVC provides approximately 50&#x0025; coding gain compared to its predecessor advanced video coding (AVC) (H.264/MPEG-4), its adoption is still relatively slow. In addition, larger bitrate savings than those provided by HEVC are currently in demand. At the same time, work on the versatile video coding (VVC) and essential video coding (EVC) standards started in 2018. After intensive development efforts that continued for two and a half years, these two video coding standards have been recently finalized. VVC (H.266/MPEG-I) was developed jointly by the MPEG and the International Telecommunication Union-Telecommunication (ITU-T) Video Coding Experts Group (VCEG). On the other hand, EVC (MPEG-5) is an MPEG-only effort. In this paper, we compare the performance of EVC and VVC, in terms of both coding gains and computational complexity, to their predecessor&#x2014;the HEVC video coding standard. In addition, given the growing popularity of the AV1 video codec, which was recently developed by the Alliance for Open Media (AOM), we also include AV1 as an alternative baseline and provide corresponding comparison results. According to the experimental results, which have been carried out in a constant bitrate (CBR) mode, EVC provides about 30&#x0025; bitrate savings compared to HEVC for encoding 4K/2160p entertainment content (such as VoD) in terms of Bj&#x00F8;ntegaard-delta bitrate (BD-BR) peak-signal-to-noise ratio (PSNR)_YUV, while introducing an encoding computational complexity increase of approximately five times. VVC provides larger bitrate savings of about 40&#x0025; at a price of a significant encoding computational complexity increase of more than nine times. When the performance of HEVC CBR encoding (i.e., with the rate control disabled) was compared to that of AV1 VBR encoding (i.e., with the rate control enabled), it was found that AV1 provides bitrate savings of about 20&#x0025; compared to HEVC for encoding 4K/2160p video sequences, as a tradeoff of an encoding computational complexity increase by a factor of approximately four. The authors find both EVC and VVC to be very promising successors to HEVC in terms of coding gains and computational complexity, but the jury is still out on the speed of their adoption</i>.

A High-Performance CNN-Applied HEVC Steganography Based on Diamond-Coded PU Partition Modes

High efficiency video coding (HEVC) is the latest high-performance video coding standard, and HEVC video steganography has become a new way to hide data for covert communication. This paper proposes a novel multilevel steganography algorithm based on diamond-encoded prediction unit (PU) partition modes. The PU modes of smaller <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$8\times 8$</tex-math></inline-formula> and <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$16\times 16$</tex-math></inline-formula> CUs are selected as carriers for information hiding. The diamond-coding rules are adopted to enhance the expressive ability of limited PU types, allowing them to carry more information under limited modification. Based on the encoded PU partition modes, three different embedding levels with different capacities are proposed. This paper’s most outstanding contribution is the introduction of convolutional neural networks (CNNs) for the first time to improve visual quality and reduce steganographic video bitrate increases. Experimental results show that the embedding capacity of the proposed algorithm is significantly higher than the state-of-the-art work at the same bitrate, whether in high- or low-resolution HEVC videos. At the same time, the visual quality of the steganographic videos is excellent, and the resistance to video steganalysis is strong.

Steganography algorithm based on modified EMD-coded PU partition modes for HEVC videos

As High Efficiency Video Coding (HEVC) is a worldwide popular video coding standard, the steganography of HEVC videos has gained more and more attention. Prediction unit (PU) is one of the most important innovative modules of HEVC; thus, PU partition mode-based steganography is becoming a novel branch of HEVC steganography. However, the embedding capacity of this kind of steganography is limited by the types of PU partition modes. To solve the problem, modified exploiting modification direction (EMD)-coded PU partition mode-based steganography is proposed in this paper, which can hide a secret digit in a (2n + x − 1)-ary notational system in a pair of PU partition modes and thus enlarging the capacity. Furthermore, two mapping patterns for PU partition modes are analyzed, and the one that performs the better is selected as the final mapping pattern. Firstly, 8 × 8- and 16 × 16-sized PU partition modes are recorded according to the optimal mapping pattern in the video encoding process. Then, PU partition modes are modified by using the proposed method to satisfy the requirement of secret information. Finally, the stego video can be obtained by re-encoding the video with the modified PU partition modes. Experimental results show that the embedding capacity can be significantly enlarged, and compared with the state-of-the-art work, the proposed method has much larger capacity while keeping high visual quality.

Detection of transcoded HEVC videos based on in-loop filtering and PU partitioning analyses

With the increasing maturity of video editing technology, forgers are more inclined to transcode videos to High Efficiency Video Coding (HEVC) videos, as HEVC not only enables people to enjoy high-definition videos but also allows broadcasters to stream it more efficiently across networks. Therefore, to verify the originality and authenticity, it is of great significance to propose an algorithm for detecting transcoded HEVC videos. In this paper, a theoretical model of video transcoding is first constructed, and a novel transcoding detection algorithm based on In-loop Filtering and Prediction Units (PU) Partition (IFPP) is proposed. By analyzing the statistical characteristics of strong and normal filtering modes in deblocking filtering and calculating offset values in Sample Adaptive Offset (SAO) filtering, the transcoding traces in inter-coded frames can be captured. In addition, PU partition statistics are also extracted to make full use of traces in intra-coded frames. By fusing these subfeatures, the proposed IFPP feature with 17 dimensions can be obtained, which is further fed to the Support Vector Machine (SVM) classifier. Finally, experiments are conducted on datasets with various coding parameters. Results show that the proposed algorithm outperforms existing algorithms and has better robustness.

Fuzzy Based Adaptive Deblocking Filters at Low-Bitrate HEVC Videos for Communication Networks

In-loop filtering significantly helps detect and remove blocking artifacts across block boundaries in low bitrate coded High Efficiency Video Coding (HEVC) frames and improves its subjective visual quality in multimedia services over communication networks. However, on faster processing of the complex videos at a low bitrate, some visible artifacts considerably degrade the picture quality. In this paper, we proposed a four-step fuzzy based adaptive deblocking filter selection technique. The proposed method removes the quantization noise, blocking artifacts and corner outliers efficiently for HEVC coded videos even at low bit-rate. We have considered Y (luma), U (chroma-blue), and V (chroma-red) components parallelly. Finally, we have developed a fuzzy system to detect blocking artifacts and use adaptive filters as per requirement in all four quadrants, namely up 45°, down 45°, up 135°, and down 135° across horizontal and vertical block boundaries. In this context, experimentation is done on a wide variety of videos. An objective and subjective analysis is carried out with MATLAB software and Human Visual System (HVS). The proposed method substantially outperforms existing post-processing deblocking techniques in terms of YPSNR and BD_rate. In the proposed method, we achieved 0.32–0.97 dB values of YPSNR. Our method achieved a BD_rate of +1.69% for the luma component, −0.18% (U) and −1.99% (V) for chroma components, respectively, with respect to the state-of-the-art methods. The proposed method proves to have low computational complexity and has better parallel processing, hence suitable for a real-time system in the near future.

Frame-Wise Detection of Double HEVC Compression by Learning Deep Spatio-Temporal Representations in Compression Domain

Detection of double compression is regarded as one primary step in analyzing the integrity of digital videos, which is of prominent importance in video forensics. However, current methods are vulnerable with the severe lossy quantization in the recompression process such that it is challenging to obtain reliable frame-wise detection results, especially for the high efficiency video coding (HEVC) standard. In view of these issues, in this paper, a hybrid neural network is proposed to reveal abnormal frames in HEVC videos with double compression by learning robust spatio-temporal representations from coding information in the compression domain. Based on the statistical analysis of Coding Units (CUs), it is interesting to find that HEVC video streams contain “rich” coding information that could be leveraged to identify abnormal traces caused by double compression. Two types of coding information maps, including CU Size Map (CSM) and CU Prediction mode Map (CPM), are exploited. In contrast with the conventional paradigm relying on pixel-level representations of decoded frames, CSMs and CPMs of a short-time video clip are treated as the input, aiming to achieve high robustness against recompression of low quality. In our hybrid neural network, an attention-based two-stream residual network is proposed to learn hierarchical representations from CSM and CPM, which are then jointly optimized by the attention-based fusion module. Finally, the temporal variation is modeled by Long Short-Term Memory (LSTM) to obtain frame-wise detection results. We have conducted extensive experiments considering various video content and coding parameters, such as bitrates and sizes of Group of Picture. Experimental results show that our approach can obtain state-of-the-art performance compared with conventional methods, especially when videos are recompressed in the low bitrate coding scenarios.

Rate-distortion/complexity analysis of HEVC, VVC and AV1 video codecs

With the advent of smartphones and tablets, the amount of online video traffic has increased enormously. This, together with the growing popularity of high-definition video, motivated the development of the High Efficiency Video Coding (HEVC) standard, released in 2013, with the aim of achieving a 50% bitrate reduction with respect to its predecessor, namely H.264/MPEG-4 Advanced Video Coding (AVC). However, new contents with greater resolutions and requirements arise every day, making it necessary to reduce the bitrate to a further extent. In this regard, the efforts to become the leading video codec in the market resulted in two main contenders: the Joint Video Experts Team (JVET), which leads the development of the Versatile Video Coding (VVC) standard, and the Alliance for Open Media (AOMedia), which spearheads the AOMedia Video 1 (AV1) project. In this context, this paper presents a rate-distortion/complexity analysis of HEVC, VVC and AV1 main video codecs using objective measures of assessment in order to analyze their real capabilities. The analysis, which was done using well-defined test conditions, reveals that VVC considerably outperforms both HEVC and AV1 in terms of coding efficiency.

Predicting split decisions in MPEG-2 to HEVC video transcoding

This paper proposes learning-based approaches for transcoding videos compressed using the Moving Picture Experts Group 2 format into the High Efficiency Video Coding (HEVC) format. In the training mode of the transcoder, mappings between extracted features and split decisions are calculated. While in the transcoding mode, the split decisions of Coding Units of the HEVC video are predicted. Two formulations are proposed for the prediction of split decisions based on multi model and multi-tier solutions. In the former solution, multi models are generated based on the total number of split flags in a coding unit. While in the latter solution, split decisions are modelled at three different coding depths. The proposed solutions are evaluated in terms of excessive bitrate, drop in PSNR, classification accuracy, model generation time and transcoding speedup. It is shown that the multi-tier solution maintains the rate-distortion behaviour of full re-encoding at the expense of lower gain in transcoding speedup. In comparison to existing work, it is shown that the proposed solutions offer a significant enhancement in terms of rate-distortion performance and classification accuracy.

HEVC Video Encryption Algorithm Based on Integer Dynamic Coupling Tent Mapping

A selective encryption algorithm is proposed to improve the efficiency of high efficiency video coding (HEVC) video encryption and ensure the security of HEVC videos. The algorithm adopts the integer dynamic coupling tent mapping optimization model as the pseudo-random sequence generator, and multi-core parallelization is used as the sequence generation mechanism. The binstrings during the process of context adaptive binary arithmetic coding are selected for encryption, which conforms to the features of invariable binstream and compatible format in terms of video encryption. Performance tests for six types of standard videos with different resolutions were performed. The results indicated that the encryption algorithm has a large key space and benefits from a high encryption effect.

Improved Low-Bitrate HEVC Video Coding Using Deep Learning Based Super-Resolution and Adaptive Block Patching

Good-quality video coding for low-bitrate applications is essential for narrow bandwidth transmission and limited capacity storage. In this paper, we propose an adaptive downsampling-based coding model to improve the low-bitrate compression efficiency of high-efficiency video coding (HEVC). At the encoder, the video sequence is adaptively divided into key frames (KFs) and nonkey frames (NKFs), which are encoded at the original resolution and at a reduced resolution, respectively. At the decoder, a super-resolution method based on deep learning and gradient transformation is used to upscale the NKFs. To improve the quality of NKFs without additional information during decoding, we use motion estimation to find the most similar blocks between the upscaled NKFs and the associated high-resolution KFs. Then, an adaptive patching-based method is used to warp the low-quality NKF blocks with the high-quality KF blocks. Experimental results indicate that for standard high-definition test video sequences, the maximum improvement in the peak signal-to-noise ratio can reach 3.54 dB, and the critical bitrate can reach 9.89 Mb/s at a low bitrate when compared to HEVC. These results demonstrate significant improvements compared to existing methods.

Understanding the Video Codec Jungle: A Comparison of TCO and Compression Efficiency

The codec landscape has rapidly accelerated, especially with the recent announcement by Apple to fully support High-Efficiency Video Coding (HEVC) and join the Alliance for Open Media (AOM) consortium. At the 2018 National Association of Broadcasters (NAB) Show, the AOM froze its specification of the AV1 codec. In addition, the Joint Video Exploration Team (JVET), a collaborative initiative between the Motion Picture Experts Group (MPEG) and International Telecommunication Union - Telecommunication (ITU-T), is making progress with the development of the next generation of MPEG compression, which will address future applications such as 8K and virtual reality (VR) six degrees of freedom (6DoF). This paper provides a thorough comparison of different codec technologies, including Advanced Video Coding (AVC), HEVC, AV1, and Versatile Video Coding (VVC). The total cost of ownership (TCO) and compression efficiency of each codec will be evaluated based on objective testing methods [i.e., peak signal-to-noise ratio (PSNR) and Video Multimethod Assessment Fusion (VMAF)] and subjective testing. Based on the factual data presented, readers will be able to make a better decision about which codec to use.

Detection of Transcoding from H.264/AVC to HEVC Based on CU and PU Partition Types

High Efficiency Video Coding (HEVC) is a worldwide popular video coding standard due to its high coding efficiency. To make profits, forgers prefer to transcode videos from previous standards such as H.264/AVC to HEVC. To deal with this issue, an efficient method is proposed to expose such transcoded HEVC videos based on coding unit (CU) and prediction unit (PU) partition types. CU and PU partitioning are two unique syntactic units of HEVC that can reflect a video’s compression history. In this paper, CU and PU partition types of I pictures and P pictures are firstly extracted. Then, their mean frequencies are calculated and concatenated as distinguishing features, which are further sent to a support vector machine (SVM) for classification. Experimental results show that the proposed method can identify transcoded HEVC videos with high accuracy and has strong robustness against frame-deletion and shifted Group of Pictures (GOP) structure attacks.

Data embedding in high efficiency video coding (HEVC) videos by modifying the partitioning of coding units

A data embedding solution in HEVC videos is proposed by modifying the partitioning of coding units (CUs). The partitions of a CU are first represented as a sequence of binary flags. The flags pertaining to 16 × 16 sub CUs are used as a cover for data embedding, where 6 or 4 message bits are embedded per CU. The data embedding algorithm guarantees that a maximum of one partition is modified per message segment, therefore, in a given CU, either 0, 1 or 2 partitions are modified. The proposed solution is assessed in terms of message payload, number of modified partitions, loss in video quality as indicated by the PSNR results, mean objective scores and excessive bitrate. The proposed solution can embed messages with up to an average payload of 32.6 kbit/s with a corresponding average distortion of <0.5 dB. Comparisons with existing solutions reveal that the proposed solution maintains similar message payloads with less modifications of CU partitioning and at the same time resulting in less distortions for the cover video.