Group Of Pictures Research Articles

Content-Aware Selective Encryption for H.265/HEVC using Deep Hashing Network and Steganography

Existing selective encryption schemes for High Efficiency Video Coding (HEVC) only focus on the encoding characteristics of syntax elements in entropy coding and lack an understanding of the video content. Consequently, a large amount of unnecessary encryption operations are utilized to protect insensitive video frames, resulting in low encryption efficiency. In this paper, we propose a content-aware selective encryption scheme for H.265/HEVC, which encrypts only the groups of pictures (GOPs) containing sensitive content and thus offers high efficiency. In our scheme, a deep hashing network is first adopted to retrieve video frames to determine the content-sensitive GOPs. Then, multiple prediction and residual syntax elements in sensitive GOPs are encrypted using a keystream sequence generated by the hyperchaotic Lorenz system. In addition, the direct current coefficient of each \(4\times 4\) transform block is exchanged with a pseudo-randomly selected non-zero alternating current coefficient to further offer stronger visual distortion. Finally, the sign bits used for marking each GOP-type are reversibly embedded into the encrypted syntax elements to facilitate the decoder to distinguish the GOPs that need to be decrypted. Experimental results indicate that the proposed content-aware selective encryption scheme can efficiently protect sensitive content and is robust against all common attacks. Furthermore, it outperforms other state-of-the-art HEVC selective encryption algorithms in terms of security performance.

Robust Reversible Watermarking Scheme in Video Compression Domain Based on Multi-Layer Embedding

Most of the existing research on video watermarking schemes focus on improving the robustness of watermarking. However, in application scenarios such as judicial forensics and telemedicine, the distortion caused by watermark embedding on the original video is unacceptable. To solve this problem, this paper proposes a robust reversible watermarking (RRW)scheme based on multi-layer embedding in the video compression domain. Firstly, the watermarking data are divided into several sub-secrets by using Shamir’s (t, n)-threshold secret sharing. After that, the chroma sub-block with more complex texture information is filtered out in the I-frame of each group of pictures (GOP), and the sub-secret is embedded in that frame by modifying the discrete cosine transform (DCT) coefficients within the sub-block. Finally, the auxiliary information required to recover the coefficients is embedded into the motion vector of the P-frame of each GOP by a reversible steganography algorithm. In the absence of an attack, the receiver can recover the DCT coefficients by extracting the auxiliary information in the vectors, ultimately recovering the video correctly. The watermarking scheme demonstrates strong robustness even when it suffers from malicious attacks such as recompression attacks and requantization attacks. The experimental results demonstrate that the watermarking scheme proposed in this paper exhibits reversibility and high visual quality. Moreover, the scheme surpasses other comparable methods in the robustness test session.

An Anti-Forensics Video Forgery Detection Method Based on Noise Transfer Matrix Analysis.

The dispute over the authenticity of video has become a hot topic in judicial practice in recent years. Despite detection methods being updated rapidly, methods for determining authenticity have limitations, especially against high-level forgery. Deleting the integral group of pictures (GOP) length in static scenes could remove key information in the video, leading to unjust sentencing. Anyone can conduct such an operation using publicly available software, thus escaping state-of-the-art detection methods. In this paper, we propose a detection method based on noise transfer matrix analysis. A pyramid structure and a weight learning module are adopted to improve the detection rate and reduce the false positive rate. In total, 80 videos were examined through delicate anti-forensic forgery operations to verify the detection performance of the proposed method and three previously reported methods against anti-forensic forgery operations. In addition, two of the latest learning-based methods were included in our experiments to evaluate the proposed method. The experimental results show that the proposed method significantly improves the detection of frame deletion points compared with traditional and learning-based methods, especially in low false positive rate (FPR) intervals, which is meaningful in forensic science.

Region-of-Interest Based Coding Scheme for Live Videos

In this paper, we introduce a novel rate control scheme specifically tailored for live broadcasting scenarios. Notably, in high-definition live transmissions of sports events and video game competitions that typically exceed 1080 p resolution and run at frame rates of 60 fps or higher, the transcoding speed of encoders often becomes a limiting factor, leading to streams with substantial bitrates but unsatisfactory quality metrics. To enhance the overall Quality of Service (QoS) without increasing the bitrate, it is essential to improve the quality of Regions of Interest (ROI).Our proposed solution presents an ROI-based rate reservoir model that ingeniously leverages Convolutional Neural Networks (CNNs) to predict rate control parameters. This approach aims to optimize the bitrate allocation within high bitrate live broadcasts, thus enhancing the image quality within ROIs. Experimental outcomes demonstrate that this algorithm manages to increase the bitrate by no more than 5%, effectively redistributing the reduced bitrate across the entire Group of Pictures (GOP). As a result, it ensures a gradual decrease in the quality of Regions of Uninterest (ROU), thereby maintaining a balanced quality experience throughout the broadcasted content.

Real-time motion estimation based video steganography with preserved consistency and local optimality

AbstractThis paper introduces a novel steganographic technique for H.264 video that achieves an outstanding performance against state-of-the-art steganalysis techniques while maintaining real-time encoding performance constraints. The proposed technique embeds the secret message by altering the motion vectors (MVs) while preserving their local optimality and consistency feature to withstand the recently emerged steganalysis methods. Thanks to its macro-block (MB) basis architecture, the proposed technique satisfies the real-time constraints, eliminating the need to wait for the whole frame or group of pictures (GOP) and avoiding the need to perform any additional re-encoding step(s). Additionally, altering the MVs is performed in the motion estimation (ME) sub-pixel-refinement stage through a rule-based scheme that ensures each MB’s compatibility for embedding without detection by the aforementioned steganalysis methods. The proposed technique is integrated with the OpenH264 real-time video encoder and evaluated on widely used video sequences. The results prove that the proposed technique achieves a significant security performance against the steganalysis methods while maintaining an acceptable embedding rate, outperforming other state-of-the-art MV-based steganographic methods in real-time constrained environments. The proposed technique adds about $$1-2\%$$ 1 - 2 % overhead beyond the encoder running time. The source code is publicly available here: https://github.com/HassanMohamedGit/OpenH264-RealTime-steg.

Adaptive HEVC Video Steganography With High Performance Based on Attention-Net and PU Partition Modes

With the increasing popularity of digital video, video steganography has become a hot research topic in the field of covert communication and privacy protection. The existing prediction unit (PU) based video steganography often tends to result in large bit rate increase, which is also easily noticeable to the steganography analyst. To solve this problem, an adaptive steganography for HEVC video based on attention-net and PU partition modes is proposed. First, the distortion of modified PUs is analyzed from the perspective of rate distortion optimization at the group of pictures (GOP) level, and we find that modifying PU will lead to distortion accumulation and abnormal bitrate increase. Therefore, an adaptive distortion function based on the improved rate distortion cost is designed, and the embedding distortion is minimized by using Syndrome-Trellis Code (STC) steganography coding. Meanwhile, a super-resolution convolutional neural network with non-local sparse attention-net filter is proposed to replace the in-loop filter in HEVC to reconstruct the reference frame, thereby reducing the bitrate cost and improving the visual quality of stego-video. Experimental results show that the proposed algorithm can achieve superior perceptual quality and bitrate performance comparing with the sate-of-the-art works.

A Robust Video Watermarking Algorithm Based on Two-Dimensional Discrete Fourier Transform

Due to the continuous development and popularity of digital video technology, the copyright protection of digital video content has become an increasingly prominent issue. Digital video watermarking, as an effective means of digital copyright protection, has attracted widespread attention from academia and industry. The two-dimensional discrete Fourier transform (2D-DFT) template-based method has the advantages of good real-time performance and robustness, but the embedding capacity is small and cannot resist frame-dropping attack. To address this problem, a new template construction method is proposed in this paper, which can extend the embedding capacity from 1 bit each group of pictures (GOP) to theoretically n bits each GOP. In addition, by changing the GOP pattern, the method gains the ability to resist frame-dropping attacks. Experimental results show that the proposed method can achieve larger watermark capacity while maintaining strong robustness against image-processing attacks, geometric attacks, video-processing attacks and compression attacks.

Precise Encoding Complexity Control for Versatile Video Coding

Complexity reduction is a commonly used method to deal with complicated video coding standards, such as High Efficiency Video Coding (HEVC) and Versatile Video Coding (VVC). But the unstable performance under different video contents and Quantization Parameters (QPs) makes it difficult to precisely specify the target encoding time of every single sequence, which limits the practical use of the encoder. Inspired by rate control, in this paper, encoding time budget is regarded as a resource. We incorporate the hierarchical Group of Picture (GOP) encoding structure, the non-accelerated proportion, and block content variation within the frame, and design a top-down allocation and bottom-up feedback scheme, to achieve precise control of encoding complexity by controlling the maximum depth of QuadTree with nested Multi-type Tree (QTMT). In the scheme, Temporal ID (Tid)-based, Sum of Absolute Transformed Difference (SATD)-based methods and Linear (L) Model are designed to facilitate weighted allocation and feedback. The relationship between Planar Cost and encoding time is exploited as a Time-Cost (T-C) model, which guides the selection of Largest Coding Unit (LCU) encoding strategies in I-frames. For B-frames, we investigate a switching suppression assisted status-based method, to efficiently decide the encoding strategy of each LCU. Through the collaboration of the proposed technologies in the scheme, given any encoding time target achievable, encoding strategies will be automatically switched to help the actual encoding time gradually approach the target. To the best of our knowledge, this work is the first work on VVC-based encoding complexity control. The proposed scheme supports directly specifying the target encoding time or target Frame Per Second (FPS), and accurately realizing it within one pass. According to experimental results, under the target encoding time ratio of 80%, 60% and 40%, the average encoding time error is kept under 0.24%, 0.03% and 0.02% by our method, outperforming state-of-the-art works base on HEVC. These results prove the advantage of the proposed scheme.

Distributed Video Coding Based on Polar Codes

In this letter we present an improved distributed video coding (DVC) scheme based on polar coding techniques. Firstly, we adapt log-likelihood ratios (LLRs) for DVC with integer implementation of a discrete cosine transform (DCT). We propose a computationally efficient and numerically stable modification of these LLRs based on the simplified methods of polar codes decoding. We show that on average this approach provides 0.3 dB PSNR gain for DVC with LDPC accumulated (LDPCA) codes. Secondly, we introduce the nested shortened polar codes construction algorithm. We demonstrate that replacement of LDPCA by polar codes improves PSNR by 0.1 dB on average, whereas, for videos with relatively high motion level, the gain reaches up to 0.23, 0.39 and 0.55 dB for Group of Pictures (GOP) lengths 2, 4 and 8 frames, respectively. Finally, experimental results demonstrate that DVC with polar codes and Tal-Vardy list decoder operates up to two times faster than DVC with LDPCA code and belief propagation (BP) decoder.

Lightweight Video Super-Resolution for Compressed Video

Video compression technology for Ultra-High Definition (UHD) and 8K UHD video has been established and is being widely adopted by major broadcasting companies and video content providers, allowing them to produce high-quality videos that meet the demands of today’s consumers. However, high-resolution video content broadcasting is not an easy problem to be resolved in the near future due to limited resources in network bandwidth and data storage. An alternative solution to overcome the challenges of broadcasting high-resolution video content is to downsample UHD or 8K video at the transmission side using existing infrastructure, and then utilizing Video Super-Resolution (VSR) technology at the receiving end to recover the original quality of the video content. Current deep learning-based methods for Video Super-Resolution (VSR) fail to consider the fact that the delivered video to viewers goes through a compression and decompression process, which can introduce additional distortion and loss of information. Therefore, it is crucial to develop VSR methods that are specifically designed to work with the compression–decompression pipeline. In general, various information in the compressed video is not utilized enough to realize the VSR model. This research proposes a highly efficient VSR network making use of data from decompressed video such as frame type, Group of Pictures (GOP), macroblock type and motion vector. The proposed Convolutional Neural Network (CNN)-based lightweight VSR model is suitable for real-time video services. The performance of the model is extensively evaluated through a series of experiments, demonstrating its effectiveness and applicability in practical scenarios.

PRAM: Penalized Resource Allocation Method for Video Services

Abstract The human visual system response to picture quality degradation due to packet loss is very different from the responses of objective quality measures. While video quality due to packet loss may be impaired by at most for one Group of Pictures (GOP), its subjective quality degradation may last for several GOPs. This has a great impact on resource allocation strategies, which normally make decisions on instantaneous conditions of multiplexing buffer. This is because, when the perceptual impact of degraded video quality is much longer than its objective degradation period, any assigned resources to the degraded flow is wasted. This paper, through both simulations and analysis shows that, during resource allocation, if the quality of a video stream is significantly degraded, it is better to penalize this degraded flow from getting its full bandwidth share and instead assign the remaining share to other flows preventing them from undergoing quality degradation.

Reinforcement Learning for Rate-Distortion Optimized Hierarchical Prediction Structure

Video coding standards use a prediction structure to arrange video frames and exploit temporal correlations. In this aspect, it is crucial to resolve complicated temporal dependencies among frames to improve coding efficiency because the coding of a preceding frame affects the rate-distortion (R-D) performance of the subsequent frames. Previous algorithms have attempted to address the problem using handcrafted features or analytical models even though natural videos display various temporal characteristics. In this paper, we propose a reinforcement learning (RL)-based decision algorithm to build the optimal hierarchical prediction structure under a random-access configuration (RA-HPS) in Versatile Video Coding (VVC). Our goal is to maximize coding efficiency by selecting a series of optimal group of pictures (GOP) structures for coding. Accordingly, we formulate an adaptive GOP selection algorithm with a binary tree to represent a policy. We generate an optimal binary tree to minimize the sum of the R-D costs among all plausible binary trees. A new RL policy representation is defined, and the optimal policy is obtained by a sequential update. The tree grows with a hierarchical state-action and a reward sequence in each node. For efficient learning, the proposed technique uses a deep Q-network architecture to capture the temporal correlation between frames, which helps learn the policy of the tree-based RL framework effectively. Experimental results demonstrate that the proposed technique achieves a significant Bjontegaard-Delta (BD)-rate reduction compared with state-of-the-art GOP size-selection algorithms.

Group-based bi-directional recurrent wavelet neural network for efficient video super-resolution (VSR)

Video super-resolution (VSR) is an important technology for enhancing the quality of video frames. The recurrent neural network (RNN)-based approach is suitable for sequential data because it can use accumulated temporal information. However, since existing methods only tend to capture slow and symmetrical motion with low frame rate, there are still limitations to restore the missing details for more dynamic motion. Most of the previous methods using spatial information treat different types of the spatial features identically. It leads to lack of obtaining meaningful information and enhancing the fine details. We propose a group-based bi-directional recurrent wavelet neural network (GBR-WNN) to exploit spatio-temporal information effectively. The proposed group-based bi-directional RNN (GBR) framework is built on the well-structured process with the group of pictures (GOP). In a GOP, we resolves the low-resolution (LR) frames from border frames to center target frame. Because super-resolved features in a GOP are cumulative, neighboring features are improved progressively and asymmetrical motion can be dealt with. Also, we propose a temporal wavelet attention (TWA) adopting attention module for both spatial and temporal features simultaneously based on discrete wavelet transform. Experiments show that the proposed scheme achieves superior performance compared with state-of-the-art methods.

Joint video compression and encryption using parallel compressive sensing and improved chaotic maps

In this paper, we propose a method for joint compression and encryption of video using parallel compressive sensing (PCS) and improved chaotic maps. The method is shown to be computationally efficient without compromising the quality of the recovered video. The processing of the video frames are carried out at group of pictures (GOP) level which consists of both reference and non-reference frames. We apply DCT for sparsification of the reference frames and permute the DCT coefficients to improve the quality of the reconstructed video. We introduce a Logistic Tent Infinite Collapse Map (LT-ICM) which has a higher chaotic range and complex behavior, unpredictability, and sensitivity towards initial values and controlling parameters. The measurement matrix for PCS is generated by using one-dimensional LT-ICM. A novel substitution method is introduced using circular shift, xor, and modular addition operations. We finally shuffle the substituted frames using frame dependent two-dimensional LT-ICM chaotic sequence. Experimental analysis including comparisons with state-of-the-art approaches establishes the effectiveness of the proposed solution.

Cross-Modal Reconstruction for Tactile Signal in Human-Robot Interaction.

A human can infer the magnitude of interaction force solely based on visual information because of prior knowledge in human–robot interaction (HRI). A method of reconstructing tactile information through cross-modal signal processing is proposed in this paper. In our method, visual information is added as an auxiliary source to tactile information. In this case, the receiver is only able to determine the tactile interaction force from the visual information provided. In our method, we first process groups of pictures (GOPs) and treat them as the input. Secondly, we use the low-rank foreground-based attention mechanism (LAM) to detect regions of interest (ROIs). Finally, we propose a linear regression convolutional neural network (LRCNN) to infer contact force in video frames. The experimental results show that our cross-modal reconstruction is indeed feasible. Furthermore, compared to other work, our method is able to reduce the complexity of the network and improve the material identification accuracy.

Performance Comparison of H.264 and H.265 Encoders in a 4K FPV Drone Piloting System

With the rapid growth of video data traffic on the Internet and the development of new types of video transmission systems, the need for ad hoc video encoders has also increased. One such case involves Unmanned Aerial Vehicles (UAVs), widely known as drones, which are used in drone races, search and rescue efforts, capturing panoramic views, and so on. In this paper, we provide an efficiency comparison of the two most popular video encoders—H.264 and H.265—in a drone piloting system using first-person view (FPV). In this system, a drone is used to capture video, which is then transmitted to FPV goggles in real time. We examine the compression efficiency of 4K drone footage by varying parameters such as Group of Pictures (GOP) size, Quantization Parameter (QP), and target bitrate. The quality of the compressed footage is determined using four objective video quality measures: PSNR, SSIM, VMAF, and BRISQUE. Apart from video quality, encoding time and encoding energy consumption are also compared. The research was performed using numerous nodes on a supercomputer.

Neural texture transfer assisted video coding with adaptive up-sampling

Deep learning techniques have been extensively investigated for the purpose of further increasing the efficiency of traditional video compression. Some deep learning techniques for down/up-sampling-based video coding were found to be especially effective when the bandwidth or storage is limited. Existing works mainly differ in the super-resolution models used. Some works simply use a single image super-resolution model, ignoring the rich information in the correlation between video frames, while others explore the correlation between frames by simply concatenating the features across adjacent frames. This, however, may fail when the textures are not well aligned. In this paper, we propose to utilize neural texture transfer which exploits the semantic correlation between frames and is able to explore the correlated information even when the textures are not aligned. Meanwhile, an adaptive group of pictures (GOP) method is proposed to automatically decide whether a frame should be down-sampled or not. Experimental results show that the proposed method outperforms the standard HEVC and state-of-the-art methods under different compression configurations. When compared to standard HEVC, the BD-rate (PSNR) and BD-rate (SSIM) of the proposed method are up to -19.1% and -26.5%, respectively.

Mastering Quantization is Key for Video Compression

<italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">This article discusses some key elements and solutions for efficient video compression. More precisely, we introduce key differentiating techniques for video signal quantization within modern hybrid video codecs. First, we recall the fundamentals of scalar quantization and rate-distortion (R-D) theory and then describe the different control points and levels of granularity for optimizing signal quantization in modern video compression standards. From this common knowledge, we discuss approaches in how to jointly optimize those various levels of quantization refinements for improved video encoding quality. Most notably, we explain how to take advantage of a look-ahead module (available in most industrial encoder implementations) to model spatio-temporal coding dependencies, and how to further compute optimal quantization information from an R-D standpoint for a group of pictures (GOPs). Complementary to this first approach, we share some insights about a local quantization refinement (LQR) algorithm. Such an algorithm is often ignored in practical encoder implementation due to its apparent complexity; we develop how and why it can efficiently work in realtime software encoding</i> .

A Rate Control Model of MPEG-4 Encoder for Video Transmission over Wireless Sensor Network

Recently, multimedia application has a lot of attention in the research community, especially when transmitting video over IEEE 802.15.4 standard. This is due to the capability of providing low complexity with low cost, but still maintaining the quality of video in term of packet received. However, transmitting video over Wireless Sensor Network (WSN) posed a new research challenges with high bandwidth demand and energy constrained of sensor nodes. MPEG-4 video codec is one of the compression techniques that used to decrease the amount of bandwidth required to meet WSN environment. Therefore, video encoding is a useful tool for rate control to control the video bit rate and maintaining the video quality especially in real-time communication applications. Video bit rate is affected by quantization scale, frame rate, and Group of Picture (GOP) size. A rate control model called enhanced Video Motion Classification based (e-ViMoC) model is proposed in this paper to produce the desired bit rate that complies to the IEEE 802.15.4 standard, while at the same time preserving the video quality. The analysis has shown that, the video transmission using e-ViMoC rate control achieves enhancement in delivery ratio, energy consumption and video quality (PSNR) when compared to video transmission using uncompressed video.

Motion Estimation and Coding Structure for Inter-Prediction of LiDAR Point Cloud Geometry

In this paper, we investigate two fundamental problems of inter-prediction for Light Detection and Ranging (LiDAR) point cloud geometries: motion estimation (ME) and coding structure under the inter-exploration model of geometry-based point cloud compression (G-PCC). Under the inter-exploration model of G-PCC, the key to a good ME algorithm is to design an accurate criterion for estimating the bit cost of an octree node. In the previous work, a logarithmic relationship between the prediction distortion and the bit cost was used as the criterion. We first note that the multiscale binary prediction residue, instead of the prediction distortion, is the key factor in determining the bit cost. Then, a linear relationship between the number of 1s and 0s in the multiscale binary prediction residue and the bit cost is built and used as the ME criterion. In terms of the coding structure, only the IPPP coding structure is investigated in all previous geometry inter-prediction algorithms. The use of the hierarchical coding structure is first investigated in this paper. We further propose determining the use of the IPPP or hierarchical coding structure at the group of pictures (GoP)-level based on rate distortion optimization to improve the performance. The proposed algorithms are implemented in the inter-exploration model of G-PCC. The experimental results show that compared with the inter-exploration model of G-PCC, the proposed algorithms can provide an average of 2.1% bitrate savings.