Adaptive Group Of Picture Research Articles

Reinforcement Learning for Rate-Distortion Optimized Hierarchical Prediction Structure

Video coding standards use a prediction structure to arrange video frames and exploit temporal correlations. In this aspect, it is crucial to resolve complicated temporal dependencies among frames to improve coding efficiency because the coding of a preceding frame affects the rate-distortion (R-D) performance of the subsequent frames. Previous algorithms have attempted to address the problem using handcrafted features or analytical models even though natural videos display various temporal characteristics. In this paper, we propose a reinforcement learning (RL)-based decision algorithm to build the optimal hierarchical prediction structure under a random-access configuration (RA-HPS) in Versatile Video Coding (VVC). Our goal is to maximize coding efficiency by selecting a series of optimal group of pictures (GOP) structures for coding. Accordingly, we formulate an adaptive GOP selection algorithm with a binary tree to represent a policy. We generate an optimal binary tree to minimize the sum of the R-D costs among all plausible binary trees. A new RL policy representation is defined, and the optimal policy is obtained by a sequential update. The tree grows with a hierarchical state-action and a reward sequence in each node. For efficient learning, the proposed technique uses a deep Q-network architecture to capture the temporal correlation between frames, which helps learn the policy of the tree-based RL framework effectively. Experimental results demonstrate that the proposed technique achieves a significant Bjontegaard-Delta (BD)-rate reduction compared with state-of-the-art GOP size-selection algorithms.

Neural texture transfer assisted video coding with adaptive up-sampling

Deep learning techniques have been extensively investigated for the purpose of further increasing the efficiency of traditional video compression. Some deep learning techniques for down/up-sampling-based video coding were found to be especially effective when the bandwidth or storage is limited. Existing works mainly differ in the super-resolution models used. Some works simply use a single image super-resolution model, ignoring the rich information in the correlation between video frames, while others explore the correlation between frames by simply concatenating the features across adjacent frames. This, however, may fail when the textures are not well aligned. In this paper, we propose to utilize neural texture transfer which exploits the semantic correlation between frames and is able to explore the correlated information even when the textures are not aligned. Meanwhile, an adaptive group of pictures (GOP) method is proposed to automatically decide whether a frame should be down-sampled or not. Experimental results show that the proposed method outperforms the standard HEVC and state-of-the-art methods under different compression configurations. When compared to standard HEVC, the BD-rate (PSNR) and BD-rate (SSIM) of the proposed method are up to -19.1% and -26.5%, respectively.

Double compression detection for H.264 videos with adaptive GOP structure

As a blind forensic method, double compression detection is valid to multiple manipulations. However, the existing methods only consider to detect the videos with fixed Group of Pictures (GOP). In this paper, we put forward a novel double compression detection method for videos with both fixed and adaptive GOP structure in H.264 videos. Considering that video may contain adaptive GOPs caused by fast moving contents or scene changes, in our double compression detection scheme, temporal segmentation is first used to divide video into static and rapid periods which contain normal fixed and adaptive GOPs respectively. Then, new artifacts based on the sequence of frame’s byte count (FBC) are analyzed. A feature sequence composed of recognizable distances is generated by combining the artifacts in the static and rapid periods of video. Finally, to reveal the intrinsic property of the feature sequence, a scoring strategy is designed to determine whether or not double compression. The experiments demonstrate that the proposed scheme is effective to detect double compression of H.264 videos, and it outperforms other existing state-of-the-art methods.

ADAPTIVE GOP STRUCTURE TO H.264/AVC BASED ON SCENE CHANGE

This paper proposes an adaptive GOP structure with the new logic of frame comparison in H.264/AVC to achieve better quality and reduce bit rate. Initially Group of Pictures (GOP) is set to a fixed size. Frames are compared within that GOP using correlation. According to the correlation, GOP is changed within that fixed size. So, there will be no GOP size greater than that fixed size. This method does not calculate any threshold. Hence the time needed to calculate global or local threshold is eliminated. It is integrated with conventional video codec H.264/AVC. This method is compared with H.264/AVC of fixed GOP structure of sizes 4, 8, 12, 16, 32 and GOP structure with the length of entire video. The proposed method achieved gain in bit rate from 0.49% to 69.75% and PSNR gain from 2.5% to 0.3%.

Content-based group-of-picture size control in distributed video coding

Controlling the group of picture (GOP) size in distributed video coding (DVC) is a difficult but important task since it has a direct impact on the coding performance. This paper presents a framework to adaptively control the size of GOPs in a Wyner–Ziv encoder by means of encoder-side decisions based on support vector machines (SVM) that uses simple features extracted from the original video content. To train the SVM, firstly this work proposes how to compute the sequence of GOP sizes with the best rate-distortion performance given the set of GOP sizes that can be used during the encoding process. Then, an algorithm based on the previously trained SVMs is presented to control the actual GOP size each time a new decision can be taken at the encoder. Results show that the proposed algorithm can achieve a rate distortion performance close to the ideal one. Moreover, comparisons with a reference adaptive GOP size selection algorithm in the literature shows gains up to 2dB PSNR in the best conditions.

Balanced Parallel Scheduling for Video Encoding with Adaptive GOP Structure

Due to the nature of a dynamic group of picture (GOP) structure, parallel scheduling for video encoding becomes challenging. To address this, the balanced frame-level parallel scheduling algorithms are developed. The proposed approaches first determine the frame priority and then the thread priority assignment for scheduling. The concept of the algorithms lies in the analysis of coding complexity, temporal influence, and the required temporal burden to finish coding. To complete the scheduling with the dynamic GOP structure, a block-based abrupt and gradual scene change detection algorithm is also proposed to determine the GOP structure adaptively. The experiments show that the scheduling performance is close to the optimal. In addition, the concept of batch processing is incorporated so that the required buffer can be reduced.

Screen image sequence compression method utilizing adaptive block size coding and hierarchical GOP structure

To compress screen image sequence in real-time remote and interactive applications, a novel compression method is proposed. The proposed method is named as CABHG. CABHG employs hybrid coding schemes that consist of intra-frame and inter-frame coding modes. The intra-frame coding is a rate-distortion optimized adaptive block size that can be also used for the compression of a single screen image. The inter-frame coding utilizes hierarchical group of pictures (GOP) structure to improve system performance during random accesses and fast-backward scans. Experimental results demonstrate that the proposed CABHG method has approximately 47%–48% higher compression ratio and 46%–53% lower CPU utilization than professional screen image sequence codecs such as TechSmith Ensharpen codec and Sorenson 3 codec. Compared with general video codecs such as H.264 codec, XviD MPEG-4 codec and Apple’s Animation codec, CABHG also shows 87%–88% higher compression ratio and 64%–81% lower CPU utilization than these general video codecs.

High-performance adaptive group-of-picture rate control for H.264/AVC

In this paper, an adaptive group-of-picture (AGOP) control is proposed to improve the coding performance for the H.264/AVC system. The efficient algorithm is developed to find the quantized parameter with temporal correlation between inter-frames. According to the estimated parameters, the GOP coding type can be adaptively selected as AGOP or BGOP (basic GOP). The rate control with some decision functions is proposed to modify the quantization parameter (QP). The coding rate can accurately meet the target rate with adjusting QP value dynamically, while keeping low computational overhead. Simulation results show that the proposed algorithm can improve PSNR by about 1 dB in average compared with the original H.264/AVC reference software (JM12.4), from various sequences testing.

Embedded hardware low cost JPEG 2000 video coding system

In this paper, we introduce an embedded hardware low complexity JPEG 2000 video coding system. The hardware implementation itself exploits temporal redundancy by coding differential frames which are arranged in an adaptive group of picture (GOP) structure. The GOP structure is determined by statistical analysis of previously coded differential frames. We present a hardware video coding system that applies this interframe coding scheme to a digital signal processor environment. This system basically consists of a microprocessor (ADSP-BF533 Blackfin processor from analog devices) and a JPEG 2000 compression engine (ADV202). Our straightforward system is predestined for encoding surveillance-type videos with low cost.

An adaptive GOP structure selection for haar-like MCTF encoding based on mutual information

In conventional motion compensated temporal filtering based wavelet coding scheme, where the group of picture structure and low-pass frame position are fixed, variations in motion activities of video sequences are not considered. In this paper, we propose an adaptive group of picture structure selection scheme, which the group of picture size and low-pass frame position are selected based on mutual information. Furthermore, the temporal decomposition process is determined adaptively according to the selected group of picture structure. A large amount of experimental work is carried out to compare the compression performance of proposed method with the conventional motion compensated temporal filtering encoding scheme and adaptive group of picture structure in standard scalable video coding model. The proposed low-pass frame selection can improve the compression quality by about 0.3---0.5 dB comparing to the conventional scheme in video sequences with high motion activities. In the scenes with un-even variation of motion activities, e.g. frequent shot cuts, the proposed adaptive group of picture size can achieve a better compression capability than conventional scheme. When comparing to adaptive group of picture in standard scalable video coding model, the proposed group of picture structure scheme can lead to about 0.2~0.8 dB improvements in sequences with high motion activities or shot cut.

Adaptive group-of-pictures and scene change detection methods based on existing H.264 advanced video coding information

The H.264 advanced video coding (H.264/AVC) standard provides several advanced features such as improved coding efficiency and error robustness for video storage and transmission. In order to improve the coding performance of H.264/AVC, coding control parameters such as group-of-pictures (GOP) sizes should be adaptively adjusted according to different video content variations (VCVs), which can be extracted from temporal deviation between two consecutive frames. The authors present a simple VCV estimation to design adaptive GOP detection (AGD) and scene change detection (SCD) methods by using the obtained motion information, where the motion vectors and the sum of absolute transformed differences as VCV features are effectively used to design the AGD and SCD algorithms, respectively. In order to avoid unnecessary computation, the above VCV features are obtained only in the 4times4 inter-frame prediction mode. Simulation results show that the proposed AGD with SCD methods can increase the peak signal-to-noise ratio by 0.62 dB on average over the H.264/AVC operated with a fixed GOP size. Besides, the proposed SCD method can reach a scene change detection rate of 98%.

Adaptive Intra-Frame Assignment and Bit-Rate Estimation for Variable GOP Length in H.264

Video sequences usually have many scene changes and contain rapid motion. The coding efficiency of such sequences is generally low, and the image quality fluctuates. If the rapid motion and scene changes are taken into consideration for video compression, the compression performance can be improved and the fluctuation of image quality can be reduced. An adaptive group of picture (GOP)-length structure is proposed here to consider the scene changes and rapid motion in video sequences. Our proposed algorithm for this adaptive GOP-length structure is composed of three steps. First, a correlation measure is proposed to estimate the scene changes. Second, a new intra-frame assignment method is proposed. Third, an intra-quantization parameter rule for avoiding buffer overflow and image quality fluctuation is also suggested. Our proposed algorithm has an initial delay of one-GOP length, which makes it difficult to apply for real-time communication, but this type of short initial delay is acceptable in a streaming service

A Real-Time Chip Implementation for Adaptive Video Coding Control

The paper presents an adaptive coding control for real-time video coding systems. Based on temporal correlations, the group-of-pictures (GOP) of a video sequence is split into one basic GOP (BGOP) and many adaptive GOPs (AGOPs) and then processed accordingly. The advantage of this method is to improve the coding efficiency, particularly solving the scene change problem. Even if the coding bit rate is over the budget, the coding scheme does not require re-encoding, hence it is especially suitable for real-time coding. Based on the adaptive algorithm, the chip is realized to process the functions of picture type decision, coding rate estimation, quantization scale decision, scene-change detection and coding mode decision in the operation. The gate count of this chip is only about 2000 and its silicon area is 2.85 mm/sup 2/ using a 0.35 /spl mu/m CMOS process.

An Adaptive Video Coding Control Scheme for Real-Time MPEG Applications

This paper proposes a new rate control scheme to increase the coding efficiency for MPEG systems. Instead of using a static group of picture (GOP) structure, we present an adaptive GOP structure that uses more P-and B-frame coding, while the temporal correlation among the video frames maintains high. When there is a scene change, we immediately insert intramode coding to reduce the prediction error. Moreover, an enhanced prediction frame is used to improve the coding quality in the adaptive GOP. This rate control algorithm can both achieve better coding efficiency and solve the scene change problem. Even if the coding bit rate is over the predefined level, this coding scheme does not require re-encoding for real-time systems. Simulations demonstrate that our proposed algorithm can achieve better quality than TM5, and satisfactory reliability for detecting scene changes.