Group Of Pictures Research Articles

Optimal Bit Allocation at Frame Level for Rate Control in HEVC

Rate control plays an important role in video coding systems, which makes the output bitrate of a video encoder equal to the target bitrate while minimizing the distortion of the compressed video. However, most of the existing rate control schemes achieve accurate bitrate control at the loss of rate-distortion (R-D) performance. This paper proposes an effective frame level bit allocation method to improve the R-D performance whilst maintaining the high accuracy of the bitrate control in HEVC. First, an improved R-D model is presented at the frame level, which, by making use of the information of encoded frames more completely, achieves higher accuracy with lower bitrate mismatch rate. Second, a group of picture (GOP) level bit allocation approach is introduced with the consideration of the temporal R-D dependency among different GOPs, which can further enhance the R-D performance. Finally, to achieve optimal R-D performance globally, a formulation for optimal frame level bit allocation is developed with a GOP level Lagrange multiplier introduced, which takes into account the coding effects of a current frame on the other frames within the same GOP. A scheme of recursive Taylor expansion is employed to find the GOP level Lagrange multiplier. Experimental results demonstrate that the proposed method can achieve considerable R-D performance improvement and accurate bitrate control. Specifically, our method shows 3.9% and 3.8% BD-rate saving in average compared against the HEVC reference software HM16.7 with rate control in the low-delay B and low-delay P coding structures, respectively.

Load balancing algorithm for a local video network

Server grows as client increases. If single server hardware up scaling has reached the limit, then multiple servers become solution. For local server requirement, the aforementioned solution has been sufficient. Load balancer distributes tasks to participating servers through designated algorithm. This paper assesses some load balancer algorithms for local video on demand network through experimental network that is developed in the lab for further research. As socket programming implemented, the WW is outperformed other algorithm in term of delay, jitter, losses as well as group of picture (GoP) delay.

A Rate Control Model of MPEG-4 Encoder for Video Transmission over Wireless Sensor Network

Recently, multimedia application has a lot of attention in the research community, especially when transmitting video over IEEE 802.15.4 standard. This is due to the capability of providing low complexity with low cost, but still maintaining the quality of video in term of packet received. However, transmitting video over Wireless Sensor Network (WSN) posed a new research challenges with high bandwidth demand and energy constrained of sensor nodes. MPEG-4 video codec is one of the compression techniques that used to decrease the amount of bandwidth required to meet WSN environment. Therefore, video encoding is a useful tool for rate control to control the video bit rate and maintaining the video quality especially in real-time communication applications. Video bit rate is affected by quantization scale, frame rate, and Group of Picture (GOP) size. A rate control model called enhanced Video Motion Classification based (e-ViMoC) model is proposed in this paper to produce the desired bit rate that complies to the IEEE 802.15.4 standard, while at the same time preserving the video quality. The analysis has shown that, the video transmission using e-ViMoC rate control achieves enhancement in delivery ratio, energy consumption and video quality (PSNR) when compared to video transmission using uncompressed video.

Simulation of the effect of bit assignment to multi-hypothesis pictures per group for motion compensated video compression (MCVC)

This paper describes the Lagrange based formulation of bit-assignment for multi-hypothesis frames over number of pictures in a group for motion compensated video compression. Theoretical insight on bit assignment is accomplished by applying Lagrange multiplier and its cost function for a real-time computational analysis. Video signals are compressed at optimal bit allocation for qualitative reconstruction at the decoder (receiver) depending upon the number of hypothesis used in coding the signal. This has made multi-hypothesis motion compensation (MHMC) a great process in the prediction of the actual signal through the combination of more than one motion compensated prediction (MCP). Optimal Bits were allocated to multi-hypothesis frames using MATLAB to optimize the number of hypothesis frames per group of pictures (GOP) decoded for an improved signal compression. Simulation results show that an optimal bit assignment of the range 0.089 bpp to 0.159 bpp was realized with eight numbers of hypotheses frames per (GOP). This implies that varying number of hypotheses actualizes optimal bit assignment for motion compensation. This shows a qualitative signal compression performance analysis for high technology motion pictures in communication services and storage processes. It also addresses a better prediction due to greater time correlation between hypotheses and helps in the improvement of coding and reconstruction quality of video signals. Key words: Bit-rate, frame, Group of Pictures (GOP), Motion Compensation,multi-hypothesis, Video Compression.

Scalable Video Coding Using Accordion Discrete Wavelet Transform and Tucker Decomposition for Multimedia Applications

The digital world demands the transmission and storage of high quality video for streaming and broadcasting applications, the constraints are the network bandwidth and the memory of devices for the various multimedia and scientific applications, the video consists of spatial and temporal redundancies. The objective of any video compression algorithm is to eliminate the redundant information from the video signal during compression for effective transmission and storage. The correlation between the successive frames has not been exploited enough by the current compression algorithms. In this paper, a novel method for video compression is presented. The proposed model, applies the transformation on set of group of pictures (GOP). The high spatial correlation is achieved from the spatial and temporal redundancy of GOP by accordion representation and this helps to bypass the computationally demanding motion compensation step. The core idea of the proposed technique is to apply Tucker Decomposition (TD) on the Discrete Wavelet Transform (DWT) coefficients of the Accordion model of the GOP. We use DWT to separate the video in to different sub-images and TD to efficiently compact the energy of sub-images. The blocking artifacts will be considerably eliminated as the block size is huge. The proposed method attempts to reduce the spatial and temporal redundancies of the video signal to improve the compression ratio, computation time, and PSNR. The experimental results prove that the proposed method is efficient especially in high bit rate and with slow motion videos.

Detection of Double Compressed HEVC Videos Using GOP-Based PU Type Statistics

Double compression detection is of great significance to recover the compression history of suspicious videos and help analyzers locate tampering points in forged videos. However, the lack of efficient methods still haunts researchers when it comes to double HEVC compression detection with a shifted group of pictures (GOP) structures. To tackle this problem, we first analyze the properties of re-encoded frames in double compressed HEVC videos. It is found that the abnormal statistics of prediction unit (PU) types in relocated I-frames can be used as the clue to expose double compression. Relocated I-frames are re-encoded P-frames, which were I-frames in the first compression. Different from the existing works, we designed the GOP-based features instead of frame-based features, enabling more efficient utilization of temporal variation patterns of PU-type statistics within each GOP unit. In the proposed method, the ratio of I-PUs and S-Pus is first computed to construct PU sequences for each GOP unit. The three-tap median filter is applied to PU sequences in order to mitigate the influence of various video contents. Several first-order statistics, including average, variation, and maximum/minimum, are selected to generate a 6-D detection feature. Then, multi-layer perceptron (MLP) is used to classify GOP-based features. Finally, the score fusion strategy is applied to obtain the detection result of the input video. Several publicly available YUV sequences are used to construct the double compression dataset with various coding parameters. In experiments, the proposed method outperforms several state-of-the-art methods under different settings of bitrates and GOP structures. The proposed method has more robust detection capability even when tampered videos are generated by combining two video clips with different GOP sizes.

GOP Structure-Independent Quantization Parameter Cascading in Video Coding

The popular lossy coding standards set unequal quantization parameter (Qp) values to its successive frames or slices aimed at minimizing its encoder distortion subject to a bitrate constraint. Nowadays, this Qp cascading (QPC) strategy is normally achieved by the derivation of an optimization problem with regard to the group-of-picture (GOP) structure of successive frames. In this paper, we propose to extract coding unit (CU)-level predicting information and project them to frame-level dependency with real-time neural networks. Thus, we successfully estimate and update the frame-level dependency in a GOP without prior knowledge to the GOP-level prediction structure. Finally, the dependency is utilized in adaptive QPC for diverse GOP structures. The experimental results on high efficiency video coding (H.265/HEVC) demonstrate the effectiveness of our proposed framework, which achieves adaptive QPC for both random access (RA) and low delay (LD) structures with superior rate-distortion (RD) performances.

The impact of successive B frames on TV signal using different compression techniques and video resolution

The aim of this paper is to examine the impact of successive B frames using compression techniques (H.264 and H.265) and resolution (4K, 2K, Full HD and SD) when changing their quality in a television system. The analysis was done for a case when the number of successive B frames in the Group of Pictures (GOP) was B = 0 and B = 4. The results obtained for analysis parameters: compression ratio, Peak Signal Noise Ratio (PSNR), bitrate, P frames and B frames are presented by tables and graphics, based on which conclusions were made.

Equivalence class formation as a function of preliminary training with pictorial stimuli.

The present experiment investigated the effects of preliminary training with pictorial stimuli on the subsequent formation of three 5-member equivalence classes (A➔B➔C➔D➔E) in 84 university students assigned to seven groups of 12. In the Abstract (ABS) group, all stimuli were abstract shapes. In the Picture (PIC) group, the C stimuli were pictures, and the remaining stimuli were the same abstract shapes as in the ABS group. For the remaining five groups, all stimuli were the same abstract shapes as in the ABS group. However, across groups, preliminary training involved either the establishment of conditional relations with simultaneous (SMTS) or delayed (DMTS) matching-to-sample with 0 s, 3 s, 6 s, or 9 s between the abstract C stimuli and the meaningful pictures. For the ABS and the PIC groups, 16.7% and 83.3% of participants formed classes, respectively. Preliminary training with SMTS and DMTS with 0 s, 3 s, and 6 s produced a linear increase in the likelihood of equivalence class formation, 41.7%, 50%, and 75%, respectively. Increasing the duration of delay further from 6 s to 9 s produced a substantial decline, 50%. This experiment extends knowledge about how including meaningful pictures enhances equivalence class formation.

Novel PID-Fuzzy Video Rate Controller for High-Delay Applications of the HEVC Standard

The new High Efficiency Video Coding (HEVC) standard is targeted for high-resolution video contents that are suitable for many high-delay video communications applications, such as video streaming. A few-rate control algorithms (RCAs) have been proposed for the HEVC in the literature. Almost all state-of-the-art RCAs proposed for the HEVC are optimized for constant bit rate low-delay applications. In order to reach a higher visual quality in high-delay applications, such as video streaming, a proportional integral derivative (PID)-fuzzy RCA with the buffering constraint is proposed for the HEVC in this paper. The proposed RCA is designed to minimize fluctuations of quantization parameter (QP) while the buffering constraints are fully obeyed. Unlike conventional fuzzy-PID algorithms that use a PID controller for control and a fuzzy system for tuning the PID parameters, the proposed algorithm uses a fuzzy controller with proportional, integral, and derivative inputs. A base QP is calculated for each group of picture (GOP) by the PID-fuzzy controller and determined QP offsets are added to the base QP to obtain the QP of frames at different temporal layers in the GOP. Experimental results indicate that high rate-distortion performance, fulfilled buffering constraint, and the high visual quality of compressed video are gained simultaneously by the proposed algorithm with low computational complexity.

Multilevel optimal λ decision for low-delay rate control in high-efficiency video coding

By considering the rate distortion optimization (RDO) characteristics, a multilevel optimal λ decision algorithm for the rate control is proposed. With the process of getting the optimal λ for the group of picture (GoP)-level rate control, the frame-level rate control, and the coding tree unit (CTU)-level rate control, a special relationship of the current level λ and the next level λ can be built. First, an approximate linear bit allocation relationship is used between the bit rate and the motion compensation prediction error to assign the bit rate more reasonably for every frame. Second, the multilevel optimal λ decision scheme is proposed. To minimize the whole distortion of a sequence, an effective way is to get the optimal RDO of every GoP. Then, with the RDO process of the GoP-level rate control, an optimal frame λ ratio for each frame in current GoP can be obtained. For the optimal frame λ ratio, its value is mainly decided by the distortion effect, which is calculated by an employed source distortion propagation chain. Finally, an optimal CTU λ clip scheme is proposed to get the optimal RDO of every CTU. The experimental results show that the coding quality of the proposed rate control algorithm is improved significantly with little loss of the bit rate accuracy.

Efficient Delivery of Scalable Video Using a Streaming Class Model

When we couple the rise in video streaming with the growing number of portable devices (smart phones, tablets, laptops), we see an ever-increasing demand for high-definition video online while on the move. Wireless networks are inherently characterised by restricted shared bandwidth and relatively high error loss rates, thus presenting a challenge for the efficient delivery of high quality video. Additionally, mobile devices can support/demand a range of video resolutions and qualities. This demand for mobile streaming highlights the need for adaptive video streaming schemes that can adjust to available bandwidth and heterogeneity, and can provide a graceful changes in video quality, all while respecting viewing satisfaction. In this context, the use of well-known scalable/layered media streaming techniques, commonly known as scalable video coding (SVC), is an attractive solution. SVC encodes a number of video quality levels within a single media stream. This has been shown to be an especially effective and efficient solution, but it fares badly in the presence of datagram losses. While multiple description coding (MDC) can reduce the effects of packet loss on scalable video delivery, the increased delivery cost is counterproductive for constrained networks. This situation is accentuated in cases where only the lower quality level is required. In this paper, we assess these issues and propose a new approach called Streaming Classes (SC) through which we can define a key set of quality levels, each of which can be delivered in a self-contained manner. This facilitates efficient delivery, yielding reduced transmission byte-cost for devices requiring lower quality, relative to MDC and Adaptive Layer Distribution (ALD) (42% and 76% respective reduction for layer 2), while also maintaining high levels of consistent quality. We also illustrate how selective packetisation technique can further reduce the effects of packet loss on viewable quality by leveraging the increase in the number of frames per group of pictures (GOP), while offering a means of reducing overall error correction and by providing equality of data in every packet transmitted per GOP.

Detecting double compressed MPEG videos with the same quantization matrix and synchronized group of pictures structure

Double compression detection is a crucial stage in digital image and video forensics. However, the detection of double compressed videos is challenging when the video forger uses the same quantization matrix and synchronized group of pictures (GOP) structure during the recompression history to conceal tampering effects. A passive approach is proposed for detecting double compressed MPEG videos with the same quantization matrix and synchronized GOP structure. To devise the proposed algorithm, the effects of recompression on P frames are mathematically studied. Then, based on the obtained guidelines, a feature vector is proposed to detect double compressed frames on the GOP level. Subsequently, sparse representations of the feature vectors are used for dimensionality reduction and enrich the traces of recompression. Finally, a support vector machine classifier is employed to detect and localize double compression in temporal domain. The experimental results show that the proposed algorithm achieves the accuracy of more than 95%. In addition, the comparisons of the results of the proposed method with those of other methods reveal the efficiency of the proposed algorithm.

Detection of Double Compression for HEVC Videos With Fake Bitrate

Bitrate is an important criterion for digital video quality. Forgers prefer to up-convert bitrate of videos without improving the video quality at all, especially for videos over the Internet. Therefore, proposing an effective algorithm to expose fake bitrate videos becomes an important issue in digital video forensics. In this paper, a novel method based on prediction unit (PU) partition types is proposed to detect high efficient video coding (HEVC) videos with fake bitrate. PU partition type is one unique syntactic unit of HEVC and could be reflected by compression history, while it has not been addressed in the topic of fake bitrate detection. The proposed method adopts support vector machine classifiers with feature vectors formed by histograms of PU partition types in the first P-frame in each group of pictures (GOPs). The performance of the algorithm is compared with state-of-the-art algorithms on videos of various resolutions and bitrates. The experimental results show that the proposed method can identify the fake bitrate videos with high accuracy, and is robust to frame-deletion, copy-paste, and shifted GOP structure attacks.

Detection of Double-Compressed H.264/AVC Video Incorporating the Features of the String of Data Bits and Skip Macroblocks

Today’s H.264/AVC coded videos have a high quality, high data-compression ratio. They also have a strong fault tolerance, better network adaptability, and have been widely applied on the Internet. With the popularity of powerful and easy-to-use video editing software, digital videos can be tampered with in various ways. Therefore, the double compression in the H.264/AVC video can be used as a first step in the study of video-tampering forensics. This paper proposes a simple, but effective, double-compression detection method that analyzes the periodic features of the string of data bits (SODBs) and the skip macroblocks (S-MBs) for all I-frames and P-frames in a double-compressed H.264/AVC video. For a given suspicious video, the SODBs and S-MBs are extracted for each frame. Both features are then incorporated to generate one enhanced feature to represent the periodic artifact of the double-compressed video. Finally, a time-domain analysis is conducted to detect the periodicity of the features. The primary Group of Pictures (GOP) size is estimated based on an exhaustive strategy. The experimental results demonstrate the efficacy of the proposed method.

Block-Composed Background Reference for High Efficiency Video Coding

A block-composed background reference method is proposed in this paper for High Efficiency Video Coding (HEVC). For a group of picture (GoP), the first reconstructed picture is served as an initial background reference, which probably includes foreground content. In the subsequent coding, some background coding tree units (CTUs) in every picture are selected to be compressed with high quality. These reconstructed CTUs are used to update the background reference as well as replace the foreground content. Finally, a high-quality background reference is generated to better exploit the long-term temporal correlation in the video. There are three key technical contributions in the proposed coding scheme. First, the background reference is generated gradually by block updating instead of picture updating, which makes the scheme free of bit-rate burst and more suitable for real-time applications and can generate high-quality background reference even with complicated foreground. Second, we propose an approach to select background CTUs by taking both temporal and spatial smoothness into account. Third, we propose a model to decide the coding parameters of the selected background CTUs based on the overall picture activity, which essentially pursues the GoP-level optimal performance when making CTU-level decision. The proposed background reference is implemented into HEVC, and the experimental results demonstrate a significant improvement in coding efficiency. Compared with HEVC, our method can averagely save 14% bits in surveillance and conferencing sequences with negligible increase of encoding and decoding complexity. In particular, it can still averagely save 7.3% bits in HEVC general test sequences. Obviously, the proposed scheme can be applied to more general video contents.

A Hybrid Tactic Model Intended for Video Compression Using Global Affine Motion and Local Free-Form Transformation Parameters

Video compression marks its necessity when a huge sized video needs to be transmitted. The process starts with the identification of GoP (group of pictures), which depends on I- (intra), B- (bidirectional) and P- (predicted) frames determination. GoP is fixed, where consecutive frames are placed in an orderly manner based on the GoP size. Conventionally, B-frames lead to buffering of memory within the past and future frames consuming more computational time. Such issues are handled by an adaptive framework for determining frames based on matching criteria rather than fixed GoP. NSEW (North–South–East–West) affine translation (NAT) is proposed for replacing B with either I- or P-frame. The proposed framework involves video compression using affine motion-based free-form transformation and video decompression using warping methodologies for the purpose of compressing and decompressing the video sequence, based on the resulted I- and P-frames. B-spline transformation was also initiated at local level along with global affine transformation to improve the subjective quality of the decompressed video sequence. The methodology was investigated for the file size, computational time, peak-signal-to-noise ratio (PSNR) and Structural Similarity index (SSIM), which proved the superiority of the proposed technique. Further, the methodology was also investigated with optimizing the affine motion parameters (AMP) using nonlinear least squares, Broyden–Fletcher–Goldfarb–Shanno (BFGS) and limited-memory BFGS which yet again proved to be far more superior to conventional techniques with an average PSNR of 38.98 dB with LBFGS. To further improve the subjective quality, affine B-spline-based motion estimation using LBFGS was implemented and observed the average PSNR gain to be 42.03 dB.

SSIM-Motivated Two-Pass VBR Coding for HEVC

We propose a structural similarity (SSIM)motivated two-pass variable bit rate control algorithm for High Efficiency Video Coding. Given a bit rate budget, the available bits are optimally allocated at group of pictures (GoP), frame, and coding unit (CU) levels by hierarchically constructing a perceptually uniform space with an SSIM-inspired divisive normalization mechanism. The Lagrange multiplier λ, which controls the tradeoff between perceptual distortion and bit rate, is adopted as the GoP level complexity measure. To derive λ, Laplacian distribution-based rate and perceptual distortion models are established after the first pass encoding, and the target bits are dynamically allocated by maintaining a uniform Lagrange multiplier level for each GoP through λ equalization. Within each GoP, rate control is further performed at frame and CU levels based on SSIM-inspired divisive normalization, aiming to transform the prediction residuals into a perceptually uniform space. Experiments show that the proposed scheme achieves high accuracy rate control and superior rate-SSIM performance, which is further verified by subjective visual testing.

HEVC 부호화기의 적응적 참조 구조 변경 방법

본 논문에서는 HEVC (High Efficiency Video Coding) 부호화기의 성능 향상을 위한 적응적 참조 구조 변경 방법을 제안한다. 제안하는 방법은 입력 영상 내에 장면 전환, 장면 회전, Light on/off, Fade in/out 등의 이벤트 발생 시 저하되는 화면 간 예측의 성능을 향상시키기 위해 참조 픽쳐 리스트를 수정하여 참조 구조를 변경한다. GOP (Group Of Pictures) 단위로 한정된 계층적 참조 구조에서 영상 이벤트가 발생한 픽쳐를 기준으로 두 개의 서브 그룹으로 분할하고 각각의 서브 그룹 내에서 참조 픽쳐를 결정한다. 이를 통해, 시간적 중복성이 높은 참조 픽쳐를 화면 간 예측에서 사용하여 부호화 효율을 향상시킨다. 또한, HEVC의 계층적 부호화 구조 특징을 이용하여 이벤트 발생 후 처음으로 부호화되는 픽쳐를 CRA (Clean Random Access) 픽쳐로 변경하여 부호화 속도를 향상시키는 방법을 제안한다. 제안하는 방법은 이벤트 발생 후 처음으로 부호화되는 프레임은 화면 내 예측이 매우 높은 확률로 선택된다는 통계적 특징을 기반으로 화면 간 예측을 수행하지 않는다. 실험 결과 본 논문에서 제안하는 적응적 참조 구조 변경 방법은 HM 16.0 대비 CTC (Common Test Condition)에서 평균 0.3%의 BD-rate를 개선하였으며 부호화 속도는 평균 4.9% 향상시켰다. 또한, 이벤트 발생에 따른 픽쳐 타입 변경 방법은 평균 0.11%의 BD-rate 저하가 있었지만 부호화 속도를 평균 12.2% 향상시킬 수 있었다. This paper proposes adaptive reference structure decision method to improve the performance of HEVC (High Efficiency Video Coding) encoder. When an event occurs in the input sequence, such as scene change, scene rotation, fade in/out, or light on/off, the proposed algorithm changes the reference structure to improve the inter prediction performance. The proposed algorithm divides GOP (Group Of Pictures) into two sub-groups based on the picture that has such event and decides the reference pictures in the divided sub-groups. Also, this paper proposes fast encoding method which changes the picture type of first encoded picture in the GOP that has such event to CRA (Clean Random Access). With the statistical feature that intra prediction is selected by high probability for the first encoded picture in the GOP carrying such event, the proposed fast encoding method does not operate inter prediction. The experimental result shows that the proposed adaptive reference structure decision method improves the BD-rate 0.3% and reduces encoding time 4.9% on average under the CTC (Common Test Condition) for standardization. In addition, the proposed reference structure decision method with the picture type change reduces the average encoding time 12.2% with 0.11% BD-rate loss.

Evaluation of Impairment Caused by MPEG Video Frame Loss

This article presents a study on the impact of video frame losses on the quality perceived by users. Video compression standards, such as MPEG, use a sequence of frames called Group of Pictures (GOP), which is a video compression method which a frame is expressed in terms of one or more neighboring frames. This dependence between frames impacts directly in the quality because a loss of a reference frame prevents the decoding of other frames in GOP, thereby reducing the user-perceived quality. The assessment of quality in this article is estimated by Peak Signal Noise Ratio (PSNR), which compares the original and the received images. Computer simulations were used to show that the degradation on the quality may vary for different patterns of GOPs and type of lost frames.