Advanced Motion Vector Prediction Research Articles

An HEVC steganalytic approach against motion vector modification using local optimality in candidate list

Abstract Current research of video steganalysis is mainly oriented to H.264 or even earlier formats. As the H.265/HEVC standard is becoming more and more popular, there have been steganographic methods specifically designed for HEVC videos. Previous steganalytic methods are no longer effective due to the new features of HEVC. The advanced motion vector prediction technique employed by HEVC provides a new way for motion vector (MV) modification, utilizing the index of the candidate MV list. Such modification cannot be detected by previous steganalytic methods because the value of MV is not changed. To solve this problem, we proposed a new steganalytic strategy employing not only the local optimality of MV but also the local optimality in the candidate list. Combining the two types of local optimality, a 40-dimensional feature set is designed. Extensive experiments are carried out to evaluate the effectiveness of the proposed feature set. It is shown that compared with three current steganalyzers, our approach improves the performance against typical MV modification methods under various settings. In particular, our approach significantly boosts the detection accuracy of the index modification method that is exclusive to HEVC videos. To our best knowledge, this is the first work of video steganalytic method exploiting the new features of HEVC format.

Non-Degraded Adaptive HEVC Steganography by Advanced Motion Vector Prediction

Current video steganography operates with either the decoded frame images or the compression coding parameters, which could cause quality degradation of the reconstructed frames. In this letter, by exploiting the advanced motion vector prediction (AMVP) technique of High Efficiency Video Coding (HEVC) standard, we propose a non-degraded adaptive steganographic approach for H.265/HEVC videos. The index value in the candidate list of the prediction unit (PU) is used for embedding. Experimental results demonstrate the superiority of the proposed steganographic approach against both hand-crafted feature-based and deep learning network-based steganalytic detectors. Our work explores a new embedding space that is not previously studied. It is a significant development in finding new ways to escape from video quality change-based steganalysis.

Intra Block Copy Mirror Mode for Screen Content Coding in Versatile Video Coding

Intra Block Copy (IBC) cannot work well for blocks with symmetry or mirror characteristic, because it is based on the translational block matching technique. In order to solve this problem, in this paper, an adequately designed IBC-Mirror mode is proposed for Versatile Video Coding (VVC), which extends our previous work. The newly proposed IBC-Mirror mode consists of two sub-modes, including IBC-Mirror Advanced Motion Vector Prediction (AMVP) mode and IBC-Mirror Merge mode. In IBC-Mirror AMVP mode, various Motion Estimation (ME) methods are applied, including previously proposed spatial search, and newly designed methods, i.e., Hash-based, Cache-based, Predictor-based and History-based IBC-Mirror search. As for IBC-Mirror Merge mode, Motion Information, composed of Block Vector (BV) and mirror flipping information, is added to the merge list to improve coding efficiency. Compared to the traditional IBC mode in VTM-10.0, the latest test model, the proposed mode can achieve 1~2% BD-rate saving. Compared to our previous work, methods in this paper can further achieve 0.5% coding gains and reduce 4% coding time. Higher coding efficiency and lower complexity show the superiority of extensions and improvements in this paper.

Novel skip motion estimation for efficient inter coding in HEVC

Video coding is necessary to transmit large amounts of video information. HEVC, which is the latest video coding standard, includes many advanced tools to achieve higher coding efficiency than previous standards, such as MPEG-1/2/4 and H.264/AVC. In particular, new techniques in inter prediction, such as skip, merge, and advanced motion vector prediction (AMVP) modes, drastically improve coding performance by reducing temporal redundancies between consecutive frames within a video. However, the skip mode, which does not perform a motion estimation, can deteriorate the coding performance, because of inaccurate motion vector prediction in videos with fast and complex motions. In order to solve this problem, a novel skip motion estimation method is proposed for efficient inter coding in HEVC. Unlike the conventional skip mode, the proposed skip mode performs the motion estimation, and then transmits the associated motion information. Experimental results demonstrate that the proposed method achieves the high coding performance with the low decoding complexity.

Deep Video Prediction Network-Based Inter-Frame Coding in HEVC

In this paper, we propose a novel Convolutional Neural Network (CNN) based video coding technique using a video prediction network (VPN) to support enhanced motion prediction in High Efficiency Video Coding (HEVC). Specifically, we design a CNN VPN to generate a virtual reference frame (VRF), which is synthesized using previously coded frames, to improve coding efficiency. The proposed VPN uses two sub-VPN architectures in cascade to predict the current frame in the same time instance. The VRF is expected to have higher temporal correlation than a conventional reference frame, and, thus it is substituted for a conventional reference frame. The proposed technique is incorporated into the HEVC inter-coding framework. Particularly, the VRF is managed in a HEVC reference picture list, so that each prediction unit (PU) can choose a better prediction signal through Rate-Distortion optimization without any additional side information. Furthermore, we modify the HEVC inter-prediction mechanisms of Advanced Motion Vector Prediction and Merge modes adaptively when the current PU uses the VRF as a reference frame. In this manner, the proposed technique can exploit the PU-wise multi-hypothesis prediction techniques in HEVC. Since the proposed VPN can perform both the video interpolation and extrapolation, it can be used for Random Access (RA) and Low Delay B (LD) coding configurations. It is shown in experimental results that the proposed technique provides -2.9% and -5.7% coding gains, respectively, in RA and LD coding configurations as compared to the HEVC reference software, HM 16.6 version.

Fast Integer Motion Estimation With Bottom-Up Motion Vector Prediction for an HEVC Encoder

Although advanced motion vector prediction (AMVP) modes based on motion estimation (ME) are selected significantly less due to the merge mode newly adopted in the high-efficiency video coding (HEVC), integer ME (IME) still occupies a large amount of computation in HEVC because the HEVC supports a highly flexible block partitioning structure. Introduction of the merge mode in HEVC substantially affects the optimal search algorithm of IME. Therefore, this gives a chance to reduce the computational complexity of ME with marginal drop of the compression efficiency. This paper proposes a new IME algorithm that significantly reduces its search ranges. Computational complexity of the proposed algorithm is reduced without serious degradation in coding efficiency by obtaining additional accurate motion vector predictors (MVPs) in a bottom-up order and searching narrow regions around multiple MVPs. These bottom-up MVPs are obtained from the prediction units (PUs) in the coding units (CUs) in the hierarchically lower level, which share the pixel area either completely or partially with the current PU. Although the encoder should perform the proposed IME in a bottom-up order from the smallest CUs to the larger CUs, several other processes, such as fractional ME and merges mode are performed in a top-down order to exploit several fast algorithms already adopted in the HEVC test model encoder. To keep compatibility with HEVC standard, the bottom-up MVPs are utilized for only IME. The bitstream is generated using standard AMVP. According to the simulation results, it is confirmed that the proposed algorithm improves the accuracy of MVPs, which leads to the reduction of IME computational complexity up to 81.95% on average with the Bjontegaard delta bitrate of 0.39%.

Fast inter-prediction algorithm based on motion vector information for high efficiency video coding

High Efficiency Video Coding (HEVC/H.265) is the latest international video coding standard, which achieves better compression ratio and supports higher resolution than Advanced Video Coding (H.264/AVC). However, HEVC/H.265 increases the computational burden. To reduce the coding complexity of the HEVC encoder, this paper proposes a fast inter-prediction algorithm to speed up coding time. We collect the average rate-distortion costs (RD-cost) of Skip modes and Merge modes to accelerate prediction unit (PU) mode decisions. In addition, we also acquire and analyze the motion vector range from Merge modes and Inter 2N × 2N modes to decide whether to execute Merge and advanced motion vector prediction (AMVP) of other PUs. The experimental results show that the proposed algorithm provides 48.54% time saving on average in random-access configuration and maintains good rate-distortion performance and video quality at the same time. The proposed algorithm also outperforms previous works.

Efficient Inter-View Motion Vector Prediction in Multi-View HEVC

When multi-view video sequences are encoded, the efficient motion vector prediction (MVP) is one of the most important techniques to increase the coding performance. The predictive MV should be highly correlated with the actual MV of the current block to be encoded. In this paper, we explain the inefficiency of the MVP algorithm in the multi-view high efficiency video codec (MV-HEVC) standard. In order to solve the inefficiency problem, we propose an enhanced advanced motion vector prediction algorithm in which the geometry interrelation between two neighboring views is derived based on epipolar geometry, similarity transform, and affine transform, and then predicted motion vectors (PMVs) for efficient MV coding are generated using obtained geometry relation. The experiment results show that the proposed scheme outperforms conventional MV-HEVC with a 1.32%–1.19% coding gain because it provides more efficient PMVs than conventional algorithms.

An Efficient Four-Parameter Affine Motion Model for Video Coding

In this paper, we study a simplified affine motion model-based coding framework to overcome the limitation of a translational motion model and maintain low-computational complexity. The proposed framework mainly has three key contributions. First, we propose to reduce the number of affine motion parameters from 6 to 4. The proposed four-parameter affine motion model can not only handle most of the complex motions in natural videos, but also save the bits for two parameters. Second, to efficiently encode the affine motion parameters, we propose two motion prediction modes, i.e., an advanced affine motion vector prediction scheme combined with a gradient-based fast affine motion estimation algorithm and an affine model merge scheme, where the latter attempts to reuse the affine motion parameters (instead of the motion vectors) of neighboring blocks. Third, we propose two fast affine motion compensation algorithms. One is the one-step sub-pixel interpolation that reduces the computations of each interpolation. The other is the interpolation-precision-based adaptive block size motion compensation that performs motion compensation at the block level rather than the pixel level to reduce the number of interpolation. Our proposed techniques have been implemented based on the state-of-the-art high-efficiency video coding standard, and the experimental results show that the proposed techniques altogether achieve, on average, 11.1% and 19.3% bits saving for random access and low-delay configurations, respectively, on typical video sequences that have rich rotation or zooming motions. Meanwhile, the computational complexity increases of both the encoder and the decoder are within an acceptable range.

Adaptive Bit Allocation for Consistent Video Quality in Scalable High Efficiency Video Coding

Scalable video coding (SVC) is a coding paradigm that allows once-encoded video content to be used in diverse scenarios. SVC-coded videos can be transmitted and rendered at specified bitrates according to network bandwidth and end device requirements. In this paper, an adaptive bit allocation algorithm is proposed for the emerging scalable High Efficiency Video Coding (SHVC) standard. The bit budget at the group-of-pictures level is allocated according to buffer occupancy. Picture complexity, measured using the predicted mean absolute difference (MAD), buffer occupancy, and hierarchical level, is proposed for regulating the bitrate at the picture level. The MAD of the current picture is predicted using a novel mean prediction error (MPE) model, which is obtained from the advanced motion vector prediction, and the test zone search specified in SHVC and the associated reference software of SHVC. Moreover, MPE is used to determine the number of assigned bits at the coding-tree-unit level. The bit budget of each level is incorporated with the $R-\lambda $ model for computing the required quantization parameter. Experimental results reveal that the proposed method achieves accurate bitrates with enhanced and consistent visual quality and more satisfactorily controls buffer occupancy compared with the state-of-the-art approaches reported in the literature.

Hardware-friendly advanced motion vector prediction method and its architecture design for high efficiency video coding

High efficiency video coding (HEVC) has been standardized as a means of meeting the coding requirements of 4 K (3840 × 2160) video. However, HEVC has a high computational complexity and a challenging hardware implementation. As a result, 4 K video applications are still limited. Consequently, the present study proposes a hardware-friendly advanced motion vector prediction (AMVP) method for HEVC which avoids the data dependency problem during the hardware pipeline operation. In the proposed method, the motion vector relationship between the largest coding unit (LCU) and the smaller coding units (CUs) and prediction units (PUs) is observed first. Based on the observation results, a linear model is constructed to estimate the motion vectors of the CUs and PUs from the motion vector of the LCU. It is shown that the proposed prediction method improves the hardware coding throughput by at least 53.8% compared to a traditional AMVP hardware realization, and increases the BD-rate by no more than 0.99% on average. To reduce the hardware implementation costs, a coefficient approximation and control signal sharing technique are also proposed in this paper to realize the proposed linear model. In addition, since the motion vectors of the small CUs and PUs are estimated in advance, a data pre-fetch technique can be employed to further increase the hardware-coding throughput. The experimental results show that the proposed AMVP design has a gate count of just 10 k.

Multidirectional parabolic prediction-based interpolation-free sub-pixel motion estimation

Motion estimation is one of the most effective key components in video codecs to eliminate temporal redundancies between successive frames. Sub-pixel motion estimation, including sub-pixel interpolation and sub-pixel motion vector prediction, has been employed in many recent international video coding standards to further improve the rate-distortion performance. Although sub-pixel motion estimation plays an important role in improving coding efficiency, it simultaneously brings significant computational complexity, especially in sub-pixel interpolation and motion vector prediction. In this paper, we propose a multidirectional parabolic prediction-based interpolation-free sub-pixel motion estimation scheme. First, four different directional parabolic prediction patterns are developed to estimate the candidate optimal sub-pixel positions, making full use of the available rate-distortion costs of neighboring positions in the integer-pixel motion estimation. Second, an improvement for each directional parabolic prediction pattern is realized by projection operations to generate a more precise candidate optimal sub-pixel position. Third, an optimal sub-pixel position decision method is presented to select the most likely predictive position from the four candidate sub-pixel positions derived in the previous steps. Finally, by fully reusing the information of the powerful advanced motion vector prediction (AMVP) in HEVC, a prediction enhancement mechanism is presented in which the final optimal sub-pixel motion vector (MV) is derived between the MV corresponding to the most likely predictive position and the predictive motion vector (PMV), without requiring extra interpolation operations. Experimental results show that the proposed algorithm significantly reduces the computational complexity of sub-pixel motion estimation and outperforms state-of-the-art interpolation-free sub-pixel motion estimation algorithms with much improved encoding performance and comparable computational complexity reduction.

Towards an FPGA-Based HEVC Encoder: A Low-Complexity Rate Distortion Scheme for AMVP

Advanced motion vector prediction (AMVP) is a new technique adopted in the latest high efficiency video coding (HEVC) standard. AMVP block predicts an initial motion vector of the current block from a given set of candidates by means of rate distortion (RD) optimization process. Due to the large number of different-sized blocks, simplification of RD optimization process in AMVP block is highly appreciated. Therefore, we present a new RD optimization technique for AMVP block in HEVC encoder. The proposed RD calculation approach finds the best AMVP candidate by processing less number of feature pixels per every block. Experimental results show notable speedup in terms of AMVP processing time with tolerable quality degradation (PSNR) and bitrate requirement. The proposed RD calculation technique reduces the RD computational complexity of the AMVP block by 87.5% as maximum (i.e. 1.7% of the whole encoder complexity). This improvement is accompanied with a modest average PSNR loss of 0.10 dB and an increase by 2.4% in terms of bitrate. On the other hand, we present an FPGA-based architecture for AMVP unit in HEVC encoder. The proposed architecture was prototyped, simulated and synthesized on Xilinx Virtex-7 XC7VX550T FPGA. At 188 MHz clock frequency, the proposed architecture processes 8 K (7680 \(\times \) 4320) YCrCb resolution at 60 fps while utilizing less than 1% of the FPGA resources.

HEVC fast reference picture selection

The new high efficiency video coding (HEVC) standard facilitates the realisation of ultrahigh-definition video applications. Although the compression efficiency of the HEVC standard is double that of the well-established H.264 standard, HEVC incurs a considerably higher encoding complexity. This Letter presents an algorithm aimed at expediting reference picture selection processes in HEVC encoding. The proposed algorithm is primarily based on a reference picture and block discovered using the advanced motion vector prediction (AMVP) algorithm. Two acceleration conditions are included in the proposed algorithm. The first early-decision condition entails examining the rate-distortion (RD) cost associated with AMVP, and if this cost is lower than a predefined threshold, motion estimation is confined to the AMVP-selected reference picture. Furthermore, for coding units of size 64 × 64 (N = 32) with a prediction-unit (PU) partitioning of 2N × N or N × 2N, the second PU uses the same reference picture as the first PU if the first PU satisfies the first early-decision condition. The experimental results revealed that the proposed algorithm considerably reduced the total encoding time with negligible RD degradation.

Fast motion estimation for HEVC with directional search

The new high-efficiency video coding (HEVC) standard is becoming popular owing to its excellent rate-distortion performance especially for emerging high-resolution applications. In HEVC more flexible coding options and partitioning types are provided for prediction units (PUs) and consequently motion estimation (ME) is more involved than previous video-coding standards. The existing fast ME algorithms, including the test zone search provided in the HEVC reference software, cannot generate high-quality video with reasonable computational complexity. A new fast integer-pixel ME algorithm for HEVC is presented. Starting with the initial motion vector obtained by the advanced motion vector prediction, the proposed directional search extends three search paths from the initial motion vector and considers the highly probable horizontal or vertical movement. Compared to the test zone search, the proposed method saves, on average, 60% time in integer-pixel ME with negligible rate-distortion degradation.

High Efficiency Video Coding

In recent years,with the development of video coding technology and the increasing popularity of high definition(HD)video content,the ISO/IEC Moving Picture Experts Group(MPEG)and the ITU-T Video Coding Experts Group(VCEG)formed the joint collaborative team on video coding(JCV-VC)in 2010which aims to develop the next-generation video coding standard for HD video application,called High Efficiency Video Coding(HEVC).So far the test model HM has been used for performance evaluation and algorithm test.The final international standard of HEVC has been completed and published in the end of 2012based on the working plan of JCV-VC.In this paper,the key technologies about HEVC will be surveyed,specifically including the quad-tree structure of coding unit,prediction unit and transform unit,the advanced motion vector prediction and merging technology,angular intra prediction,DCT-based fractional pixel interpolation filter and context adaptive arithmetic coding etc.Finally the coding performance of some coding tools and complexity of HEVC are analyzed in detail.

Advanced motion vector coding framework for multiview video sequences

The inconsistency between the temporal motion vector and the disparity vector fields causes a drastic loss in coding performance during multiview video coding. In order to resolve this inconsistency problem, we investigate a new approach for motion vector coding. In this paper, degradation in accuracy using a conventional method is experimentally analyzed using mathematical tools. Then, we introduce a new framework for motion vector coding, including modified Inter-mode and Adaptive Direct-mode, in which an advanced motion vector prediction method is employed. The main idea of the proposed motion vector prediction method is to separate each motion vector field from the others. For this purpose, we introduce three alternative motion vectors; a virtual disparity vector, a virtual temporal motion vector and a scaled motion vector. The experimental results show that the proposed method can resolve the inconsistency problem and consequently improve the overall rate-distortion performance. The average bit-rate savings is about 7.3% compared to JMVC 6.0. At maximum, the gain obtained is about 20%, corresponding to a PSNR of 1.42 dB.