Prediction In Video Coding Research Articles

Spatio-Temporal Convolutional Neural Network for Enhanced Inter Prediction in Video Coding.

This paper presents a convolutional neural network (CNN)-based enhancement to inter prediction in Versatile Video Coding (VVC). Our approach aims at improving the prediction signal of inter blocks with a residual CNN that incorporates spatial and temporal reference samples. It is motivated by the theoretical consideration that neural network-based methods have a higher degree of signal adaptivity than conventional signal processing methods and that spatially neighboring reference samples have the potential to improve the prediction signal by adapting it to the reconstructed signal in its immediate vicinity. We show that adding a polyphase decomposition stage to the CNN results in a significantly better trade-off between computational complexity and coding performance. Incorporating spatial reference samples in the inter prediction process is challenging: The fact that the input of the CNN for one block may depend on the output of the CNN for preceding blocks prohibits parallel processing. We solve this by introducing a novel signal plane that contains specifically constrained reference samples, enabling parallel decoding while maintaining a high compression efficiency. Overall, experimental results show average bit rate savings of 4.07% and 3.47% for the random access (RA) and low-delay B (LB) configurations of the JVET common test conditions, respectively.

Chroma Intra Prediction With Lightweight Attention-Based Neural Networks

Neural networks can be successfully used for cross-component prediction in video coding. In particular, attention-based architectures are suitable for chroma intra prediction using luma information because of their capability to model relations between different channels. However, the complexity of such methods is still very high and should be further reduced, especially for decoding. In this paper, a cost-effective attention-based neural network is designed for chroma intra prediction. Moreover, with the goal of further improving coding performance, a novel approach is introduced to utilize more boundary information effectively. In addition to improving prediction, a simplification methodology is also proposed to reduce inference complexity by simplifying convolutions. The proposed schemes are integrated into H.266/Versatile Video Coding (VVC) pipeline, and only one additional binary block-level syntax flag is introduced to indicate whether a given block makes use of the proposed method. Experimental results demonstrate that the proposed scheme achieves up to -0.46%/-2.29%/-2.17% BD-rate reduction on Y/Cb/Cr components, respectively, compared with H.266/VVC anchor. Reductions in the encoding and decoding complexity of up to 22% and 61%, respectively, are achieved by the proposed scheme with respect to the previous attention-based chroma intra prediction method while maintaining coding performance.

Low Computational Coding-Efficient Distributed Video Coding: Adding a Decision Mode to Limit Channel Coding Load

Distributed video coding (DVC) is based on distributed source coding (DSC) concepts in which video statistics are used partially or completely at the decoder rather than the encoder. The rate-distortion (RD) performance of distributed video codecs substantially lags the conventional predictive video coding. Several techniques and methods are employed in DVC to overcome this performance gap and achieve high coding efficiency while maintaining low encoder computational complexity. However, it is still challenging to achieve coding efficiency and limit the computational complexity of the encoding and decoding process. The deployment of distributed residual video coding (DRVC) improves coding efficiency, but significant enhancements are still required to reduce these gaps. This paper proposes the QUAntized Transform ResIdual Decision (QUATRID) scheme that improves the coding efficiency by deploying the Quantized Transform Decision Mode (QUAM) at the encoder. The proposed QUATRID scheme's main contribution is a design and integration of a novel QUAM method into DRVC that effectively skips the zero quantized transform (QT) blocks, thus limiting the number of input bit planes to be channel encoded and consequently reducing both the channel encoding and decoding computational complexity. Moreover, an online correlation noise model (CNM) is specifically designed for the QUATRID scheme and implemented at its decoder. This online CNM improves the channel decoding process and contributes to the bit rate reduction. Finally, a methodology for the reconstruction of the residual frame (R^) is developed that utilizes the decision mode information passed by the encoder, decoded quantized bin, and transformed estimated residual frame. The Bjøntegaard delta analysis of experimental results shows that the QUATRID achieves better performance over the DISCOVER by attaining the PSNR between 0.06 dB and 0.32 dB and coding efficiency, which varies from 5.4 to 10.48 percent. In addition to this, results determine that for all types of motion videos, the proposed QUATRID scheme outperforms the DISCOVER in terms of reducing the number of input bit-planes to be channel encoded and the entire encoder's computational complexity. The number of bit plane reduction exceeds 97%, while the entire Wyner-Ziv encoder and channel coding computational complexity reduce more than nine-fold and 34-fold, respectively.

Tree-Structured Data Clustering-Driven Neural Network for Intra Prediction in Video Coding.

Intra prediction is a crucial part of video compression, which utilizes local information in images to eliminate spatial redundancy. As the state-of-the-art video coding standard, Versatile Video Coding (H.266/VVC) employs multiple directional prediction modes in intra prediction to find the texture trend of local areas. Then the prediction is made based on reference samples in the selected direction. Recently, neural network-based intra prediction has achieved great success. Deep network models are trained and applied to assist the HEVC and VVC intra modes. In this paper, we propose a novel tree-structured data clustering-driven neural network (dubbed TreeNet) for intra prediction, which builds the networks and clusters the training data in a tree-structured manner. Specifically, in each network split and training process of TreeNet, every parent network on a leaf node is split into two child networks by adding or subtracting Gaussian random noise. Then data clustering-driven training is applied to train the two derived child networks using the clustered training data of their parent. On the one hand, the networks at the same level in TreeNet are trained with non-overlapping clustered datasets, and thus they can learn different prediction abilities. On the other hand, the networks at different levels are trained with hierarchically clustered datasets, and thus they will have different generalization abilities. TreeNet is integrated into VVC to assist or replace intra prediction modes to test its performance. In addition, a fast termination strategy is proposed to accelerate the search of TreeNet. The experimental results demonstrate that when TreeNet is used to assist the VVC Intra modes, TreeNet with depth = 3 can bring an average of 3.78% bitrate saving (up to 8.12%) over VTM-17.0. If TreeNet with the same depth replaces all VVC intra modes, an average of 1.59% bitrate saving can be reached.

Attention-Based Neural Networks for Chroma Intra Prediction in Video Coding

Neural networks can be successfully used to improve several modules of advanced video coding schemes. In particular, compression of colour components was shown to greatly benefit from usage of machine learning models, thanks to the design of appropriate attention-based architectures that allow the prediction to exploit specific samples in the reference region. However, such architectures tend to be complex and computationally intense, and may be difficult to deploy in a practical video coding pipeline. This work focuses on reducing the complexity of such methodologies, to design a set of simplified and cost-effective attention-based architectures for chroma intra-prediction. A novel size-agnostic multi-model approach is proposed to reduce the complexity of the inference process. The resulting simplified architecture is still capable of outperforming state-of-the-art methods. Moreover, a collection of simplifications is presented in this paper, to further reduce the complexity overhead of the proposed prediction architecture. Thanks to these simplifications, a reduction in the number of parameters of around 90% is achieved with respect to the original attention-based methodologies. Simplifications include a framework for reducing the overhead of the convolutional operations, a simplified cross-component processing model integrated into the original architecture, and a methodology to perform integer-precision approximations with the aim to obtain fast and hardware-aware implementations. The proposed schemes are integrated into the Versatile Video Coding (VVC) prediction pipeline, retaining compression efficiency of state-of-the-art chroma intra-prediction methods based on neural networks, while offering different directions for significantly reducing coding complexity.

Improved CNN-Based Learning of Interpolation Filters for Low-Complexity Inter Prediction in Video Coding

The versatility of recent machine learning approaches makes them ideal for improvement of next generation video compression solutions. Unfortunately, these approaches typically bring significant increases in computational complexity and are difficult to interpret into explainable models, affecting their potential for implementation within practical video coding applications. This paper introduces a novel explainable neural network-based inter-prediction scheme, to improve the interpolation of reference samples needed for fractional precision motion compensation. The approach requires a single neural network to be trained from which a full quarter-pixel interpolation filter set is derived, as the network is easily interpretable due to its linear structure. A novel training framework enables each network branch to resemble a specific fractional shift. This practical solution makes it very efficient to use alongside conventional video coding schemes. When implemented in the context of the state-of-the-art Versatile Video Coding (VVC) test model, 0.77%, 1.27% and 2.25% BD-rate savings can be achieved on average for lower resolution sequences under the random access, low-delay B and low-delay P configurations, respectively, while the complexity of the learned interpolation schemes is significantly reduced compared to the interpolation with full CNNs.

A simple encoder scheme for distributed residual video coding

Rate-Distortion (RD) performance of Distributed Video Coding (DVC) is considerably less than that of conventional predictive video coding. In order to reduce the performance gap, many methods and techniques have been proposed to improve the coding efficiency of DVC with increased system complexity, especially techniques employed at the encoder such as encoder mode decisions, optimal quantization, hash methods etc., no doubt increase the complexity of the encoder. However, low complexity encoder is a widely desired feature of DVC. In order to improve the coding efficiency while maintaining low complexity encoder, this paper focuses on Distributed Residual Video Coding (DRVC) architecture and proposes a simple encoder scheme. The main contributions of this paper are as follows: 1) propose a bit plane block based method combined with bit plane re-arrangement to improve the dependency between source and Side Information (SI), and meanwhile, to reduce the amount of data to be channel encoded 2) present a simple iterative dead-zone quantizer with 3 levels in order to adjust quantization from coarse to fine. The simulation results show that the proposed scheme outperforms DISCOVER scheme for low to medium motion video sequences in terms of RD performance, and maintains a low complexity encoder at the same time.

Analysis of Affine Motion-Compensated Prediction in Video Coding

Motion-compensated prediction is used in video coding standards like High Efficiency Video Coding (HEVC) as one key element of data compression. Commonly, a purely translational motion model is employed. In order to also cover non-translational motion types like rotation or scaling (zoom), e. g. contained in aerial video sequences such as captured from unmanned aerial vehicles (UAV), an affine motion model can be applied. In this work, a model for affine motion-compensated prediction in video coding is derived. Using the rate-distortion theory and the displacement estimation error caused by inaccurate affine motion parameter estimation, the minimum required bit rate for encoding the prediction error is determined. In this model, the affine transformation parameters are assumed to be affected by statistically independent estimation errors, which all follow a zero-mean Gaussian distributed probability density function (pdf). The joint pdf of the estimation errors is derived and transformed into the pdfof the location-dependent displacement estimation error in the image. The latter is related to the minimum required bit rate for encoding the prediction error. Similar to the derivations of the fully affine motion model, a four-parameter simplified affine model is investigated. Both models are of particular interest since they are considered for the upcoming video coding standard Versatile Video Coding (VVC) succeeding HEVC. Both models provide valuable information about the minimum bit rate for encoding the prediction error as a function of affine estimation accuracies.

Distortion propagation modeling and its applications on frame level quantization control for predictive video coding

Inter-frame temporal prediction in video coding causes distortion propagation among adjacent frames. As a result, frame level bit allocation and quantization control are intrinsically to be optimized with dependent rate distortion optimization (RDO). Taking inter-frame distortion propagation into consideration, quantization parameter cascading (QPC) is an efficient technique for frame level quantization control in terms of dependent RDO. This paper proposes a temporal distortion propagation model by quantitatively evaluating the temporal distortion dependency. Block-level temporal trajectory is tracked via inter-prediction on down-sampled original frames to construct the temporal analysis chain, and then tree-style dependent analysis is implemented along the trajectory. The amounts of equivalent distortion propagated from the temporally adjacent frames are quantitatively measured. Then, a new distortion model is developed to facilitate measuring the RD (rate distortion) cost in terms of dependent RDO. Finally, a simplified trellis comprised of candidate quantization parameters (Qp) of the frames within one GOP, and the optimal Qp is searched via simplified dynamic programming to achieve global optimization. The simulation results verify that the frame level QPC algorithm achieves 3.46% BD-RATE saving on average, which is contributed by efficient bit allocation along temporal trajectory.

Transform Competition for Temporal Prediction in Video Coding

Transform plays a crucial role in video coding, since it is in the transform domain that the bitrate reduction happens. Recently, coding with transform competition has emerged, especially inside the ITU-T and ISO/MPEG Joint Video Exploration Team, where the adaptive multiple transforms have been thoroughly tested. In this paper, it is proposed to investigate the design of transform competition systems for the purpose of inter-predicted residuals. Several systems are proposed and evaluated in terms of coding efficiency, storage requirement, and encoding complexity. A first approach selects the best transform for a residual block in a pre-established list of transforms. An extended version is also considered, where the number of transforms varies within an image area. Experimental results show that the proposed designs achieve significant bitrate savings over high efficiency video coding from −0.2% to −1.4%.

Intra-Prediction Mode Propagation for Video Coding

Intra-prediction and inter-prediction are two major ways of prediction in video coding. In the state-of-the-art video coding schemes such as joint exploration model (JEM), these two kinds of prediction are performed independently as different modes. An inter-coded coding unit (CU) is only assigned with the motion information, without an intra-prediction mode (IPM). To capture the texture direction in an inter-coded CU with low complexity, we present an IPM propagation approach in this paper. In the proposed scheme, each $4\times 4$ sub-block in an inter-coded CU is assigned with a propagated IPM, which is fetched from a reference block, located by a motion vector. The propagated IPM is utilized in two aspects. First, the propagated IPM can help to perform inter–intra-prediction in the inter-coded CU. Inter-prediction controlled by the motion information and intra-prediction controlled by the propagated IPM are both performed. The final prediction is generated by a position-dependent weighted sum of the two kinds of prediction. Second, the propagated IPMs of neighboring inter-coded blocks can help to predict the IPM of the intra-coded CU in an inter-picture. The propagated IPM of an inter-coded neighboring block can also be treated as a most probable mode. Simulation results show that BD rate savings of 0.5%, 1.0%, and 1.0% on average for Y, Cb, and Cr components, respectively, are achieved in random access configurations under the common test condition, compared with JEM-7.1. On ultra-high definition sequences, the BD rate savings can reach 1.0%, 1.7%, and 2.3% for Y, Cb, and Cr components, respectively.

Reference Clip for Inter Prediction in Video Coding

Inter prediction is a fundamental technology in video coding to remove the temporal redundancy between video frames. Traditionally, the reconstructed frames are directly put into a reference frame buffer to serve as references for inter prediction. Using multiple reference frames increases the accuracy of inter prediction, but also incurs a waste of memory of the buffer since the content of reference frames is highly similar. To address this problem, we propose to organize the references at clip level in addition to frame level, i.e. the reference buffer stores not only reference frames, but also reference clips that are cropped regions selected from the reconstructed frames. Using clip-level references, we can manage the reference content more economically, since the content of multiple reference frames is divided into the singular content of each frame as well as the repetitive content that appears in multiple frames. For the repetitive content, only one copy is stored in reference clips so as to avoid duplicate. Moreover, using reference clips also facilitates the bit-rate allocation among reference content, i.e. the quality of each clip can be decided adaptively to achieve the rate-distortion optimization. In this paper, we propose a complete video coding framework using reference clips, and investigate the problems including how to generate reference clips as either singular content clips or repetitive content clips, how to manage the clips, how to utilize the clips for inter prediction, and how to allocate bit-rate among clips, in a systematic manner. The proposed video coding framework is implemented upon the state-of-the-art video coding scheme, High Efficiency Video Coding (HEVC). Experimental results show that our scheme achieves on average 5.1% and 5.0% BD-rate reduction than the HEVC anchor, in low-delay B and low-delay P settings, respectively. We believe that reference clip opens up a new dimension for optimizing inter prediction in video coding, and thus is worthy of further study.

Hybrid Wyner-Ziv Video Coding with No Feedback Channel

In this paper, we propose a hybrid Wyner-Ziv video coding structure that combines conventional motion predictive video coding and Wyner-Ziv video coding to eliminate the feedback channel, which is a major practical problem in applications using the Wyner-Ziv video coding approach. The proposed method divides a hybrid frame into two regions. One is coded by a motion predictive video coder, and the other by the Wyner-Ziv coding method. The proposed encoder estimates side information with low computational complexity, using the coding information of the motion predictive coded region, and estimates the number of syndrome bits required to decode the region. The decoder generates side information using the same method as the encoder, which also reduces the computational complexity in the decoder. Experimental results show that the proposed method can eliminate the feedback channel without incurring a significant rate-distortion performance loss.

Rate Allocation in Predictive Video Coding Using a Convex Optimization Framework.

Optimal rate allocation is among the most challenging tasks to perform in the context of predictive video coding, because of the dependencies between frames induced by motion compensation. In this paper, using a recursive rate-distortion model that explicitly takes into account these dependencies, we approach the frame-level rate allocation as a convex optimization problem. This technique is integrated into the recent HEVC encoder, and tested on several standard sequences. Experiments indicate that the proposed rate allocation ensures a better performance (in the rate-distortion sense) than the standard HEVC rate control, and with a little loss with respect to an optimal exhaustive research, which is largely compensated by a much shorter execution time.

Multi-directional Mode Reduction Algorithm for Intra Prediction in Video Coding

ABSTRACTThe recent digital video-coding standard achieves very high data compression by using several new coding features, such as intra/inter predictions, variable block size motion compensation, multiple motion vectors, quarter pixel for motion compensation and so on. Among all the features, one of the key features is intra prediction, which is based on the rate-distortion optimization (RDO) procedure that requires a true encoding of the intra block by trying all intra prediction modes. In this paper, we propose a multi-directional mode reduction (MDMR) algorithm using the prediction error to reduce the number of coding modes required for the RDO procedure. The proposed MDMR algorithm effectively estimates the prediction error of the block to narrow down the predictive modes to reduce RDO computation. Experimental results show that the proposed algorithm achieves a time reduction of more than 64% with negligible PSNR loss and bitrate increment.

Distributed video coding: Assessing the HEVC upgrade

In the distributed video coding (DVC) context, predictive Intra coding plays an important role not only as one of the most meaningful (low encoding complexity) benchmarks but also as the adopted coding mode for the so-called key frames used in the most popular DVC architectures. While the first DVC codecs adopted the H.263 Intra codec to play this role, it became clear around 2005 that the (standard) Intra coding reference codec had evolved and should become the H.264/AVC Intra codec, which has been used since then by the DVC literature. However, while DVC solutions have been evolving since those times, also predictive video coding has evolved and a new predictive video coding standard has recently emerged, the High Efficiency Video Coding (HEVC) standard. To acknowledge this evolution and be fair with the predictive coding technology, the improved DVC solutions should now integrate and be compared with the improved and more relevant HEVC standard and not anymore with the H.264/AVC 10-year-old standard. In this context, this paper proposes to upgrade the DVC tools and benchmarking by adopting the HEVC Intra codec to take the role that the H.264/AVC Intra standard was previously playing. To assess the impact of this inevitable upgrade, an advanced, representative DVC solution has been adopted and its performance assessed and compared in the context of the H.264/AVC and HEVC standard frameworks. The compression efficiency and computational complexity experiments performed allow concluding that upgrading DVC solutions with HEVC Intra coding allows strengthening the DVC positioning in the video coding landscape. This conclusion highlights how the experiments performed in this paper are critical to define the new DVC status quo.

Recursive Prediction for Joint Spatial and Temporal Prediction in Video Coding

Video compression systems use prediction to reduce redundancies present in video sequences along the temporal and spatial dimensions. Standard video coding systems use either temporal or spatial prediction on a per block basis. If temporal prediction is used, spatial information is ignored. If spatial prediction is used, temporal information is ignored. This may be a computationally efficient approach, but it does not effectively combine temporal and spatial information. In this letter, we provide a framework where available temporal and spatial information can be combined effectively to perform joint spatial and temporal prediction in video coding. Experimental results obtained from one sample realization of this framework show its potential.

On the Information Theoretic Performance Comparison of Causal Video Coding and Predictive Video Coding

Causal video coding is a coding paradigm where video source frames X <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sub> , X <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sub> ,..., X <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">N</sub> are encoded in a frame-by-frame manner, the encoder for each frame can use all previous source frames and all previous encoded frames, and the corresponding decoder can use only all previous encoded frames. In the special case where the encoder for each frame X <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">k</sub> is further restricted to enlist help only from all previous encoded frames, causal video coding is reduced to predictive video coding, which all MPEG-series and H-series video coding standards proposed so far are based upon. In this paper, we compare the rate distortion performance of causal video coding with that of predictive video coding from an information theoretic perspective by modeling each frame X <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">k</sub> itself as a source X <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">k</sub> ={X <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">k</sub> (i)} <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">i=1</sub> <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">∞</sup> . Let R <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">c</sub> *(D <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1,</sub> ...,D <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">N</sub> ) (R <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">p</sub> *(D1,...,DN), respectively) denote the minimum total rate required to achieve a given distortion level D <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sub> ,...,D <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">N</sub> in causal video coding (predictive video coding, respectively). We first show that like R <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">c</sub> *(D1,..., D <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">N</sub> ), for jointly stationary and totally ergodic sources X <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sub> , X <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sub> ,..., XN, R <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">p</sub> *(D <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sub> ,...,D <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">N</sub> ) is equal to the infimum of the nth order total rate distortion function R <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">p,n</sub> (D1,...,DN) over all n, where R <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">p,n</sub> (D <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sub> ,...,D <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">N</sub> ) itself is given by the minimum of an information quantity over a set of auxiliary random variables. We then prove that if the jointly stationary and totally ergodic sources X <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sub> ,..., X <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">N</sub> form a (first-order) Markov chain, we have R <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">p</sub> *(D <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sub> ,...,D <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">N</sub> )=R <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">c</sub> *(D <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sub> ,...,D <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">N</sub> ). However, this is not true in general if X <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sub> ,..., X <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">N</sub> do not form a (first-order) Markov chain. Specifically, we demonstrate that for independent and identically distributed vector source (X <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sub> ,..., X <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">N</sub> ), if X <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sub> ,..., X <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">N</sub> do not form a (first-order) Markov chain, then under some conditions on source frames and distortion, R <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">c</sub> *(D <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sub> ,..., D <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">N</sub> ) is strictly less than R <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">p</sub> *(D <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sub> ,..., D <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">N</sub> ) in general. Our techniques allow us to compare R <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">p</sub> *(D <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sub> ,..., D <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">N</sub> ) with R <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">c</sub> *(D <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sub> ,..., D <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">N</sub> ) even when the single-letter characterization of R <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">p</sub> *(D <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sub> ,..., D <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">N</sub> ), if any, is unknown.

Content-based irregularly shaped macroblock partition for inter frame prediction in video coding

Although rate–distortion adaptive, variable block-size motion compensation has significantly contributed to the performance improvement in compression efficiency of H.264/AVC over the other existing video coding standards, the shape of the block for motion compensation prediction (MCP) in H.264/AVC is still limited to the regular rectangle. To further improve the accuracy of MCP, a novel content-adaptive macroblock partition method for inter frame prediction is proposed in this paper, where irregularly shaped partitions are introduced to MCP. The proposed method irregularly segments the current coding macroblock (MB) by referring to the matching reference region (MRR) in the reference frames, which is obtained relying on the inherent structural similarity of the video contents among successive frames and can be traced with the relative displacement vector between the current MB and the MRR. Consequently, the irregular partition of the current MB is content adaptive and the partition information is implicitly indicated by the MRR, without resorting to an explicit description of the shape information of the irregular segmentation. Experimental results demonstrate that the proposed method outperforms the MCP based on the regular rectangle blocks in terms of coding efficiency, by providing more precise motion estimation for macroblocks at little cost of extra side information.

Lossless Image Compression Using Super-Spatial Structure Prediction

We recognize that the key challenge in image compression is to efficiently represent and encode high-frequency image structure components, such as edges, patterns, and textures. In this work, we develop an efficient lossless image compression scheme called super-spatial structure prediction. This super-spatial prediction is motivated by motion prediction in video coding, attempting to find an optimal prediction of structure components within previously encoded image regions. We find that this super-spatial prediction is very efficient for image regions with significant structure components. Our extensive experimental results demonstrate that the proposed scheme is very competitive and even outperforms the state-of-the-art lossless image compression methods.