Interframe Predictive Coding Research Articles

Interframe coding of feature descriptors for mobile augmented reality.

Streaming mobile augmented reality applications require both real-time recognition and tracking of objects of interest in a video sequence. Typically, local features are calculated from the gradients of a canonical patch around a keypoint in individual video frames. In this paper, we propose a temporally coherent keypoint detector and design efficient interframe predictive coding techniques for canonical patches, feature descriptors, and keypoint locations. In the proposed system, we strive to transmit each patch or its equivalent feature descriptor with as few bits as possible by modifying a previously transmitted patch or descriptor. Our solution enables server-based mobile augmented reality where a continuous stream of salient information, sufficient for image-based retrieval, and object localization, is sent at a bit-rate that is practical for today's wireless links and less than one-tenth of the bit-rate needed to stream the compressed video to the server.

Research on Key Algorithm of a New Generation Video Coding Standard HEVC

The video coding standard of a new generation, high efficiency video coding ( HEVC ), is a video coding standard of JCT-VT under planning, mainly orienting toward high definition television （HDTV）and video coding system. From the start of the basic structure of HEVC, this paper not only introduces comprehensively the key HEVC technologies in intra-frame and inter-frame predictive estimation, orthogonal transformation, filter compensation and entropy coding but also points out the hot issues and the latest research direction.

INTERFRAME CODING OF CANONICAL PATCHES FOR LOW BIT-RATE MOBILE AUGMENTED REALITY

Local features are widely used for content-based image retrieval and augmented reality applications. Typically, feature descriptors are calculated from the gradients of a canonical patch around a repeatable keypoint in the image. In this paper, we propose a temporally coherent keypoint detector and design efficient interframe predictive coding techniques for canonical patches and keypoint locations. In the proposed system, we strive to transmit each patch with as few bits as possible by simply modifying a previously transmitted patch. This enables server-based mobile augmented reality where a continuous stream of salient information, sufficient for image-based retrieval and localization, can be sent over a wireless link at a low bit-rate. Experimental results show that our technique achieves a similar image matching performance at 1/15 of the bit-rate when compared to detecting keypoints independently frame-by-frame and allows performing streaming mobile augmented reality at low bit-rates of about 20–50 kbps, practical for today's wireless links.

Real-time lossless compression of mosaic video sequences

This paper presents a simple, fast coding technique for lossless compression of mosaic video data. The design of a video codec needs to strike a balance between the compression performance and the codec throughput. Aiming to make the encoding throughput high enough for real-time lossless video compression, we propose a hybrid scheme of inter and intraframe coding. Interframe predictive coding is invoked only when the motion between adjacent frames is modest and a simple motion compensation operation can significantly improve the compression performance. Otherwise, still frame compression is performed to keep the complexity low. Experimental results show that the proposed scheme achieves higher lossless video compression ratio than existing methods such as JPEG-LS and JPEG-2000.

Multigrid block matching motion estimation for generic video coding

Motion estimation and compensation techniques have shown their efficiency to reduce temporal redundancies in video coding applications. Motion estimation analyzes the movement of objects in the scene. The resulting motion information allows to improve interframe predictive coding. This work deals with the study of motion estimation algorithms in the framework of image sequence coding. The desired features of a motion estimation algorithm in order to achieve high performances are the following. First, the algorithm provides an accurate motion compensated prediction. Second, it requires a low overhead information. Third, it leads to a smooth motion field close to the true motion in the scene. However, it is straightforward that more precise motion vectors need a higher overhead information, and vice versa. Consequently, a trade-off on the motion estimation complexity has to be found in order to optimally balance these two conflicting features. The motion estimation algorithm developed in this dissertation takes into account the above remarks. Block matching techniques are a promising approach for motion estimation in image sequence coding. In this framework, the classical full-search algorithm is widely used due to its simplicity and ease of hardware implementation. Nevertheless, it suffers serious drawbacks. In particular, it performs poorly on moving edges and introduces block artifacts in the motion compensated frame. Furthermore, it tends to produce noisy motion fields. Finally, it requires a high computational complexity. In this study, we aim at overcoming the above drawbacks in order to fulfill all the above desired features of a motion estimation algorithm. In this dissertation, we propose a locally adaptive multigrid block matching motion estimation technique. The motion vectors are iteratively refined on a set of grids with different resolution. Due to this multigrid structure, the algorithm produces a low energy prediction error and a robust motion field close to the true motion in the scene. Furthermore, the computation complexity is greatly reduced. Introducing a locally varying grid size allows to improve the motion vectors accuracy on moving edges and to reduce the overhead information in uniform areas. Therefore, the locally adaptive multigrid block matching motion estimation outperforms the full-search technique in terms of motion vectors accuracy and smoothness, amount of overhead information, coding performances, visual quality of the reconstructed sequences, and computational complexity. In order to avoid the block artifacts related to the block matching techniques, a VQ-based segmentation of the motion field is proposed. Blocks which contain several objects moving in different directions are segmented by means of VQ and a different motion vector is assigned to each of the resulting regions. The method improves motion compensated prediction along moving edges, resulting in higher coding performances and enhanced visual quality. In a further stage, an entropy criterion is introduced to control the motion estimation procedure. By evaluating the transmission cost of both the prediction error and the overhead motion information, it achieves an optimization of the motion estimation and compensation. More precisely, it leads to the optimal trade-off on the motion estimation complexity given an allotted bandwidth. This method is applied in both the locally adaptive multigrid block matching technique and the VQ-based segmentation of the motion field. Finally, a generic video coding system is presented. It supports a wide range of applications and is suitable for multimedia services. Simulation results show good coding performances and high visual quality.

Fast lattice-based gain-shape vector quantisation for image-sequence coding

A fast lattice-based gain/shape vector-quantisation method (FLGSVQ) is presented. The proposed technique combines the basic ideas of gain/shape vector quantisation (GSVQ) with lattice codes to offer an encoding process which is extremely fast compared with the conventional GSVQ. The main aim is to show how vector quantisers designed using the LBG algorithm can be substituted by lattice-based ones with vast reductions in encoding complexity, at a competitive image quality. A simple single-/two-stage coding system based on lattice quantisation is tested for interframe predictive coding of image sequences. Simulation results are provided to compare the performance of the proposed method with the conventional LBG-designed vector quantisation. These results, along with subjective evaluations, clearly demonstrate that lattice quantisation can be competitive with vector quantisation in terms of signal-to-noise performance, only at a very small fraction (4%) of the vector-quantiser complexity.

Hybrid picture coding with wavelet transform and overlapped motion-compensated interframe prediction coding

A novel video coding scheme using an orthonormal wavelet transform is proposed. The wavelet transform is used in a motion compensated interframe coder in which a blockless motion compensation technique is employed to increase efficiency of wavelet transform coding. A new scanning method for wavelet coefficients is also proposed which is rather different from subband coding. Simulation work is carried out to evaluate the proposed coding method. Significant improvement in subjective quality is obtained over that obtained with conventional hybrid coding methods that use blockwise motion compensation and DCT. Some improvement has also been realized in the signal to noise ratio. Although wavelet coding is still in its early stages of development, it appears to hold great promise for motion picture coding. >

Quadtree-structured linear prediction models for image sequence processing

A summary is presented of a study on two-dimensional linear prediction models for image sequence processing and its application to change detection and scene coding. The study focused on two-dimensional joint process modeling of interframe relationships, the derivation of computationally efficient matching algorithms, and the implementation of a block-adaptive interframe predictor for use in interframe predictive coding and change detection. In the approach presented, the spatial nonstationarity is handled by an underlying quadtree segmentation structure. A maximum-likelihood criterion and a simpler minimum-variance criterion are discussed as detection and segmentation rules. The results of this research indicate that a constrained joint process model involving only a single gain parameter and a shift parameter is the best tradeoff between performance and computational complexity.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

Lattice predictive modelling of 3-D random fields with application to interframe predictive coding of picture sequences

This paper further develops the recently proposed two-dimensional quarter-plane autoregressive (AR) lattice parameter modelling of two-dimensional (2-D) fields to three-dimensional (3-D) case with cubic-space support. Starting from the original 3-D random field, eight prediction error fields are generated at each lattice stage. The relationships between the prediction errors of successive lattice stage is defined by seven reflection coefficients. Besides the basic theory, an algorithm for determining the 3-D cubic-space recursive transfer function from the reflection coefficients is presented. A synthesis model for regenerating the original random field from a white noise field and the sufficient lattice parameter stability conditions are also included. In order to verify the theory, two numerical examples on modelling of random fields are given. Finally, an interframe predictive coding system based on the 3-D lattice model is presented to demonstrate the usefulness of the formulation. Results indicate that its performance is almost the same as that of the conventional DPCM system.