Block-based Motion Estimation Research Articles

A Complexity and Quality Evaluation of Block Based Motion Estimation Algorithms

Motion estimation is a method, by which temporal redundancies are reduced, which is an important aspect of video compression algorithms. In this paper we present a comparison among some of the well-known block based motion estimation algorithms. A performance evaluation of these algorithms is proposed to decide the best algorithm from the point of view of complexity and quality for noise-free video sequences and also for noisy video sequences.

Fast Block-Based Motion Estimation Using Integral Frames

Block-based motion estimation is widely used in video compression for exploitation of video temporal redundancy. Although effective, the process is arguably the most computationally intensive part of a typical video encoder. To speed up the process, a large number of fast block-matching algorithms (BMAs) have been proposed for motion estimation by limiting the number of search locations or simplifying the measure of match between two blocks under comparison. In this paper we propose a new BMA that measures the match by using the block sum of pixel values, a quantity that can be easily computed from an integral frame. Experimental results show that the proposed BMA can reduce notably the computational load and achieve compression performance very close to that of existing BMAs using conventional block-matching measures.

Motion Estimation Using Spatio-Temporal Contextual Information

Motion estimation is an important and computationally intensive task in video coding applications. Block-matching-based fast algorithms reduce the computational complexity of motion estimation at the expense of accuracy. An analysis of computation and performance trade-offs in motion estimation algorithms helps in choosing a suitable algorithm for video/visual communication applications. Fast motion estimation algorithms often assume a monotonic error surface in order to speed up the computations involved in motion estimation. The argument against this assumption is that the search might be trapped in local minima resulting in inaccurate motion estimates. The paper investigates state-of-the-art techniques for block-based motion estimation and presents an approach to improving the performance of block-based motion estimation algorithms. Specifically, the paper suggests a simple scheme that includes spatio-temporal neighborhood information for obtaining better estimates of the motion vectors. The paper also investigates the effects of the monotonic error surface assumption and suggests a remedy to reduce its impact on the motion field estimates. The presented experiments demonstrate the efficiency of the proposed approach.

Segmentation of the Face and Hands in Sign Language Video Sequences Using Color and Motion Cues

We present a hand and face segmentation methodology using color and motion cues for the content-based representation of sign language video sequences. The methodology consists of three stages: skin-color segmentation; change detection; face and hand segmentation mask generation. In skin-color segmentation, a universal color-model is derived and image pixels are classified as skin or nonskin based on their Mahalanobis distance. We derive a segmentation threshold for the classifier. The aim of change detection is to localize moving objects in a video sequences. The change detection technique is based on the F test and block-based motion estimation. Finally, the results from skin-color segmentation and change detection are analyzed to segment the face and hands. The performance of the algorithm is illustrated by simulations carried out on standard test sequences.

Motion compensated frame interpolation by new block-based motion estimation algorithm

In this paper, a motion compensated frame interpolation (MCI) algorithm adapting new block-based motion estimation (BME), which is overlapped block-based motion estimation (OBME), is proposed for frame rate up-conversion (FRC). Unlike conventional BME algorithms, where a video frame is divided into many non-overlapping square blocks of pixels, the proposed BME is executed using an overlapped matching block in order to get more accurate motion trajectory. To reduce computational complexity caused by the OBME, a sub-sampled pixel block is used for the OBME. The proposed OBME is executed with the various overlapped blocks that have different block sizes and sub-sampling ratios. In this paper, instead of mean absolute difference (MAD) used for general BME, a modified MAD is applied for the OBME. The MAD proposed is weighted using the magnitude of MV that corresponds to the estimating position. From the simulation results, the proposed methods show better performance in estimating "true motion" and reducing blocking artifacts with less complexity in calculation compared to a conventional method.

A new adaptive search strategy for fast block based motion estimation algorithms

Block based motion estimation algorithms are used for video sequence compression, among which full search algorithm is the most widely used because it provides the best match in the search area but it is computationally intensive. Hence many fast block search algorithms are proposed. All fast block search algorithm (FBSA) are based on the assumption that the block distortion measure decreases monotonically as the search point moves towards the global minimum. Hence the search becomes inefficient if the complexity of the error surface increases (i.e., for multimodal or rough surface) which in turn increases the error. In this paper, we propose a new strategy which makes this FBSA adaptive for multimodal surface by taking the relative difference between the local minima and other search points and selecting more than one winner for the next step, depending upon the distortion among the search points.

Temporal video segmentation and classification of edit effects

The process of shot break detection is a fundamental component in automatic video indexing, editing and archiving. This paper introduces a novel approach to the detection and classification of shot transitions in video sequences including cuts, fades and dissolves. It uses the average inter-frame correlation coefficient and block-based motion estimation to track image blocks through the video sequence and to distinguish changes caused by shot transitions from those caused by camera and object motion. We present a number of experiments in which we achieve better results compared with two established techniques.

Motion estimation using a frame-based adaptive thresholding approach

This paper proposes two frame-based adaptive thresholding algorithms for reducing the amount of computation involved in block-based motion estimation in the real-time and low-rate video coders. The proposed algorithms use an adaptive threshold for each of the frames to prejudge if each of the block matching computation is worthwhile. Based on the difference between the current and reference frames, the first algorithm determines a threshold in terms of the optimal trade-off between run-time and distortion, and the second algorithm determines a threshold according to a user specified percentage of total number of blocks. The experimental results demonstrate that the proposed algorithms can significantly reduce the amount of computation compared to the previous algorithms, while almost fully maintaining the quality of the reconstructed image.

Motion compensation using backward prediction and prediction refinement

This paper presents new methods for use of dense motion fields for motion compensation of interlaced video. The motion estimation is based on previously decoded field-images. The motion is then temporally predicted and used for motion compensated prediction of the field-image to be coded. The motion estimation algorithm is phase-based and uses two or three field-images to achieve motion estimates with sub-pixel accuracy. To handle non-constant motion and the specific characteristics of the field-image to be coded, the initially predicted image is refined using forward motion compensation, based on block-matching. Tests show that this approach achieves higher PSNR than forward block-based motion estimation, when coding the residual with the same coder. The subjective performance is also better.

Motion compensated film restoration

Motion picture films are susceptible to local degradations such as dust spots. Other deteriorations are global such as intensity and spatial jitter. It is obvious that motion needs to be compensated for before the detection/correction of such local and dynamic defects. Therefore, we propose a hierarchical motion estimation method ideally suited for high resolution film sequences. This recursive block-based motion estimator relies on an adaptive search strategy and Radon projections to improve processing speed. The localization of dust particles then becomes straightforward. Thus, it is achieved by simple inter-frame differences between the current image and motion compensated successive and preceding frames. However, the detection of spatial and intensity jitter requires a specific process taking advantage of the high temporal correlation in the image sequence. In this paper, we present our motion compensation-based algorithms for removing dust spots, spatial and intensity jitter in degraded motion pictures. Experimental results are presented showing the usefulness of our motion estimator for film restoration at reasonable computational costs.

Fast motion estimation of arbitrarily shaped video objects in MPEG-4

Block-based motion estimation is widely used for exploiting temporal redundancies in arbitrarily shaped video objects, which is computationally the most demanding part within the MPEG-4 standard. One of the main differences between MPEG-4 video and the previously standardized video coding schemes is the support of arbitrarily shaped video objects, for which most of the existing fast motion estimation algorithms are not suitable. In MPEG-4, motion estimation is performed on two kinds of macroblocks (MBs): the boundary MB and the opaque MB. Since the motion activities in opaque MBs are highly correlated with the neighbouring boundary MBs, a new priority search algorithm (PSA) for motion estimation is proposed in this paper, which performs motion estimation on all boundary MBs first in contrast to the conventional raster-scanning approach. This search strategy works well if the motion vectors in the boundary MBs truly represent the moving video object. Consequently, the full search algorithm is applied to the boundary MB in order to ensure its accuracy. However, the computational burden of motion estimation of the boundary MBs must be reduced. Fast search algorithms offered in the past tend to reduce the amount of computation by limiting the number of locations to be searched. Nearly all of these algorithms assume that the distortion function increases monotonically as the search location moves away from the global minimum. Unfortunately, this is usually not true in boundary MBs. We can reasonably assume that it is monotonic in a small neighbourhood around the global minimum. Consequently, one simple strategy, but perhaps the most efficient and reliable one, is to place the checking point as close as possible to the global minimum. In this paper, we also propose a fast search algorithm, which incorporates the binary alpha-plane to predict accurately the motion vectors of boundary MBs such that these motion vectors can be used in the PSA. Experimental results show that, when compared to conventional methods, our approach requires low computational complexity and provides significant improvement in terms of accuracy in motion-compensated video object planes.

Enhancement of coded video sequences via an adaptive nonlinear post-processing

Abstract Most video data compression standards use block-based motion estimation and the discrete cosine transform. This scheme takes advantage of the local spatial correlation property of images. However, it induces the well-known blocking artifacts, corner outliers, and ringing noise when the image is highly compressed. Conventional 2-D techniques for contrast sharpening cannot be applied to the enhancement of block-coded image sequences, because they emphasize such artifacts that become very visible and annoying in an image sequence. The local spectra and bandwidth of both the noise and the signal vary spatially, and the characteristics of the filters need to be locally adapted. For this purpose, in this paper we present a computationally simple post-filtering method that enhances the true details, limits the overshoot near sharp edges and attenuates the coding artifacts. Moreover, a mechanism is proposed that allows the behaviour of the filter to change according to the presence and the intensity of the coding artifacts. To this purpose, we also introduce a new distortion measure for blocking and ringing artifacts.

Fast motion estimation with search-center prediction

This paper presents a new block-based motion estimation al- gorithm that employs motion-vector prediction to locate an initial search point, which is called a search center, and an outward spiral search pattern with motion-vector refinement, to speed up the motion estimation process. It is found that the proposed algorithm is only slightly slower than cross search, but has a peak signal-to-noise ratio (PSNR) very close to that of full search (FS). Our research shows the motion vector of a target block can be predicted from the motion vectors of its neighboring blocks. The predicted motion vector can be used to locate a search center in the search window. This approach has two distinct merits. First, as the search center is closer to the optimum motion vector, the possi- bility of finding it is substantially higher. Second, it takes many less search points to achieve this. Results show that the proposed algorithm can achieve 99.7% to 100% of the average PSNR of FS, while it only requires 1.40% to 4.07% of the computation time of FS. When compared with six other fast motion estimation algorithms, it offers the best trade- off between two objective measures: average PSNR and search time. © 2001 Society of Photo-Optical Instrumentation Engineers. (DOI: 10.1117/1.1367865) Subject terms: block matching; motion estimation; search center; motion vector prediction; search pattern.

An efficient low bit-rate video-coding algorithm focusing on moving regions

Block-based motion estimation and compensation are the most popular techniques for video coding. However, as the shape and the structure of an object in a picture are arbitrary, the performance of such conventional block-based methods may not be satisfactory. In this paper, a very low bit-rate video coding algorithm that focuses on moving regions is proposed. The aim is to improve the coding performance, which gives better subjective and objective quality than that of the conventional coding methods at the same bit rate. Eight patterns are pre-defined to approximate the moving regions in a macroblock. The patterns are then used for motion estimation and compensation to reduce the prediction errors. Furthermore, in order to increase the compression performance, the residual errors of a macroblock are rearranged into a block with no significant increase of high-order DCT coefficients. As a result, both the prediction efficiency and the compression efficiency are improved.

Motion estimation using low-bit oriented edge image for DCT-based video compression

A novel block-based motion estimation algorithm using a low-bit resolution oriented edge image (LROEI) is proposed. The algorithm produces difference blocks with little texture for DCT coding and enhances the encoding efficiency. Simulation based on a hierarchical subsampling telescope search confirms that the quality of the LROEI is superior to that of luminance image compared with HSTS using an 8 bit luminance image. The operational loads are also reduced to 75 or 50% without quality degradation.

Hybrid algorithm for estimating true motion fields

A hybrid algorithm is proposed for true motion field estimation. This algorithm consists of three steps: block-based initial motion estimation, image segmentation, and wrong-motion-vector detection and correction based on objects. The hierarchical block-matching algorithms are improved for the initial motion estimation. The improved algorithm uses an adaptive technique to propagate motion vectors between hierarchical levels. It produces accurate motion field everywhere, except in the areas of motion occlusion. In order to correct wrong motion vectors in the areas of motion occlusion, the current image is segmented into objects and an object-based method is proposed to process the estimated motion fields. With the object-based method, wrong-motion vectors are detected by approximating the estimated motion field in each object with a motion model, and are corrected using an object-adaptive interpolator. Experimental results show that the improved hierarchical block-matching algorithm outperforms the conventional hierarchical block-matching algorithms. The proposed algorithm results in dense motion fields that are smooth within every object, discontinuous between objects of different motion, and very close to the true motion fields.

Vector-tracing algorithms for motion estimation in large search windows

Despite several efforts aimed at reducing the complexity, block-based motion estimation remains the most computationally demanding stage of video compression algorithms. This is particularly evident when sequences contain large displacements; very large search windows are needed to achieve high-quality coding when such critical conditions occur. This paper presents a new block-motion-estimation algorithm based on the systematic application of the tracing of motion trajectories on a modified genetic search heuristic. The described technique is able to provide motion estimates in very large search windows with optimal coding results. The complexity-reduction factor ranges up to more than two orders of magnitude. The technique can be applied to any macroblock-based video compression standard and to any group of picture structure.

Rate-distortion optimal motion estimation algorithms for motion-compensated transform video coding

Motion estimation and compensation is widely used for exploiting temporal correlation within an image sequence. To find motion vectors that lead to high compression, most motion estimation approaches use a source distortion measure, such as mean-square error (MSE) or mean-absolute error (MAE), as a search criterion. When incorporated into a closed-loop motion compensated (MC) transform video coder, these schemes produce noisy motion fields which significantly increase the bit-rates required to represent motion vectors. In view of this problem, this paper presents a rate-distortion optimal motion estimation algorithm. The proposed scheme improves rate performance of the estimated motion field while maintaining the peak signal-to-noise ratio (PSNR) prediction quality of the distortion-based methods, thereby enabling an efficient bit allocation between motion information and transform-coded prediction residuals. For coders in which motion vectors are differentially encoded, the rate-distortion optimization process is formulated as a shortest-path-finding problem. Adopting this framework, we show that the optimal solution for the conventional block-based motion estimation, followed by one-dimensional (1-D) differential coding and Huffman coding, can be obtained by using dynamic programming or the Viterbi algorithm. We propose an effective fast algorithm that closely approximates the optimal performance while requiring considerably less complexity. Our experimental results demonstrate overall gains in the range of 0.3-1.5 dB.

An efficient computation-constrained block-based motion estimation algorithm for low bit rate video coding

We present an efficient computation constrained block-based motion vector estimation algorithm for low bit rate video coding that yields good tradeoffs between motion estimation distortion and number of computations. A reliable predictor determines the search origin, localizing the search process. An efficient search pattern exploits structural constraints within the motion field. A flexible cost measure used to terminate the search allows simultaneous control of the motion estimation distortion and the computational cost. Experimental results demonstrate the viability of the proposed algorithm in low bit rate video coding applications. The resulting low bit rate video encoder yields essentially the same levels of rate-distortion performance and subjective quality achieved by the UBC H.263+ video coding reference software. However, the proposed motion estimation algorithm provides substantially higher encoding speed as well as graceful computational degradation capabilities.

A fast matching criterion for VLSI implementation of block-based motion estimation

The main bottleneck in the real-time implementation of block matching algorithms is their huge computational load. One of the ways of reducing the complexity is to use a simpler matching criterion than the mean-squared error criteria. In this paper, we propose a matching criterion which is computationally more efficient than the existing criterion. The proposed criterion is similar to the pel difference classification criterion but applied to a reduced number of bits as the reduced bit mean absolute difference criterion. From the simulation results on the displaced frame difference and the displacement vector field, it is expected that motion compensation with the proposed matching criterion will have similar rate-distortion performance as with other existing criteria.