Mesh-based Motion Estimation Research Articles

Active Mesh Based Motion Compensation Algorithm for Video Compression in Wavelet Sub Bands

The method of assessing the motion vectors is termed as Motion Estimation and consequently reconstructing the original frame using these motion vectors is termed as Motion Compensation. In this paper a new active mesh based Motion estimation and compensation algorithm in wavelet domain at subpixel level is proposed. To achieve the accuracy in estimating the motion vector from reference frame to predicted frame in sub pixel level a Wavelet based Counterlet transform is used. This new mesh based method is based by minimizing the mesh energy with energy functions as its input coefficients. The accuracy of the estimation and the compensation is improved by the feature points obtained by the energy functions. The analysis of the experimental results describes that there is improvement in the distortion of the performance rate for the video compression and this is achieved by the generation of the structured and the unstructured active mesh motion techniques in the wavelet sub-bands. The energy criteria of the input coefficients helps to retain the energy in the sub-bands which is a condition that helps in significant gaining of the PSNR required for better outcome.

DualEMC: energy efficient mobile multimedia communication with cloud

Video streaming has become one of the most popular networked applications and, with the increased bandwidth and computation power of mobile devices, anywhere and anytime streaming has become a reality. Unfortunately, it remains a challenging task to compress high-quality video in real-time in such devices given the excessive computation and energy demands of compression. On the other hand, transmitting the raw video is simply unaffordable from both energy and bandwidth perspective. In this paper, we propose DualEMC, a novel cloud-assisted video compression mechanism for mobile devices. DualEMC leverages the abundant cloud server resources for motion estimation (ME), which is known to be the most computation-intensive step in video compression, accounting for over 90 % of the computation time. With DualEMC, a mobile device selects and uploads only the key information of each picture frame to cloud servers for mesh-based ME, eliminating most of the local computation operations. We develop smart algorithms to identify the key mesh nodes, resulting in minimum distortion and data volume for uploading. Our simulation results demonstrate that DualEMC saves almost 30 % energy for video compression and transmission.

Feature-based motion compensated interpolation for frame rate up-conversion

In this paper, we propose to use feature point matching and mesh-based motion estimation for frame rate up-conversion. Matched local feature points between successive frames in a video sequence are employed to constitute a discrete and sparse motion field. Afterwards, a proximity-constraint marking process regulates the raw correspondence set to a content/motion adaptive nodal set. These correspondences are organized by a Delaunay triangular mesh so as to generate a dense motion field through the affine transformation. The experimental results demonstrate that the proposed scheme outperforms three other popular methods in terms of both subjective and objective evaluations. They also suggest that the proposed method is more suitable for videos containing a complex motion field.

Motion estimation and compensation in the redundant-wavelet domain using triangle meshes

In this paper, a technique is presented that incorporates an irregular triangle mesh into wavelet-domain motion estimation and compensation using a shift-invariant redundant-wavelet transform. The main contribution of this work resides in a demonstration that triangle-mesh motion estimation and compensation can be deployed more effectively in the redundant-wavelet domain thanks to a simple correlation operator that is robust to the prediction residual, a noise-like signal that hinders spatial-domain gradient-based efforts to locate image edges which persist from frame to frame. In the proposed technique, motion compensation takes place through an affine transform mapping triangles from one frame to the next, while the motion-compensated residual is downsampled to a non-redundant form which is then coded using any wavelet-based still-image coder. Experimental results indicate that the proposed technique significantly outperforms not only the usual spatial-domain approach to mesh-based motion estimation and compensation, but also the emerging wavelet-based coders that employ traditional block-based structures, particularly for sequences with fast or complex motion.

An affine-based algorithm and SIMD architecture for video compression with low bit-rate applications

This paper presents a new affine-based algorithm and SIMD architecture for video compression with low bit rate applications. The proposed algorithm is used for mesh-based motion estimation and it is named mesh-based square-matching algorithm (MB-SMA). The MB-SMA is a simplified version of the hexagonal matching algorithm [1]. In this algorithm, right-angled triangular mesh is used to benefit from a multiplication free algorithm presented in [2] for computing the affine parameters. The proposed algorithm has lower computational cost than the hexagonal matching algorithm while it produces almost the same peak signal-to-noise ratio (PSNR) values. The MB-SMA outperforms the commonly used motion estimation algorithms in terms of computational cost, efficiency and video quality (i.e., PSNR). The MB-SMA is implemented using an SIMD architecture in which a large number of processing elements has been embedded with SRAM blocks to utilize the large internal memory bandwidth. The proposed architecture needs 26.9 ms to process one CIF video frame. Therefore, it can process 37 CIF frames/s. The proposed architecture has been prototyped using Taiwan Semiconductor Manufacturing Company (TSMC) 0.18-/spl mu/m CMOS technology and the embedded SRAMs have been generated using Virage Logic memory compiler.

A COMPUTATIONAL-RAM (C-RAM) ARCHITECTURE FOR REAL-TIME MESH-BASED VIDEO MOTION TRACKING PART 1: MOTION ESTIMATION

This paper presents a new Computational-RAM (C-RAM) architecture for real-time mesh-based video motion tracking. The motion tracking consists of two operations: mesh-based motion estimation and compensation. The proposed motion estimation architecture is presented in Part 1 and the proposed motion compensation architecture is presented in Part 2. The motion estimation architecture stores two frames and computes motion vectors for a regular triangular mesh structure as defined by MPEG-4 Part 2.1 The motion estimation architecture uses the block-matching algorithm (BMA) to estimate the vertical and horizontal motion vectors for each mesh node. Parallel and pipelined implementations have been used to overcome the huge computational requirements of the motion estimation process. The two frames are stored in embedded S-RAMs generated with Virage™ Memory Compiler. The proposed motion estimation architecture has been prototyped, simulated and synthesized using the TSMC 0.18 μm CMOS technology. At 100 MHz clock frequency, the proposed architecture processes one CIF video frame (i.e., 352×288 pixels) in 1.48 ms, which means it can process up to 675 frames per second. The core area of the proposed motion estimation architecture is 24.58 mm2 and it consumes 46.26 mW.

New kernels for fast mesh-based motion estimation

Mesh-based motion estimation-also known as control grid interpolation or warping-provides a smoother estimated intensity field compared to the traditional block-matching algorithm (BMA), resulting in most cases in a more realistic motion field and smaller estimation error. In mesh-based motion, unlike BMA, the computation of a motion vector is affected by its neighboring vectors. This interdependence necessitates a costly, iterative computation of motion vectors. The computational cost of mesh-based motion has been a main drawback of this otherwise powerful technique. We propose to use noniteratively computed motion vectors, such as BMA motion vectors, for node motions in the mesh model. However, we found that a straightforward insertion of BMA motion vectors in the deformable mesh leads to unpredictable and erratic results, and were thus motivated to carefully analyze the interaction of motion vectors and interpolation kernels in mesh models. This analysis leads to a methodology for computing optimal motion interpolation kernels for a given set of motion vectors (e.g., BMA motion vectors). We find a generalized orthogonality condition for these kernels; optimality is achieved only if the projections of vertex motions on the local intensity gradients are statistically orthogonal to mesh-based estimation errors. Experiments show that optimal kernels are often very different from the traditional bilinear kernels, and exhibit interesting variations. The new kernels benefit a variety of applications, including motion estimated interpolation, denoising, and compression.

Evaluation of mesh-based motion estimation in H.263-like coders

We present two mesh-based motion estimation algorithms, and evaluate their performance when incorporated in an H.263-like block-based video coder. Both algorithms compute nodal motions in a hierarchical manner. Within each hierarchy level, the first algorithm (HMMA) minimizes the prediction error in the four elements surrounding each node, where the prediction is accomplished by a bilinear mapping. The optimal solution is obtained by a full search within a range defined by the topology of the mesh. The second algorithm (HBMA) minimizes the error in a block surrounding each node, assuming the motion in the block is constant. In both cases, bilinear mapping is used for motion-compensated prediction based on nodal displacements. The two algorithms are compared with an exhaustive block-matching algorithm (EBMA) by evaluating their performance in temporal prediction and in an H.263/TMN4 coder. For prediction only, the HMMA and HBMA algorithms yield visually more satisfactory results, even though the PSNRs of the predicted images are on average lower. The coded images also have lower PSNRs at similar bit rates. The coding artifacts are different: while the block-based method leads to more severe block distortions, the mesh-based method experiences some warping artifacts. The HMMA algorithm outperforms the HBMA slightly for certain sequences at the expense of higher computational complexity.