Proposed Motion Estimation Algorithm Research Articles

Role Of Pattern Characteristics In Cross Correlation Based Motion Estimation

Objectives: To establish a pattern tracking based motion estimation algorithm for the stereovision based system and to investigate the effect of threshold value (Thv), size and population of patterns on the pattern tracking value. Methods: Proposed motion estimation algorithm correlates the set of motion frames captured from two high speed cameras configured in stereovision system. The correlation scheme was based on grayscale pattern tracking in moving frames. Pattern development and correlation algorithms were developed. A spherical object was given small random displacements and the motion was captured using stereovision system. The effectiveness of algorithm is evaluated with the pattern tracking value which should be close to 1 for perfect pattern match. Findings: The correlation results indicate that pattern tracking value were found to be 0.920 and 0.899 for left and right cameras respectively when the threshold value (>10) and size 10 pixels are considered with high population (200 patterns). With the increase in pattern size, the pattern tracking value decreased. Another study revealed that pattern tracking value was comparatively higher when the pattern population was maximum (200 patterns). The pattern tracking value again decreased when the threshold value (>15) is considered. It was concluded that pattern size of 10 pixels with threshold value (>10) is more pronounced for motion estimation. The proposed algorithm is verified for the 3D displacements of a rectangular plate mounted on XYZ translation stage. The pattern tracking values were 0.97, 0.95, 0.96 and 0.96, 0.94, 0.95 for X, Y and Z displacements respectively. The correlation algorithm is also coupled with the compression technique using wavelet based data compression. Novelty/Applications: The proposed algorithm can be efficiently applied for both in plane and out of plane motion estimation. The algorithm can provide constructive outcomes for small motion prediction with proper selection of pattern size and threshold value. Keywords: Camera calibration; Image acquisition; Image correlation; Motion estimation; Pattern tracking; Compression

Motion-based camera localization system in colonoscopy videos

Optical colonoscopy is an essential diagnostic and prognostic tool for many gastrointestinal diseases, including cancer screening and staging, intestinal bleeding, diarrhea, abdominal symptom evaluation, and inflammatory bowel disease assessment. However, the evaluation, classification, and quantification of findings from colonoscopy are subject to inter-observer variation. Automated assessment of colonoscopy is of interest considering the subjectivity present in qualitative human interpretations of colonoscopy findings. Localization of the camera is essential to interpreting the meaning and context of findings for diseases evaluated by colonoscopy. In this study, we propose a camera localization system to estimate the relative location of the camera and classify the colon into anatomical segments. The camera localization system begins with non-informative frame detection and removal. Then a self-training end-to-end convolutional neural network is built to estimate the camera motion, where several strategies are proposed to improve its robustness and generalization on endoscopic videos. Using the estimated camera motion a camera trajectory can be derived and a relative location index calculated. Based on the estimated location index, anatomical colon segment classification is performed by constructing a colon template. The proposed motion estimation algorithm was evaluated on an external dataset containing the ground truth for camera pose. The experimental results show that the performance of the proposed method is superior to other published methods. The relative location index estimation and anatomical region classification were further validated using colonoscopy videos collected from routine clinical practice. This validation yielded an average accuracy in classification of 0.754, which is substantially higher than the performances obtained using location indices built from other methods.

Longitudinal-Scanline-Based Arterial Traffic Video Analytics with Coordinate Transformation Assisted by 3D Infrastructure Data

High-resolution vehicle trajectory data can be used to generate a wide range of performance measures and facilitate many smart mobility applications for traffic operations and management. In this paper, a Longitudinal Scanline LiDAR-Camera model is explored for trajectory extraction at urban arterial intersections. The proposed model can efficiently detect vehicle trajectories under the complex, noisy conditions (e.g., hanging cables, lane markings, crossing traffic) typical of an arterial intersection environment. Traces within video footage are then converted into trajectories in world coordinates by matching a video image with a 3D LiDAR (Light Detection and Ranging) model through key infrastructure points. Using 3D LiDAR data will significantly improve the camera calibration process for real-world trajectory extraction. The pan-tilt-zoom effects of the traffic camera can be handled automatically by a proposed motion estimation algorithm. The results demonstrate the potential of integrating longitudinal-scanline-based vehicle trajectory detection and the 3D LiDAR point cloud to provide lane-by-lane high-resolution trajectory data. The resulting system has the potential to become a low-cost but reliable measure for future smart mobility systems.

Complexity reduction for HEVC encoder using one-dimensional filtering based constrained one-bit transform

This paper presents a one-dimensional-filtering-based constrained one-bit transformation (1D-C1BT)-based motion estimation to reduce the complexity of the High Efficiency Video Coding (HEVC) encoder. The proposed 1D-C1BT requires integer arithmetic additions, subtraction and shift operations to convert full resolution video frames to two bit depth 1D-C1BT frames. By embedding the 1D-C1BT in the HEVC encoder, the motion estimation (ME) complexity of the HEVC encoder is greatly reduced in terms of Motion Estimation Time (MET) and total Encoding Time (ET) in uni-prediction and bi-prediction respectively. From the experimental results, it is shown that the 1D-C1BT accompanying full search (FS) and test zonal search (TZS) algorithms greatly reduces MET and ET as compared to the FS and TZS algorithms of the HEVC encoder respectively. To further reduce the intricacy of the HEVC encoder in the fast search mode, star-diamond (SD) search accompanying 1D-C1BT is proposed. The 1D-C1BT accompanying SD further reduces MET and ET with small increment in Bjontegaard Delta Rate and small decrement in Bjontegaard Delta Peak Signal to Noise Ratio, compared to the TZS algorithm of the HEVC encoder. In the FS mode of the HEVC encoder, the proposed 1D-C1BT, accompanying FS, reduces MET by 86.00% and ET by 83.94%, with BD-Rate of 0.1452% and BD-PSNR of 0.2635% in low delay-P main profile. The proposed ME algorithms, 1D-C1BT accompanying full search (FS + 1D-C1BT), 1D-C1BT accompanying test zonal search (TZS + 1D-C1BT) and 1D-C1BT accompanying star diamond search (SD + 1D-C1BT) greatly reduce ME complexity in all the three configurations (low delay-P, low delay-B and random access main profile) of the HEVC encoder.

VLSI Architecture of Block Matching Algorithms for Motion Estimation in High Efficiency Video Coding

High-efficiency video coding (HEVC) is a latest video coding standard and the motion estimation unit is the most important block. The work presents the different types of Matching Criteria for Block-Based Motion Estimation technique in HEVC standard. HEVC requires fast motion estimation algorithms to have better real time performance. The hardware implementation of motion estimation helps to achieve high speed though parallel processing. An improved block matching technique is designed with reduced blocks for HEVC. The proposed method has less execution time where only blocks having motion are compared for prediction computation. The searching time complexity is dependent on the number of blocks that are having motion. The searching time of frame having small motion can be reduced to 80–85% as compared to the traditional full search algorithm. In the paper, sum of absolute difference, mean square error and mean absolute difference are computed to find the best matching algorithm for HEVC. However, SAD has less computational complexity with compare to other matching criteria. The results suggest that proposed motion estimation algorithm has better performance with compare to similar previous work.

Motion estimation using maximum sub-image and sub-pixel phase correlation on a DSP platform

Motion estimation is a key step for many video process systems, but it usually suffers from low precision in complex imaging scenarios, as well as the problem of real-time processing. This paper proposes an accurate and fast motion estimation algorithm using maximum sub-image and sub-pixel phase correlation implemented on a DSP platform. Firstly, a maximum sub-image is extracted to contain the image content as much as possible, which both satisfies the requirement of Fourier precondition and the wide coverage of complete background. And then, the extracted sub-image is down-sampled with median filter to increase the signal-noise-ratio and decrease the computation load. Secondly, a sub-pixel motion estimation is used to compensate the losing precision due to down-sampling, and keep the range of motion estimation. Finally, the proposed motion estimation algorithm is implemented on a single core TMS320C6678 DSP platform, and it is accelerated by applying multistage data cache and advanced data access. Experiments demonstrate the accuracy of the proposed motion estimation algorithm in complex scenarios. Meanwhile, it can achieve 7.4 ms/frame for sub-image with 512 × 512 pixels size and 29.6 ms/frame for sub-image with 1024 × 1024 pixels size, respectively.

Low Power Motion Estimation Algorithm and Architecture of HEVC/H.265 for Consumer Applications

High-quality videos like high-definition (HD) and ultra HD became an essential requirement in recent applications such as security surveillance, television system, etc. However, due to increase in resolution of the videos, the volume of visual information data increases significantly, which became a challenge for storage, transmission and processing the HD video data. The new video compression standard, high efficient video coding (HEVC), achieved two-fold video efficiency improvements as compared to H.264/AVC using efficient compression techniques. Motion estimation (ME) is one of the computationally intensive blocks in video CODEC. In HEVC, the complexity of ME further increases due to a large processing unit and flexible partitioning of the prediction unit (PU). In this paper, we proposed a low power ME algorithm and architecture of the HEVC for consumer applications. The proposed algorithm and architecture utilizes sub-sampling, data reuse, pixel truncation and adaptive search range techniques for reducing the computational power. Simulations result shows that the proposed ME algorithm requires an average of 53.82% fewer search points as compared to the reference software HM with a small degradation in PSNR and little increment in bit-rate. The proposed architecture is simulated and synthesized using standard 90 nm technology. The proposed ME architecture can process $3840\times2160$ @ 30 fps video sequences with only 4.5193 mm2 of the area and 8.192 KB of SRAM. The operating frequency of the proposed architecture is 250 MHz with 151.7619 mW of power.

Direction-Select Motion Estimation for Motion-Compensated Frame Rate Up-Conversion

In this paper, we propose a new motion estimation algorithm to be used for motion-compensated frame rate up-conversion. The proposed algorithm independently carries out motion estimations in both forward and backward directions, and selects a more reliable one between forward and backward motion vectors by evaluating the motion vector reliability from the viewpoint of the interpolated frame. The proposed algorithm smooths and refines both the forward and backward motion vectors before selecting the reliable one. This procedure helps to select the reasonable motion estimation direction. In identifying the motion vector outliers, the proposed algorithm uses a circular range of which center is located at the mean of the eight neighboring motion vectors of the motion vector being processed. Experimental results using 1720 test images show that the proposed motion estimation algorithm improves the average peak signal-to-noise ratio and the average structural similarity of the interpolated frames by up to 5.31 dB and 0.053, respectively, compared to conventional motion estimation algorithms.

H.264/AVC inter prediction for heterogeneous computing systems

H.264/AVC is the latest standard for video compression and is a significant advance, but at the expense of increasing computing needs. Recently, the progress of GPUs has attracted considerable attention because they are able to offer practical and acceptable solutions for speeding up graphic and non-graphic applications. In this paper, we present an implementation of H.264/AVC Motion Estimation running on an NVIDIA GTX285 using CUDA. The algorithm is divided into three steps, all of which need to be executed sequentially but each one is exploited following a highly parallel procedure by using the GPU. The execution time of the proposed motion estimation algorithm is 53 times faster and it reduces the energy consumption by a factor of 9 compared with the JM reference encoder using a single CPU core.

Video Compression USING a New Active Mesh Based Motion Compensation Algorithm in Wavelet Sub-Bands

In this paper, a new mesh based algorithm is applied for motion estimation and compensation in the wavelet domain. The first major contribution of this work is the introduction of a new active mesh based method for motion estimation and compensation. The proposed algorithm is based on the mesh energy minimization with novel sets of energy functions. The proposed energy functions have appropriate features, which improve the accuracy of motion estimation and compensation algorithm. We employ the proposed motion estimation algorithm in two different manners for video compression. In the first approach, the proposed algorithm is employed for motion estimation of consecutive frames. In the second approach, the algorithm is applied for motion estimation and compensation in the wavelet sub-bands. The experimental results reveal that the incorporation of active mesh based motion-compensated temporal filtering into wavelet sub-bands significantly improves the distortion performance rate of the video compression. We also use a new wavelet coder for the coding of the 3D volume of coefficients based on the retained energy criteria. This coder gives the maximum retained energy in all sub-bands. The proposed algorithm was tested with some video sequences and the results showed that the use of the proposed active mesh method for motion compensation and its implementation in sub-bands yields significant improvement in PSNR performance.

Power efficient motion estimation algorithm and architecture based on pixel truncation

A new block matching algorithm and its VLSI architecture for performing Motion Estimation (ME) are presented in this paper. In the reported fast two stage search algorithm, ME is performed in two stages. In the first stage, pixel truncation is used. In the second stage, ME is performed with full pixel resolution with an adaptive search pattern. The main advantage of the proposed algorithm is the inclusion of an early termination mechanism to reduce overall power consumption for the resulting architecture. The paper also introduces a suitable architecture to implement the proposed ME algorithm. In this architecture, a new memory management scheme has been proposed so that different bit planes can be accessed at different stages of ME from the same memory module. It has been shown that the proposed architecture can save approximately 27% power compared to another recently reported architecture. The proposed architecture can therefore be considered suitable for portable, battery-powered video consumer devices <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> .

An adaptive true motion estimation algorithm for frame rate conversion of high definition video and its hardware implementations

Frame Rate Up-Conversion (FRUC) is required for displaying low frame rate video signals on high frame rate flat panel displays. In this paper, we propose an adaptive true Motion Estimation (ME) algorithm for FRUC of High Definition (HD) video. The proposed ME algorithm produces similar quality results with less number of calculations or higher quality results with similar number of calculations compared to 3D Recursive Search true ME algorithm by adaptively using optimized sets of candidate search locations and several computational complexity reduction techniques. We also propose two different complexity hardware architectures for implementing this ME algorithm. The second hardware uses an efficient data re-use technique for reducing the number of off-chip memory accesses. We implemented both hardware architectures in Verilog HDL. The hardware implementations are capable of processing 158 and 168 720p HD frames per second on average respectively. Therefore, they can be used in consumer electronics products that require real-time FRUC of HD video.

Dual Motion Estimation for Frame Rate Up-Conversion

In this letter, we present a new motion estimation algorithm for frame rate up-conversion. The proposed dual motion estimation algorithm enhances the estimation accuracy of motion vectors by using the unidirectional and bidirectional matching ratios of blocks in the previous and current frames. In addition, the proposed motion estimation approach uses motion vector validity to evaluate the accuracy of motion vectors thereby avoiding false motion vectors. In experiments using benchmark image sequences, the proposed motion estimation algorithm improved the average peak signal-to-noise ratio of interpolated frames by up to 2.272 dB, when compared to conventional motion estimation algorithms. For the comparison of the perceptual image quality using the structural similarity, the average value of the proposed dual motion estimation was by up to 0.062 higher than those of the conventional algorithms.

Activity-Based Motion Estimation Scheme for H.264 Scalable Video Coding

This paper proposes a motion estimation scheme to reduce the computational complexity of multilayer motion estimation for scalable video coding. Based on the result of the motion estimation of the lower resolution layer referred to as base layer, we developed a new approach for exploring the search range of the enhancement layer with high coding efficiency. This approach is based on the activity defined as the absolute difference between the motion vector predictor and the final motion vector. Based on the correlation of the activities between neighboring layers, an inter-layer activity model was developed using a curve-fitted linear equation to exploit the activity in the base layer for deciding the search center and the search range of the enhancement layer. Each activity pair in the neighboring layers is used to associate the relevant macroblock to one of two groups; boundary region and interior region. The base-layer motion vector predictor is basically selected over all the activity regions; for each activity region, the proposed motion estimation algorithm decides whether to include the median motion vector predictor or not. Minimal sufficient search range is also decided from the inter-layer activity prediction factor that is adjusted to the given sequence. The proposed scheme reduced the execution time of motion estimation by 99.26% at the cost of 1.56% bit-rate increase and 0.048 dB peak signal-to-noise ratio (PSNR) decrease on average compared with the conventional full-search algorithm. The fast full-search block matching algorithm can also be incorporated to obtain the extra CPU time reduction in the motion estimation process. By adopting the fast full-search block matching algorithm (FFSBMA) in JSVM reference software, the CPU time was reduced by up to 91.84% and the memory bandwidth was reduced by 90% at the sacrifice of 1.27% bit-rate increase and 0.041 dB PSNR decrease on average compared with the FFSBMA only.

An Efficient Frame Rate Up-Conversion Method for Mobile Phone with Projection Functionality

Mobile phones with the built-in projection functionality are about to be introduced in market. In order to achieve high visual quality in mobile projection, an efficient frame rate up-conversion is proposed in this paper. Computational complexity and size of memoty resources are important factors in mobile applications. The proposed motion estimation algorithm utilizes adaptive block size and search window according to motion activity levels. In order to improve accuracy of the estimated motion vectors, the two stage motion vector refinement technique is proposed. Also, the proposed method is hardware implemented. Experimental results indicate that the proposed technique exhibits better performance compared to the conventional methods .

Adaptive motion estimation schemes using maximum mutual information criterion

AbstractWe consider the motion estimation problem in video coding. In our previous work, we proposed a new motion estimation method where motion estimation is formulated as an optimization problem and an adaptive system under the minimum error entropy (MEE) criterion is used for motion estimation. In this paper, we develop an adaptive system under the criterion of maximum mutual information to address the motion estimation problem. Our proposed motion estimation algorithms have very low encoding complexity and hence are ideally suited for wireless video sensor networks where limited bandwidth, restricted computational capability, and limited battery power supply impose stringent constraints on the video encoding system. Copyright © 2007 John Wiley &amp; Sons, Ltd.

An efficient scheme for motion estimation using multireference frames in H.264/AVC

The multiple reference frame motion compensation (MRMC) supported by H.264 makes use of the redundancy which is between multiple frames to enhance the coding efficiency over a scheme using the single reference frame motion compensation (SRMC) in which motion vectors are searched over a single reference frame. And, the technique using multiple reference frames can combat the channel errors efficiently. However, searching the motion vectors in multiple frames may require a huge computing time. This paper proposes a novel motion estimation procedure, which has a lower search complexity without sacrificing image quality. To reduce the complexity of motion estimation procedure, we use a temporary motion vector generated with little computation. The temporary motion vector is calculated from the motion vector map composed of motion vectors between successive frames, and used to predict the optimal motion vector for a reference frame. The proposed scheme requires the lower complexity than conventional schemes by using the temporary motion vector and refinement process over a narrow search range around the temporary predictive motion vector. Since the temporary predictive motion vector effectively chases the optimal motion vector for each reference frame, the encoded image quality by proposed scheme is very similar to that of full search algorithm. The proposed motion estimation process consists of three phases: 1)making a vector map between two consecutive frames, where the vector map is constructed by copying motion vectors which have been estimated in first reference frame, 2) composing a temporary motion vector with element vectors which are in the vector map, and 3) finally, the temporary predictive motion vector is refined over a narrow search range. We show experimental results which demonstrate the effectiveness of the proposed method. To compare the proposed motion estimation algorithm with the conventional schemes, we check the CPU times consumed by ME module in H.264 encoder using the proposed scheme. In the results, CPU time consumed by the proposed scheme has been reduced significantly without additional distortion of the encoded video quality.

Motion Estimation and Signaling Techniques for 2D+t Scalable Video Coding

We describe a fully scalable wavelet-based 2D+t (in-band) video coding architecture. We propose new coding tools specifically designed for this framework aimed at two goals: reduce the computational complexity at the encoder without sacrificing compression; improve the coding efficiency, especially at low bitrates. To this end, we focus our attention on motion estimation and motion vector encoding. We propose a fast motion estimation algorithm that works in the wavelet domain and exploits the geometrical properties of the wavelet subbands. We show that the computational complexity grows linearly with the size of the search window, yet approaching the performance of a full search strategy. We extend the proposed motion estimation algorithm to work with blocks of variable sizes, in order to better capture localmotion characteristics, thus improving in terms of rate-distortion behavior. Given this motion field representation, we propose a motion vector coding algorithm that allows to adaptively scale the motion bit budget according to the target bitrate, improving the coding efficiency at low bitrates. Finally, we show how to optimally scale the motion field when the sequence is decoded at reduced spatial resolution. Experimental results illustrate the advantages of each individual coding tool presented in this paper. Based on these simulations, we define the best configuration of coding parameters and we compare the proposed codec with MC-EZBC, a widely used reference codec implementing the t+2D framework.

Low bit rate wavelet video codec using edge based motion estimation and context-based coding

This paper presents a new wavelet-based video codec for very low bit rate coding, where each frame of the test sequence is transformed using discrete wavelet transformation. The hierarchical edge based motion estimation is applied between adjacent transformed frames. The aim is to reduce the computational complexity of a motion estimation algorithm and to preserve coding efficiency. The edge detection is incorporated into the motion estimation process in order to determine the moving regions. The edge is assumed as a point between two different moving areas in the proposed motion estimation algorithm. All transformed frames are quantized and prediction errors are coded using quantization and probability models designed with fuzzy logic. This proposed method is comparable with the H.26L video coding standard in the rate distortion sense, and has lower computational complexity than the H.26L standard.

An efficient computation-constrained block-based motion estimation algorithm for low bit rate video coding

We present an efficient computation constrained block-based motion vector estimation algorithm for low bit rate video coding that yields good tradeoffs between motion estimation distortion and number of computations. A reliable predictor determines the search origin, localizing the search process. An efficient search pattern exploits structural constraints within the motion field. A flexible cost measure used to terminate the search allows simultaneous control of the motion estimation distortion and the computational cost. Experimental results demonstrate the viability of the proposed algorithm in low bit rate video coding applications. The resulting low bit rate video encoder yields essentially the same levels of rate-distortion performance and subjective quality achieved by the UBC H.263+ video coding reference software. However, the proposed motion estimation algorithm provides substantially higher encoding speed as well as graceful computational degradation capabilities.