Motion Estimation Technique Research Articles

A comprehensive comparison between shortest-path HARP refinement, SinMod, and DENSEanalysis processing tools applied to CSPAMM and DENSE images

We addressed comprehensively the performance of Shortest-Path HARP Refinement (SP-HR), SinMod, and DENSEanalysis using 2D slices of synthetic CSPAMM and DENSE images with realistic contrasts obtained from 3D phantoms. The three motion estimation techniques were interrogated under ideal and no-ideal conditions (with MR induced artifacts, noise, and through-plane motion), considering several resolutions and noise levels. Under noisy conditions, and for isotropic pixel sizes of 1.5 mm and 3.0 mm in CSPAMM and DENSE images respectively, the nRMSE obtained for the circumferential and radial strain components were 10.7 ± 10.8% and 25.5 ± 14.8% using SP-HR, 11.9 ± 2.5% and 29.3 ± 6.5% using SinMod, and 6.4 ± 2.0% and 18.2 ± 4.6% using DENSEanalysis. Overall, the results showed that SP-HR tends to fail for large tissue motions, whereas SinMod and DENSEanalysis gave accurate displacement and strain field estimations, being the last which performed the best.

A Respiratory Motion Estimation Method Based on Inertial Measurement Units for Gated Positron Emission Tomography.

We present a novel method for estimating respiratory motion using inertial measurement units (IMUs) based on microelectromechanical systems (MEMS) technology. As an application of the method we consider the amplitude gating of positron emission tomography (PET) imaging, and compare the method against a clinically used respiration motion estimation technique. The presented method can be used to detect respiratory cycles and estimate their lengths with state-of-the-art accuracy when compared to other IMU-based methods, and is the first based on commercial MEMS devices, which can estimate quantitatively both the magnitude and the phase of respiratory motion from the abdomen and chest regions. For the considered test group consisting of eight subjects with acute myocardial infarction, our method achieved the absolute breathing rate error per minute of 0.44 ± 0.23 1/min, and the absolute amplitude error of 0.24 ± 0.09 cm, when compared to the clinically used respiratory motion estimation technique. The presented method could be used to simplify the logistics related to respiratory motion estimation in PET imaging studies, and also to enable multi-position motion measurements for advanced organ motion estimation.

Study and implementation of K-multiple constraint shortest path for H.265 HEVC for optimal video compression

Today, everyone is demanding high-quality video services along with large bandwidth. Due to this demand, a ruthless burden is felt across the network architecture. Thus, a recently developed video codec H.265, also called high-efficiency video coding (HEVC), offers high coding efficiency than its predecessor H.264 Advanced Video Coding. Although H.265 or HEVC offers a greatly improved compression ratio, it is done at the expense of a high increase in computational complexity and time complexity to process the video compression. This disadvantage of HEVC makes it less suitable for real-time video sequences that may contain varying motion activities. Therefore, it is insufficient to use conventional motion estimation techniques that consume 80–90% of the total computational power. The research work presented in this paper deals with these two-short coming of H.265 HEVC. In this paper, first, an improved motion estimation technique is presented, which yields higher compression efficiency, computational complexities and better video quality, especially for low and medium video resolution applications such as handheld devices based videotelephony. Experimental results using improved motion estimation technique based on Quadtree decomposition and A* prune algorithm optimization reveals that total bits for different multi constraints shortest path get reduced while retaining the quality of video in reconstruction part. Besides, it can also be used in various multimedia applications with restricted computational resources.

A very fast edge map-based algorithm for accurate motion estimation

Motion estimation plays a very crucial role in many video applications. Hence, an accurate and fast algorithm is required in real-time applications. Although exhaustive block matching (EBM) algorithm is the best algorithm used for motion estimation in terms of performance, it is computationally very expensive. Several algorithms have been developed in the literature to reduce the computational time required by the EBM algorithm; however, their performance is not comparable to that of EBM. In this paper, we propose an extremely fast technique for motion estimation based on the edge map of only a fraction of the frame. Extensive experiments are carried out, and the results show that the proposed technique provides a performance comparable to that of the EBM algorithm, with a very low computational complexity.

Computer vision‐based real‐time cable tension estimation in Dubrovnik cable‐stayed bridge using moving handheld video camera

Cables are essential components of the cable-stayed bridges as they serve as the main load-bearing component. Hence, continuous monitoring of such cables becomes necessary as they are vulnerable to the fatigue damage induced by dynamic loads. Sensors are attached to the cables to examine the health of the cables; however, these contact-based sensors can malfunction in harsh weather condition, which makes impossible to estimate the cable health in such unfavorable condition. Therefore, in this paper, we propose a completely noncontact video-based stay-cable tension measurement technique where the video is recorded using a moving handheld camera at a significant distance from the structure itself. Here, the cable tension is determined from vibration-based measurement, but the vibration of the cable recorded in the video includes the true vibration of the cable along with the camera motion. Hence, we amalgamated a series of image processing techniques to nullify the camera movement. First, we detect the camera movement based on the movement of the bridge deck and pylon, which are fixed objects, using Kanade–Lucas–Tomasi (KLT) feature tracking algorithm. Then we nullify the camera movement by using the affine transformation matrix obtained by random sample consensus (RANSAC) algorithm. Subsequently from the steady video, the cable motions are estimated using the phase-based motion estimation technique. From the time history of the cable vibration, real-time frequency variations are estimated using Short-Time Fourier Transform (STFT). Finally, the real-time tension is determined from this dominant frequency variation history using the taut-string theory. This paper shows the significant potential of camera-based sensing techniques in structural health monitoring as the mean estimated tension and the design cable tension are found to be comparable.

CNN-Based Ultrasound Image Reconstruction for Ultrafast Displacement Tracking.

Thanks to its capability of acquiring full-view frames at multiple kilohertz, ultrafast ultrasound imaging unlocked the analysis of rapidly changing physical phenomena in the human body, with pioneering applications such as ultrasensitive flow imaging in the cardiovascular system or shear-wave elastography. The accuracy achievable with these motion estimation techniques is strongly contingent upon two contradictory requirements: a high quality of consecutive frames and a high frame rate. Indeed, the image quality can usually be improved by increasing the number of steered ultrafast acquisitions, but at the expense of a reduced frame rate and possible motion artifacts. To achieve accurate motion estimation at uncompromised frame rates and immune to motion artifacts, the proposed approach relies on single ultrafast acquisitions to reconstruct high-quality frames and on only two consecutive frames to obtain 2-D displacement estimates. To this end, we deployed a convolutional neural network-based image reconstruction method combined with a speckle tracking algorithm based on cross-correlation. Numerical and in vivo experiments, conducted in the context of plane-wave imaging, demonstrate that the proposed approach is capable of estimating displacements in regions where the presence of side lobe and grating lobe artifacts prevents any displacement estimation with a state-of-the-art technique that relies on conventional delay-and-sum beamforming. The proposed approach may therefore unlock the full potential of ultrafast ultrasound, in applications such as ultrasensitive cardiovascular motion and flow analysis or shear-wave elastography.

HARP-I: A Harmonic Phase Interpolation Method for the Estimation of Motion From Tagged MR Images.

We proposed a novel method called HARP-I, which enhances the estimation of motion from tagged Magnetic Resonance Imaging (MRI). The harmonic phase of the images is unwrapped and treated as noisy measurements of reference coordinates on a deformed domain, obtaining motion with high accuracy using Radial Basis Functions interpolations. Results were compared against Shortest Path HARP Refinement (SP-HR) and Sine-wave Modeling (SinMod), two harmonic image-based techniques for motion estimation from tagged images. HARP-I showed a favorable similarity with both methods under noise-free conditions, whereas a more robust performance was found in the presence of noise. Cardiac strain was better estimated using HARP-I at almost any motion level, giving strain maps with less artifacts. Additionally, HARP-I showed better temporal consistency as a new method was developed to fix phase jumps between frames. In conclusion, HARP-I showed to be a robust method for the estimation of motion and strain under ideal and non-ideal conditions.

Temporally Consistent Edge-Informed Video Super-Resolution (Edge-VSR)


 Resolution enhancement of a given video sequence is known as video super-resolution. We propose an end-to-end trainable video super-resolution method as an extension of the recently developed edge-informed single image super-resolution algorithm. A two-stage adversarial-based convolutional neural network that incorporates temporal information along with the current frame's structural information will be used. The edge information in each frame along with optical flow technique for motion estimation among frames will be applied. Promising results on validation datasets will be presented.

RETRACTED: Optimization of Motion Estimation Algorithm Based on FPGA Hardware System and Video Tracking

Motion Estimation (ME) is a computationally expensive step in video tracking. A thorough search technique produces the best execution time and the best accuracy. Computational burden, many fast search algorithms limit and report the number of places to search. Motion estimation, analysis of image sequences, computer vision, tracking a target, and is a critical process in a wide range of fields and applications such as video coding. Precision motion estimation lowers significantly bit rate, but can require a complexity of higher computational. This is different reference frames, sub-pixel estimation, and video compression standards, including techniques such as variable block size, MPEG, especially true for a new generation. Only the motion estimation in H.261 is limited to an integer pixel accuracy, rather than the pixel grid, not in the object is often located, move to move between pixels. The system proposed in the motion vector optimization motion estimation (OME) pixel accuracy. By estimating the amount of displacement at a finer resolution, improve prediction, it can expect better performance than motion estimation using an integer pixel precision. Optimization motion estimation (OME) and the compensation method is very similar to that employed in other standards. The main is that block-based motion estimation and compensation is suitable for optimization structures in MPEG-4. Motion estimation (ME) is one of the most computationally-intensive companies in the video compression technology. Video compression algorithm, MPEG1, MPEG4, such as H.261 and H.264, a lot of the standard used. Improved compression performance. Energy effective motion estimation technique is reduced to the rest of the frame involved in motion compensation.

Image Quality Improvements Based on Motion-Based Deblurring for Single-Photon Imaging

Photon counting imaging can be used to capture clearly photon-limited scenes. In photon counting imaging, information on incident photons is obtained as binary frames (bit-plane frames), which are transformed into a multi-bit image in the reconstruction process. In this process, it is necessary to apply a deblurring method to enable the capture of dynamic scenes without motion blur. In this article, a deblurring method for the high-quality bit-plane frame reconstruction of dynamic scenes is proposed. The proposed method involves the deblurring of units of object motion within a scene through the application of motion compensation to pixels sharing the same motions. This method achieves more efficient motion blur suppression than the application of simple deblurring to pixel block or spatial region units. It also applies a novel technique for accurate motion estimation from the bit-plane frame even in photon-limited situations through the statistical evaluation of the temporal variation of photon incidence. In addition to deblurring, our experimental results also revealed that the proposed method can be applied for denoising, which improves the peak signal-to-noise ratio by 1.2 dB. In summary, the proposed method for bit-plane reconstruction achieves high quality imaging even in photon-limited dynamic scenes.

Appraisal of motion estimation techniques in video compression

In this paper, plan to propose a new motion estimation technique similar to that of estimation methods such as full search 3ss,4ss, block matching approaches and parametric motion estimation. Motion Estimation is mainly used to estimate the motion or error in an object. It can easily identify and tracks the movements of object in a frame. In this paper to present a detailed survey of motion estimation techniques and various methods of searching methodologies are also analyzed. Investigation shall also be carried out to estimate the motion to enrich the quality of decompressed video by analyzing the compression error. A study on not only the design of new motion estimation algorithm but also error free compression is going to be the main objective.

Joint direct estimation of 3D geometry and 3D motion using spatio temporal gradients

Conventional image-motion based methods for structure from motion first compute optical flow, then solve for the 3D motion parameters based on the epipolar constraint, and finally recover the 3D geometry of the scene. However, errors in optical flow due to regularization can lead to large errors in 3D motion and structure. This paper investigates whether performance and consistency can be improved by avoiding optical flow estimation in the early stages of the structure-from-motion pipeline, and it proposes a new direct method based on image gradients (normal flow) only. Our main idea lies in a reformulation of the positive-depth constraint – the basis for estimating egomotion from normal flow – as a continuous piecewise differentiable function, which allows the use of well-known minimization techniques to solve for 3D motion. The 3D motion estimate is then refined and structure estimated adding a regularization based on depth. Experimental comparisons on standard synthetic datasets and the real-world driving benchmark dataset Kitti using three different optic flow algorithms show that the method achieves better accuracy in all but one case. Furthermore, it outperforms existing normal flow based 3D motion estimation techniques. Finally, the recovered 3D geometry is shown to be also very accurate.

Adaptive Compressive Video Coding for Embedded Camera Sensors: Compressed Domain Motion and Measurements Estimation

This paper presents a new framework for wireless video streaming services using the resource-constrained embedded camera sensors based on block compressive sensing. We propose adaptive encoding scheme that exploits high redundancy between successive video frames to reduce transmission cost while effectively handling occlusion effects and, at the same time, maintains the simple and energy conserving encoder design. The proposed methodology adapts the compression ratio for different blocks of the non-key frame depending on temporal correlation. We also propose compressed domain motion and measurements estimation techniques to exploit the high correlation between successive frames at the decoder. The proposed motion estimation technique makes use of restricted isometry property of the sensing matrix to seek the best matching measurement vector for motion estimation as opposed to block matching in conventional video coding. In the proposed measurement estimation technique, efficient utilization of bandwidth is achieved by skipping some measurements at the transmitter side. The skipped measurements are estimated at the receiver by exploiting the correlation between CS measurements of the non-key frame and corresponding motion predicted frame using multiple regression model. Extensive simulation results on a set of diverse video sequences are presented to demonstrate the effectiveness of the proposed video coding technique.

Motion estimation for ultrasound image sequences using deep learning

Increased number of medical images being used for medical praxis makes automatic processing of images a necessity. Conventional techniques of motion estimation in Ultrasound images such as exhaustive search-based block matching (ES-BM) are known to be computationally expensive and are unsuitable for portable devices. On the other hand, research in computer vision has helped the development of deep learning-based techniques for real-time motion estimation of day-to-day non-medical objects. In this paper, we propose to adopt one such deep neural network-based Fully Convolutional Siamese tracker along with Linear Kalman Filter (SiamFC-LKF) to track regions of interest in ultrasound image sequences. Siamese networks use two convolutional neural networks to create a feature map of the given reference block and the possible candidate blocks of subsequent frames. The candidate block of the subsequent frame with the maximum correlation value is considered as the tracked output. Since SiamFC does not consider any motion model, we introduce a linear Kalman filter to track the wall of the carotid artery. SiamFC-LKF and ES-BM were tested on five different image sequences of the longitudinal section of the carotid artery. Our experiments showed that SiamFC-LKF was 6 times faster and performed better than ES-BM in most of the cases.

RSS and Phase Kalman Filter Fusion for Improved Velocity Estimation in the Presence of Real-World Factors

Motion estimation of RFID tags has been shown to be possible using phase information, while position estimation is possible using received signal strength (RSS). This article combines phase and RSS information using a Kalman filter as sensor fusion technique. It compares the combined RSS and phase fusion Kalman filter technique for motion estimation with the phase only Kalman filter, exponential smoothing filter and raw phase motion estimation excluding a Kalman filter. The influence of frequency hopping broadcasting regulations on the performance of these techniques is investigated, an aspect that has not been properly addressed before. The primary factors found in real-world scenarios that have an effect on motion estimation techniques are identified as multipath reflections and the presence of multiple tags. The influence of such non-ideal environments are characterized and specific vulnerabilities of these techniques are investigated. Finally, the shortcoming of these techniques as well as advantages and possible environments where this technique can be successfully used are discussed.

Motion Estimation From Doppler and Spatial Data in SONAR Images

Motion estimation is critical for the localization of autonomous underwater vehicles. Current SONAR-based techniques exclusively utilize either Doppler or spatial measurements. However, these measurement domains are complementary to each other; Doppler measurements directly measure radial motion, whereas spatial measurements uniquely observe angular motion. Therefore, this article presents SONARODO (SONAR Odometry), a novel real-time motion estimation algorithm for 2-D forward-looking SONARs. It depends on a largely decoupled motion estimation process that better utilizes each measurement domain for their respective strengths. Specifically, it estimates translational motion from Doppler-azimuth images and rotational motion from range-azimuth images. While this method does require a SONAR that can provide both image types, it was designed to ensure robustness to featureless seafloor environments and low-resolution images. This article's validation with high-fidelity simulation data demonstrated that SONARODO offers accuracy and computational cost advantages over related motion estimation techniques.

U-net-based deformation vector field estimation for motion-compensated 4D-CBCT reconstruction.

For four-dimensional cone-beam computed tomography (4D-CBCT), its image quality is usually degraded by insufficient projections at each respiratory phase after phase-sorting. Recently, we developed a simultaneous motion estimation and image reconstruction (SMEIR) technique, which can improve lung 4D-CBCT reconstruction quality by incorporating an interphase motion model generated as deformation vector fields (DVFs). Simultaneous motion estimation and image reconstruction uses an intensity-driven two-dimensional (2D)-three-dimensional (3D) deformation technique to estimate these DVFs by intensity-matching 2D projections. However, 2D-3D deformation may fail to generate accurate intra-lung DVFs, since the motion of intricate, small lung structures only leads to subtle intensity variations on 2D projections that are insufficient to drive accurate DVF optimization. This study is to develop convolutional neural network (CNN)-based methods to fine-tune the 2D-3D deformation DVFs to improve the efficiency and accuracy of 4D-CBCT reconstruction. We built two U-net-based architectures for this study. The first architecture (U-net-3C) uses 2D-3D deformation-estimated DVFs (in three cardinal directions) as the input with three channels (3C), and outputs are fine-tuned DVFs. For the second architecture (U-net-4C), the reference phase CBCT image reconstructed by SMEIR was added as an additional input channel (4C) to represent patient-specific heterogeneous properties of the lung. The output fine-tuned high-quality DVFs of both models were input again into the SMEIR workflow, as an optimized motion model, to generate the final 4D-CBCT. Both methods were evaluated on 11 lung patient cases, using fivefold cross-validation. We also reconstructed 4D-CBCTs by the original SMEIR and the SMEIR-Bio (SMEIR with biomechanical modeling) algorithms for comparison. The 4D-CBCT accuracy was quantitatively assessed through metrics including root-mean-square-error (RMSE), universal quality index (UQI), and normalized cross-correlation (NCC). The DVF accuracy was evaluated by manually tracked lung landmarks. We also evaluated our proposed methods on the SPARE challenge dataset based on reconstructed 4D-CBCT quality using the above metrics. The average (±standard deviation) residual DVF errors of SMEIR-U-net-3C, SMEIR-U-net-4C, SMEIR-Bio, and SMEIR were 3.88±3.12mm, 3.71±2.90mm, 3.75±3.40mm, and 5.73±4.61mm, respectively. The SMEIR-U-net-3C and SMEIR-U-net-4C generated images of generally improved RMSE, UQI, and NCC as compared to the other methods. Compared with SMERI-U-net-3C, SMEIR-U-net-4C has slightly higher 4D-CBCT reconstruction and DVF estimation accuracy. For the SPARE dataset, the UQI for SMEIR-U-net-3C, SMEIR-U-net-4C, SMEIR-Bio, and SMEIR were 0.96, 0.97, 0.96, and 0.94. The CNN-based models can achieve fast (~10s) and accurate DVF fine-tuning to improve the efficiency and accuracy of 4D-CBCT reconstruction.

Lossy and Lossless Video Frame Compression: A Novel Approach for High-Temporal Video Data Analytics

The smart city concept has attracted high research attention in recent years within diverse application domains, such as crime suspect identification, border security, transportation, aerospace, and so on. Specific focus has been on increased automation using data driven approaches, while leveraging remote sensing and real-time streaming of heterogenous data from various resources, including unmanned aerial vehicles, surveillance cameras, and low-earth-orbit satellites. One of the core challenges in exploitation of such high temporal data streams, specifically videos, is the trade-off between the quality of video streaming and limited transmission bandwidth. An optimal compromise is needed between video quality and subsequently, recognition and understanding and efficient processing of large amounts of video data. This research proposes a novel unified approach to lossy and lossless video frame compression, which is beneficial for the autonomous processing and enhanced representation of high-resolution video data in various domains. The proposed fast block matching motion estimation technique, namely mean predictive block matching, is based on the principle that general motion in any video frame is usually coherent. This coherent nature of the video frames dictates a high probability of a macroblock having the same direction of motion as the macroblocks surrounding it. The technique employs the partial distortion elimination algorithm to condense the exploration time, where partial summation of the matching distortion between the current macroblock and its contender ones will be used, when the matching distortion surpasses the current lowest error. Experimental results demonstrate the superiority of the proposed approach over state-of-the-art techniques, including the four step search, three step search, diamond search, and new three step search.

VLSI Architecture of Block Matching Algorithms for Motion Estimation in High Efficiency Video Coding

High-efficiency video coding (HEVC) is a latest video coding standard and the motion estimation unit is the most important block. The work presents the different types of Matching Criteria for Block-Based Motion Estimation technique in HEVC standard. HEVC requires fast motion estimation algorithms to have better real time performance. The hardware implementation of motion estimation helps to achieve high speed though parallel processing. An improved block matching technique is designed with reduced blocks for HEVC. The proposed method has less execution time where only blocks having motion are compared for prediction computation. The searching time complexity is dependent on the number of blocks that are having motion. The searching time of frame having small motion can be reduced to 80–85% as compared to the traditional full search algorithm. In the paper, sum of absolute difference, mean square error and mean absolute difference are computed to find the best matching algorithm for HEVC. However, SAD has less computational complexity with compare to other matching criteria. The results suggest that proposed motion estimation algorithm has better performance with compare to similar previous work.

High Dynamic Range Video Synthesis Using Superpixel-Based Illuminance-Invariant Motion Estimation

We propose a robust high dynamic range (HDR) video synthesis algorithm using the superpixel-based illuminance-invariant motion estimation technique. The proposed algorithm first selects an input frame in an alternating exposed input video as the reference. Then, the correspondences between two adjacent frames are estimated by employing a feature descriptor, which is robust against illuminance variation, and a superpixel segmentation technique. Next, the input frames are warped to the reference frame using the estimated motion maps. Finally, the final HDR frame is synthesized by constructing a weight map, which can handle complex motions and poor exposures by considering the underlying structures in the input frames. Experimental results on real test sequences show that the proposed algorithm can provide high-quality HDR videos compared with those obtained by state-of-the-art algorithms in terms of both subjective and objective evaluations.