Motion Estimation Method Research Articles

Fast robust matching algorithm based on BRISK and GMS

In the image stitching technology, traditional feature extraction algorithms have uneven feature points distribution, many redundant features, time-consuming feature points precision matching and low image registration accuracy. In view of these problems, this paper proposes an improved image registration algorithm based on BRISK and GMS. In this method, the image is first divided into meshes and BRISK algorithm is used for extracting image features, then the Brute Force matching algorithm is used for rough image matching. Finally, the mesh motion estimation method is used for feature quantity statistics, and error matching is removed to obtain a set of fine matching feature points for image registration. This paper verified the robustness of the improved algorithm by comparing it with other methods in the Mikolajczyk data set. The experimental results show that this algorithm achieves higher matching accuracy on the basis of maintaining speed compared with the original algorithm, and the average accuracy is improved by 8.02%. It has better performance than the traditional algorithm, which can be used for occasions with higher requirements on registration accuracy and real-time performance.

Triple-Frame-Based Bi-Directional Motion Estimation for Motion-Compensated Frame Interpolation

In this paper, new motion estimation (ME) and motion vector refinement (MVR) methods for motion-compensated frame interpolation (MCFI) are proposed. To estimate motion vectors (MVs) with great reliability, triple-frame-based bi-directional ME (TFBME), which employs three sequential frames, is developed. In TFBME, the middle frame works as a reference to estimate MVs between two end frames, which leads to performance improvement especially when objects move linearly through three sequential frames. In addition, a modified bi-directional ME method, which employs only two successive frames, is combined with TFBME to handle cases in which objects move non-linearly through three sequential frames. In our proposed MVR, artifacts on interpolated frames are first detected by a convolutional neural network and then corrected by weighted vector median filtering. Experimental results show that the proposed MCFI method outperforms conventional methods in both subjective and objective evaluations.

Parallax beam: a vision-based motion estimation method robust to nearly planar scenes

In computer vision, the epipolar geometry embeds the geometrical relationship between two views of a scene. This geometry is degenerated for planar scenes as they do not provide enough constraints to estimate it without ambiguity. Nearly planar scenes can provide the necessary constraints to resolve the ambiguity. But classic estimators such as the 5-point or 8-point algorithm combined with a random sampling strategy are likely to fail in this case because a large part of the scene is planar and it requires lots of trials to get a nondegenerated sample. However, the planar part can be associated with a homographic model and several links exist between the epipolar geometry and homographies. The epipolar geometry can indeed be recovered from at least two homographies or one homgraphy and two noncoplanar points. The latter fits a wider variety of scenes, as it is unsure to be able to find a second homography in the noncoplanar points. This method is called plane-and-parallax. The equivalence between the parallax and the epipolar lines allows to recover the epipole as their common intersection and the epipolar geometry. Robust implementations of the method are rarely given, and we encounter several limitations in our implementation. Noisy image features and outliers make the lines not to be concurrent in a common point. Also off-plane features are unequally influenced by the noise level. We noticed that the bigger the parallax is, the lesser the noise influence is. We, therefore, propose a model for the parallax that takes into account the noise on the features location to cope with the previous limitations. We call our method the “parallax beam.” The method is validated on the KITTI vision benchmark and on synthetic scenes with strong planar degeneracy. The results show that the parallax beam improves the estimation of the camera motion in the scene with planar degeneracy and remains usable when there is not any particular planar structure in the scene.

A novel diamond–hexagon search algorithm for motion estimation

Motion estimation (ME) is initial way for video compression and associated to the compression efficacy by dipping temporal redundancies. ME is the most significant measure of a video encoder and half of coding intricacy or computational time depends on it. There were numerous ME algorithms suggested and realized to minimalize the computational time. H.264/AVC codec deals various coding method for realizing high compression gains as relate to other criterions. These methods affectedly rise the computational intricacy of the block based ME, which consumes up to 80% of the complete encoder’s computations. In this paper, we proposed diamond–hexagon search algorithm for block matching ME method. This proposed method delivers lessening in computational intricacy and encoding time without conceding the quality of the video sequence.

A Robust and Efficient Video Anti-Shaking Algorithm for Low-End Smartphone Platforms

Feature phones or low-end smart phones are an important part of our daily life. While lacking the powerful embedded camera and processing capability of the high-end smart phone, the feature phones with the low-end camera and processing capability are still widely used owing to its lower cost. Consequently, image and video processing algorithms are needed for many intelligent applications on the low-end smart phones. In this paper, we propose a novel robust and efficient video anti-shaking algorithm for the low-end smartphone platform. It provides an integrated application for the low-end smartphone platform, which includes a novel global motion estimation method, and a rapid and efficient local motion detection method. The overall framework is proposed considering the limited camera perception quality and limited mobile processing capability of the feature phone. The proposed algorithm can not only obtain a good visual effect but also be implemented in real-time for the low-end mobile platforms with very limited computational resource. Experimental results demonstrate that the proposed method can achieve a similar video stabilization effect to the Optical Image Stabilization hardware in high-end smartphones.

MR-PET head motion correction based on co-registration of multicontrast MR images.

Head motion is a major source of image artefacts in neuroimaging studies and can lead to degradation of the quantitative accuracy of reconstructed PET images. Simultaneous magnetic resonance-positron emission tomography (MR-PET) makes it possible to estimate head motion information from high-resolution MR images and then correct motion artefacts in PET images. In this article, we introduce a fully automated PET motion correction method, MR-guided MAF, based on the co-registration of multicontrast MR images. The performance of the MR-guided MAF method was evaluated using MR-PET data acquired from a cohort of ten healthy participants who received a slow infusion of fluorodeoxyglucose ([18-F]FDG). Compared with conventional methods, MR-guided PET image reconstruction can reduce head motion introduced artefacts and improve the image sharpness and quantitative accuracy of PET images acquired using simultaneous MR-PET scanners. The fully automated motion estimation method has been implemented as a publicly available web-service.

Fetal XCMR: a numerical phantom for fetal cardiovascular magnetic resonance imaging

BackgroundValidating new techniques for fetal cardiovascular magnetic resonance (CMR) is challenging due to random fetal movement that precludes repeat measurements. Consequently, fetal CMR development has been largely performed using physical phantoms or postnatal volunteers. In this work, we present an open-source simulation designed to aid in the development and validation of new approaches for fetal CMR. Our approach, fetal extended Cardiac-Torso cardiovascular magnetic resonance imaging (Fetal XCMR), builds on established methods for simulating CMR acquisitions but is tailored toward the dynamic physiology of the fetal heart and body. We present comparisons between the Fetal XCMR phantom and data acquired in utero, resulting in image quality, anatomy, tissue signals and contrast.MethodsExisting extended Cardiac-Torso models are modified to create maternal and fetal anatomy, combined according to simulated motion, mapped to CMR contrast, and converted to CMR data. To provide a comparison between the proposed simulation and experimental fetal CMR images acquired in utero, images from a typical scan of a pregnant woman are included and simulated acquisitions were generated using matching CMR parameters, motion and noise levels. Three reconstruction (static, real-time, and CINE), and two motion estimation methods (translational motion, fetal heart rate) from data acquired in transverse, sagittal, coronal, and short-axis planes of the fetal heart were performed to compare to in utero acquisitions and demonstrate feasibility of the proposed simulation framework.ResultsOverall, CMR contrast, morphologies, and relative proportions of the maternal and fetal anatomy are well represented by the Fetal XCMR images when comparing the simulation to static images acquired in utero. Additionally, visualization of maternal respiratory and fetal cardiac motion is comparable between Fetal XCMR and in utero real-time images. Finally, high quality CINE image reconstructions provide excellent delineation of fetal cardiac anatomy and temporal dynamics for both data types.ConclusionThe fetal CMR phantom provides a new method for evaluating fetal CMR acquisition and reconstruction methods by simulating the underlying anatomy and physiology. As the field of fetal CMR continues to grow, new methods will become available and require careful validation. The fetal CMR phantom is therefore a powerful and convenient tool in the continued development of fetal cardiac imaging.

Motion Velocity, Acceleration and Energy Expenditure Estimation Using Micro Flow Sensor

Motion velocity, acceleration, and energy expenditure estimations are important in quantitative assessments for physical recovery and exercise-based interventions for post-stroke patients with partial losses of neurological functions. This paper proposes a novel wearable motion estimation device using a micro flow sensor, which realizes motion velocity, acceleration detection, and energy expenditure estimation for human limb motion. A homemade micro thermal flow sensor is used to detect motion velocity. Jerk-Kalman based algorithm is proposed to extract motion acceleration from the flow sensor outputs. Motion velocity and acceleration are used to estimate energy expenditure of limb motion. Calibration experiments and application cases are conducted to validate the effectiveness of the methodology. The micro flow sensor based motion estimation method is free of accumulated error, robust for dynamic motion measurement, and provides a promising auxiliary approach for evaluating energy expenditures of human limbs in rehabilitation training.

Spatial Multilevel Optical Flow Architecture-based Dynamic Motion Estimation in Vehicular Traffic Scenarios

Pedestrian detection is a challenging area in the intelligent vehicles domain. During the last years, many works have been proposed to efficiently detect motion in images. However, the problem becomes more complex when it comes to detecting moving areas while the vehicle is also moving. This paper presents a variational optical flow-based method for motion estimation in vehicular traffic scenarios. We introduce a framework for detecting motion areas with small and large displacements by computing optical flow using a multilevel architecture. The flow field is estimated at the shortest level and then successively computed until the largest level. We include a filtering parameter and a warping process using bicubic interpolation to combine the intermediate flow fields computed at each level during optimization to gain better performance. Furthermore, we find that by including a penalization function, our system is able to effectively reduce the presence of outliers and deal with all expected circumstances in real scenes. Experimental results are performed on various image sequences from Daimler Pedestrian Dataset that includes urban traffic scenarios. Our evaluation demonstrates that despite the complexity of the evaluated scenes, the motion areas with both moving and static camera can be effectively identified.

High-Frame-Rate Contrast-Enhanced Echocardiography Using Diverging Waves: 2-D Motion Estimation and Compensation.

Combining diverging ultrasound waves and microbubbles could improve contrast-enhanced echocardiography (CEE), by providing enhanced temporal resolution for cardiac function assessment over a large imaging field of view. However, current image formation techniques using coherent summation of echoes from multiple steered diverging waves (DWs) are susceptible to tissue and microbubble motion artifacts, resulting in poor image quality. In this study, we used correlation-based 2-D motion estimation to perform motion compensation for CEE using DWs. The accuracy of this motion estimation method was evaluated with Field II simulations. The root-mean-square velocity errors were 5.9% ± 0.2% and 19.5% ± 0.4% in the axial and lateral directions, when normalized to the maximum value of 62.8 cm/s which is comparable to the highest speed of blood flow in the left ventricle (LV). The effects of this method on image contrast ratio (CR) and contrast-to-noise ratio (CNR) were tested in vitro using a tissue mimicking rotating disk with a diameter of 10 cm. Compared against the control without motion compensation, a mean increase of 12 dB in CR and 7 dB in CNR were demonstrated when using this motion compensation method. The motion correction algorithm was tested in vivo on a CEE data set acquired with the Ultrasound Array Research Platform II performing coherent DW imaging. Improvement of the B-mode and contrast-mode image quality with cardiac motion and blood flow-induced microbubble motion was achieved. The results of motion estimation were further processed to interpret blood flow in the LV. This allowed for a triplex cardiac imaging technique, consisting of B mode, contrast mode, and 2-D vector flow imaging with a high frame rate of 250 Hz.

Optical flow for image-based river velocity estimation

We present a novel motion estimation technique for image-based river velocimetry. It is based on the so-called optical flow, which is a well developed method for rigid motion estimation in image sequences, devised in computer vision community. Contrary to PIV (Particle Image Velocimetry) techniques, optical flow formulation is flexible enough to incorporate physics equations that govern the observed quantity motion. Over the past years, it has been adopted by experimental fluid dynamics community where many new models were introduced to better represent different fluids motions, (see [18] for a review). Our optical flow is based on the scalar transport equation and is augmented with a weighted diffusion term to compensate for small scale (non-captured) contributions. Additionally, since there is no ground truth data for such type of image sequences, we present a new evaluation method to assess the results. It is based on trajectory reconstruction of few Lagrangian particles of interest and a direct comparison against their manually-reconstructed trajectories. The new motion estimation technique outperformed traditional optical flow and PIV-based methods.

Comments on “Wind Gust Detection and Impact Prediction for Wind Turbines”

We refute statements in “Zhou, K., et al. Wind gust detection and impact prediction for wind turbines. Remote Sens. 2018, 10, 514.” about the impracticality of motion estimation methods to derive two-component vector wind fields from single scanning aerosol lidar data. Our assertion is supported by recently published results on the performance of two image-based motion estimation methods: cross-correlation (CC) and wavelet-based optical flow (WOF). The characteristics and performances of CC and WOF are compared with those of a two-dimensional variational (2D-VAR) method that was applied to radial velocity fields from a single scanning Doppler lidar. The algorithmic aspects of WOF and 2D-VAR are reviewed and we conclude that these two approaches are in fact similar and practical.

Motion Modeling and Localization of Skid-Steering Wheeled Rover on Loose Terrain

The motion behavior of a skid-steering rover on loose terrain is modeled experimentally, and a position estimation method based on the model is proposed. Skid steering is a steering method adopted for various vehicles and robots. In this method, the vehicles steer by controlling a ratio between the rotational velocities of their left and right wheels. For controlling a skid-steering rover, whose wheels can slip on the lunar and planetary surfaces, which are covered with loose regolith, it is important to model accurately the rover's motion behaviors on loose terrain. To this end, we conducted traveling tests using a four-wheeled rover in a sand field and analyzed the motion behaviors of the rover on loose terrain. In the tests, the velocities of the rover's left and right wheels were controlled, and the rover traveled with various turning patterns. From the test datasets, we derived a mathematical model that expresses a relationship between the input and output velocities of the rover's wheels. Furthermore, the effectiveness of a position and motion estimation method based on the model was confirmed from experiments conducted in the sand field.

Robust Optical Flow Estimation in Cardiac Ultrasound Images Using a Sparse Representation.

This paper introduces a robust 2-D cardiac motion estimation method. The problem is formulated as an energy minimization with an optical flow-based data fidelity term and two regularization terms imposing spatial smoothness and the sparsity of the motion field in an appropriate cardiac motion dictionary. Robustness to outliers, such as imaging artefacts and anatomical motion boundaries, is introduced using robust weighting functions for the data fidelity term as well as for the spatial and sparse regularizations. The motion fields and the weights are computed jointly using an iteratively re-weighted minimization strategy. The proposed robust approach is evaluated on synthetic data and realistic simulation sequences with available ground-truth by comparing the performance with state-of-the-art algorithms. Finally, the proposed method is validated using two sequences of in vivo images. The obtained results show the interest of the proposed approach for 2-D cardiac ultrasound imaging.

Efficient feature-based motion estimation in neurosurgery using non-maximum suppression

Abstract In this contribution we propose a feature-based method for motion estimation and correction in intraoperative thermal imaging during brain surgery. The motion is estimated from co-registered white-light images in order to perform a robust motion correction on the thermographic data. To ensure real-time performance of an intraoperative application, we optimise the processing time which essentially depends on the number of key points found by our algorithm. For this purpose we evaluate the effect of applying an non-maximum suppression (NMS) to improve the feature detection efficiency. Furthermore we propose an adaptive method to determine the size of the suppression area, resulting in a trade-off between accuracy and processing time.

Selective Gray-Coded Bit-Plane-Based Two-Bit Transform and Its Efficient Hardware Architecture for Low-Complexity Motion Estimation

In this paper, a novel low bit-depth representation-based motion estimation approach with its hardware architecture is presented. The low bit-depth representation of pixels in the proposed method is constructed by choosing them from the gray-coded bit planes in an efficient way. Additionally, a new matching creation is also presented to improve the motion estimation accuracy. Thanks to the proposed binarization approach, the computation load is considerably reduced compared to filtering-based low bit-depth representation motion estimation methods. A novel hardware architecture for the proposed method is also presented in this paper. Experimental results revealed that the proposed motion estimation method and its hardware architecture provides a good balance between motion estimation accuracy and hardware resources especially for consumer electronics applications.

Hierarchical Extended Bilateral Motion Estimation-Based Frame Rate Upconversion Using Learning-Based Linear Mapping.

We present a novel and effective learning-based frame rate upconversion (FRUC) scheme, using linear mapping. The proposed learning-based FRUC scheme consists of: 1) a new hierarchical extended bilateral motion estimation (HEBME) method; 2) a light-weight motion deblur (LWMD) method; and 3) a synthesis-based motion-compensated frame interpolation (S-MCFI) method. First, the HEBME method considerably enhances the accuracy of the motion estimation (ME), which can lead to a significant improvement of the FRUC performance. The proposed HEBME method consists of two ME pyramids with a three-layered hierarchy, where the motion vectors (MVs) are searched in a coarse-to-fine manner via each pyramid. The found MVs are further refined in an enhanced resolution of four times by jointly combining the MVs from the two pyramids. The HEBME method employs a new elaborate matching criterion for precise ME which effectively combines a bilateral absolute difference, an edge variance, pixel variances, and an MV difference among two consecutive blocks and its neighboring blocks. Second, the LWMD method uses the MVs found by the HEBME method and removes the small motion blurs in original frames via transformations by linear mapping. Third, the S-MCFI method finally generates interpolated frames by applying linear mapping kernels for the deblurred original frames. In consequence, our FRUC scheme is capable of precisely generating interpolated frames based on the HEBME for accurate ME, the S-MCFI for elaborate frame interpolation, and the LWMD for contrast enhancement. The experimental results show that our FRUC significantly outperforms the state-of-the-art non-deep learning-based schemes with an average of 1.42 dB higher in the peak signal-to-noise-ratio and shows comparable performance with the state-of-the-art deep learning-based scheme.

Motion quaternion-based motion estimation method of MYO using K-means algorithm and Bayesian probability

There are diverse types of devices based on natural user interface/experience for humanized computing. One such device, the MYO allows the measurement of arm motions and uses them as an interface based on gestures. There are several research works for measuring the arm motions using MYOs. For example, one of the studies defines two types of motions for a forearm and for an upper arm, respectively. The orientations of the two types are measured by two MYOs. Bayesian probabilities are calculated based on the measured orientations and are utilized to estimate the orientations of the upper arm that is not being measured. However, because the orientation of the MYO can be expressed by one quaternion, the Bayesian probability by quaternions is more accurate than the Bayesian probability by each element of quaternions. This paper proposes a motion estimation method to increase the accuracy of motion estimation. The orientations obtained from MYO are expressed by one quaternion and are clustered by K-means. In the experiments, the performance of the proposed method was validated by analyzing the difference between estimated motion quaternions and measured motion quaternions, which showed enhanced performance.

VLSI architecture of binary encoding technique for fast motion estimation based on Hamming distances

One-bit and two-bit transforms-based block motion estimation methods are introduced to reduce the computational complexity of the ME procedure by assisting a modest Boolean XOR operation, matched with lower bit-depth illustrations of image frames. However, due to some computational complexity and failure to develop the mathematical expression at the hardware level, a binary encoding mapping function technique is proposed. The mathematical expressions for the proposed method are easy to design and develop in hardware. The architecture of the proposed method is developed in Verilog HDL and implemented in Xilinx Artix-7. The results of the proposed method are checked for area, power and maximum frequency levels in different types of video frames.

An offline technique to evaluate residual motion of the diaphragm during deep inspiratory breath-hold from cone-beam CT datasets.

In radiation therapy, the computer-assisted deep inspiration breath-hold (DIBH) technique is one approach to deal with respiratory motion of tumors in the lung, liver, or upper abdomen. However, inter- and intra-breath-hold deviations from an optimal static tumor position might occur. Anovel method is presented to noninvasively measure the diaphragm position and thus estimate its residual deviation (as surrogate for the tumor position) based on cone-beam computed tomography (CBCT) projection data using active breathing control during acquisition. The diaphragm dome (DD) position relative to the isocenter of alinear accelerator is known from the static (DIBH) planning CT. A ball-bearing phantom (BB) is placed at this position, aCBCT dataset is acquired, and in each projection the position of the projected BB is determined automatically based on thresholding. The position of the DD is determined manually in CBCT projections of apatient. The distance between DD and BB (ideal static setting) in craniocaudal direction is calculated for agiven angle based on the distance in the projection plane and the relative position of the BB referring to the source and the detector. An angle-dependent correction factor is introduced which takes this geometrical setting into account. The accuracy of the method is assessed. The method allows aCBCT projection-based estimation of the deviation between the DD and its optimal position as defined in the planning CT, i.e., the residual motion of the DD can be assessed. The error of this estimation is 2.2 mm in craniocaudal direction. The developed method allows an offline estimation of the inspiration depth (inter- and intra-breath-hold) over time. It will be useful as areference for comparison to other methods of residual motion estimation, e.g., surface scanning.