Local Motion Field Research Articles

A unified feature-motion consistency framework for robust image matching

Establishing reliable feature matches between a pair of images in various scenarios is a long-standing open problem in photogrammetry. Attention-based detector-free matching with coarse-to-fine architecture has been a typical pipeline to build matches, but the cross-attention module with global receptive field may compromise the structural local consistency by introducing irrelevant regions (outliers). Motion field can maintain structural local consistency under the assumption that matches for adjacent features should be spatially proximate. However, motion field can only estimate local displacements between consecutive images and struggle with long-range displacements estimation in large-scale variation scenarios without spatial correlation priors. Moreover, large-scale variations may also disrupt the geometric consistency with the application of mutual nearest neighbor criterion in patch-level matching, making it difficult to recover accurate matches. In this paper, we propose a unified feature-motion consistency framework for robust image matching (MOMA), to maintain structural consistency at both global and local granularity in scale-discrepancy scenarios. MOMA devises a motion consistency-guided dependency range strategy (MDR) in cross attention, aggregating highly relevant regions within the motion consensus-restricted neighborhood to favor true matchable regions. Meanwhile, a unified framework with hierarchical attention structure is established to couple local motion field with global feature correspondence. The motion field provides local consistency constraints in feature aggregation, while feature correspondence provides spatial context prior to improve motion field estimation. To alleviate geometric inconsistency caused by hard nearest neighbor criterion, we propose an adaptive neighbor search (soft) strategy to address scale discrepancy. Extensive experiments on three datasets demonstrate that our method outperforms solid baselines, with AUC improvements of 4.73/4.02/3.34 in two-view pose estimation task at thresholds of 5°/10°/20° on Megadepth test, and 5.94% increase of accuracy at threshold of 1px in homography task on HPatches datasets. Furthermore, in the downstream tasks such as 3D mapping, the 3D models reconstructed using our method on the self-collected SYSU UAV datasets exhibit significant improvement in structural completeness and detail richness, manifesting its high applicability in wide downstream tasks. The code is publicly available at https://github.com/BunnyanChou/MOMA.

Motion Estimation for Complex Fluid Flows Using Helmholtz Decomposition

In this paper, we proposed a novel motion model with Helmholtz decomposition for complex fluid flows in a filtering-based optical flow framework, where the optimization of the regularization term is treated as a filtering process, and different motion patterns can be captured by finding appropriate filter kernels based on the designed filters. In this framework, we introduce a novel optical flow method with a joint spatial filter, which is based on the Helmholtz decomposition theorem that assumes a local motion field is composed of a curl field and a divergence field. By adjusting the scale of the weights in the filter kernels and combining a curl filter with a divergence filter in a certain ratio, it can simulate different motion patterns. In addition, if the correlation between the horizontal and vertical components of the optical flow field in the filter kernel is eliminated, it will be transformed into a linear motion model. Based on this linear motion model, we also develop a novel optical flow method with an adaptive guided filter. By finding an adaptive filter kernel driven by both the input image and the guided motion field for the designed filter, it can successfully capture different motion patterns, and yield an edge-preserving smoothing optical flow field. Most importantly, the proposed optical flow model provides a new way to design the regularizers for capturing different motion patterns in complex fluid flow. In particular, the designed optical flow method with an adaptive guided filter significantly outperforms the current state-of-the-art optical flow methods in predicting complex fluid flows.

Directional and inter-acquisition variability in diffusion-weighted imaging and editing for restricted diffusion.

PurposeTo evaluate and quantify inter‐directional and inter‐acquisition variation in diffusion‐weighted imaging (DWI) and emphasize signals that report restricted diffusion to enhance cancer conspicuity, while reducing the effects of local microscopic motion and magnetic field fluctuations.MethodsTen patients with biopsy‐proven prostate cancer were studied under an Institutional Review Board‐approved protocol. Individual acquisitions of DWI signal intensities were reconstructed to calculate inter‐acquisition distributions and their statistics, which were compared for healthy versus cancer tissue. A method was proposed to detect and filter the acquisitions affected by motion‐induced signal loss. First, signals that reflect restricted diffusion were separated from the acquisitions that suffer from signal loss, likely due to microscopic motion, by imposing a cutoff value. Furthermore, corrected apparent diffusion coefficient maps were calculated by employing a weighted sum of the multiple acquisitions, instead of conventional averaging. These weights were calculated by applying a soft‐max function to the set of acquisitions per‐voxel, making the analysis immune to acquisitions with significant signal loss, even if the number of such acquisitions is high.ResultsInter‐acquisition variation is much larger than the Rician noise variance, local spatial variations, and the estimates of diffusion anisotropy based on the current data, as well as the published values of anisotropy. The proposed method increases the contrast for cancers and yields a sensitivity of 98.8% with a false positive rate of 3.9%.ConclusionMotion‐induced signal loss makes conventional signal‐averaging suboptimal and can obscure signals from areas with restricted diffusion. Filtering or weighting individual acquisitions prior to image analysis can overcome this problem.

Three-Zone Segmentation Based Motion Compensation for Video Compression.

Motion compensation has been widely employed for removing temporal redundancies in typical hybrid video coding framework. The popular video compression standards, such as H.264/AVC and HEVC, adopt the block based partitioning model to describe the motion filed due to its high compression efficiency and relatively low computational complexity. However, block based motion compensation may not align with the actual object motion boundaries, potentially limiting the compression efficiency. In view of this, we propose a three-zone segmentation based motion compensation scheme to improve the description accuracy of motion field as well as the coding efficiency. In particular, the segmentation information is implied in the reference frame instead of being explicitly signalled. Based on the segmentation information, three motion compensation zones can be identified, including one edge, one foreground, and one background zone. The foreground zone is motion compensated by the signalled motion vector of the block, and the background zone is motion compensated by the motion information implicitly derived from the local motion field. Regarding the edge zone, it is viewed as an overlapped area and the weighted compensation strategy is adopted. The proposed algorithm is implemented into the reference software VTM-1.0 of Versatile Video Coding (VVC), and simulation results show that the algorithm can achieve 1.14% and 1.06% bitrate savings for random access and lowdelay configurations, respectively.

Hierarchical Filtered Motion for Action Recognition in Crowded Videos

Action recognition with cluttered and moving background is a challenging problem. One main difficulty lies in the fact that the motion field in an action region is contaminated by the background motions. We propose a hierarchical filtered motion (HFM) method to recognize actions in crowded videos by the use of motion history image (MHI) as basic representations of motion because of its robustness and efficiency. First, we detect interest points as the two-dimensional Harris corners with recent motion, e.g., locations with high intensities in the MHI. Then, a global spatial motion smoothing filter is applied to the gradients of the MHI to eliminate isolated unreliable or noisy motions. At each interest point, a local motion field filter is applied to the smoothed gradients of the MHI by computing structure proximity between any pixel in the local region and the interest point. Thus, the motion at a pixel is enhanced or weakened based on its structure proximity with the interest point. To validate its effectiveness, we characterize the spatial and temporal features by histograms of oriented gradient in the intensity image and the MHI, respectively, and use a Gaussian-mixture-model-based classifier for action recognition. The performance of the proposed approach achieves the state-of-the-art results on the KTH dataset that has clean background. More importantly, we perform cross-dataset action classification and detection experiments, where the KTH dataset is used for training, while the microsoft research (MSR) action dataset II that consists of crowded videos with people moving in the background is used for testing. Our experiments show that the proposed HFM method significantly outperforms existing techniques.

A Robust hierarchical motion estimation algorithm in noisy image sequences in the bispectrum domain

The present study describes a robust hierarchical motion estimation algorithm in noisy image sequences using the bispectrum. The motion can be characterized by an affine model and the parameters of an affine motion model are estimated by means third-order auto-bispectrum and cross-bispectrum measures. The basic components of this framework to obtain motion vectors are (i) pyramid construction, (ii) motion estimation and (iii) coarse-to-fine refinement. The entire motion is decomposed as a global and a local motion field, which helps accurately obtain high resolution estimates for the local motion field. Simulation results are presented and compared to those obtained from the phase correlation algorithm. The results demonstrate that the proposed method is more suited than the phase correlation algorithm to analyses complex noisy image sequences. On the other hand, our method produces smoother displacement vector field with a more accurate measure of object motion in different signal-to-noise ratio scenarios.

A robust motion estimation algorithm for PIV

The present study proposes an approach for robust estimation of the instantaneous motion field from a time-lagged pair of PIV images. The method is based on phase correlation, where the phase of the Fourier components is used for motion parameter estimation. Unlike the cross correlation-based techniques, this technique uses ‘whitening’ FIR filters to sharpen the cross correlation maxima, thereby improving the accuracy of identification of the peak. The proposed method also combines the advantages of the phase correlation and the cross correlation techniques in determining the reliability of the estimates, thus providing a method of filtering out a significantly large number of spurious vectors. This reliability metric helps reduce the possibility of over-smoothing the flow field when performing data validation. With regard to the efficiency of the technique, both phase correlation and cross correlation are derived from the Fourier components of the same image region. Each of the estimates can thus be obtained in parallel, without increasing the computational complexity of the system. Unlike many region-based methods that are currently available, the entire motion is decomposed as a global and a local motion field, which helps accurately obtain high interrogation resolution estimates for the local motion field.

Vector Rational Interpolation Schemes for Erroneous Motion Field Estimation Applied to MPEG-2 Error Concealment

A study on the use of vector rational interpolation for the estimation of erroneously received motion fields of MPEG-2 predictively coded frames is undertaken in this paper, aiming further at error concealment (EC). Various rational interpolation schemes have been investigated, some of which are applied to different interpolation directions. One scheme additionally uses the boundary matching error and another one attempts to locate the direction of minimal/maximal change in the local motion field neighborhood. Another one further adopts bilinear interpolation principles, whereas a last one additionally exploits available coding mode information. The methods present temporal EC methods for predictively coded frames or frames for which motion information pre-exists in the video bitstream. Their main advantages are their capability to adapt their behavior with respect to neighboring motion information, by switching from linear to nonlinear behavior, and their real-time implementation capabilities, enabling them for real-time decoding applications. They are easily embedded in the decoder model to achieve concealment along with decoding and avoid post-processing delays. Their performance proves to be satisfactory for packet error rates up to 2% and for video sequences with different content and motion characteristics and surpass that of other state-of-the-art temporal concealment methods that also attempt to estimate unavailable motion information and perform concealment afterwards.

Multistage motion estimation for image interpolation

Abstract In this paper we describe an algorithm for accurate motion estimation that takes into account both the motion of the camera (global motion) and the motion of the imaged objects (local motion) through the use of a multistage structure. First, global motion parameters are accurately estimated and the images are compensated for the camera motion. The local motion field is then estimated on the basis of these ‘compensated’ images. To improve the performance of this estimation, both temporal and spatial congruence constraints have been introduced. To test the performance of the proposed algorithm a motion compensated image interpolation that uses the estimated motion field has been carried out. The experimental results show the effectiveness of the proposed multistage motion estimation procedure.