Nearest-neighbor Field Research Articles

Nearest-neighbor field algorithm based on patchMatch for myocardial perfusion motion estimation/correction

Deformation correction and recovery of dynamic magnetic resonance images (DMRI) with low complexity algorithms without compromising image quality is a challenging problem. We proposed a motion estimation deformation-correction compressive sensing (DC-CS) scheme to recover dynamic images from its undersampled measurements. We simplify the complex optimization problem into three sub-problems. The contributions of this research are: introducing a global search strategy instead of the DC registration step, guaranteeing a non-explicit motion estimation that avoids any spatial alignment or registration of the images, and lowering the computational cost to the minimum by using PatchMatch (PM). The simulation result shows that the PM algorithm accelerates the recovery time without losing the quality in comparison with the DC algorithm.

W-Tree Indexing For Fast Visual Word Generation

In image retrieval and visual identification, the bag-of-visual-words model has been extensively employed. The process of creating visual words—adding visual words to the associated local characteristics in a high-dimensional space—takes the longest to complete in order to acquire this representation. In order to save time, structures constructed around multi-branch trees & forests have recently been implemented. However, without a lot of backtracking, these approaches are unable to work well. In this study, we demonstrate that the lengthy process of visual word formation may be greatly sped up while keeping accuracy by taking into account the spatial correlation of local variables. We may create a co-occurrence table for each visual word in a huge data collection since some visual terms commonly co-occur with specific structures. We are able to allocate a probabilistic weight to each node of an index structure (for example, a KD-tree and a K-means tree) by associating each visual word with a probability in accordance with the corresponding co-occurrence table. This allows us to re-direct the searching path to be nearby to its global optimum within a minimal number of backtrackings. On the Oxford data set, we carefully examine the proposed scheme by contrasting it with the fast library for approximative nearest neighbours along with the random KD-trees. Extensive experimental findings point to the new scheme's effectiveness and efficiency. FeatureMatch, a generalised approximate nearest-neighbor field (ANNF) calculation framework connecting a source and target picture, is the solution we provide in this study. The suggested approach can calculate ANNF maps between any picture pairings, including unrelated ones. By using the proper spatial-range transformations, this generalisation is accomplished. Global colour adaptation is used on the source picture as a range transform to calculate ANNF maps. Low-dimensional characteristics are employed to approximate the image patches from the pair of photos, and KD-tree and ANNF estimation are combined. Based on picture coherency and spatial transformations, this ANNF map is further enhanced. We can now handle a broader spectrum of vision applications thanks to the suggested generalisation, which we couldn't previously manage with the ANNF framework.

Deep photographic style transfer guided by semantic correspondence

The objective of this paper is to develop an effective photographic style transfer method while preserving the semantic correspondence between the style and content images for both scenery and portrait images. A semantic correspondence guided photographic style transfer algorithm is developed, which is to ensure that the semantic structure of the content image has not been changed while the color of the style images is being migrated. The semantic correspondence is constructed in large scale regions based on image segmentation and also in local scale patches using Nearest-neighbor Field Search in the deep feature domain. Based on the semantic correspondence, a matting optimization is utilized to optimize the style transfer result to ensure the semantic accuracy and transfer faithfulness. The proposed style transfer method is further extended to automatically retrieve the style images from a database to make style transfer more-friendly. The experimental results show that our method could successfully conduct the style transfer while preserving semantic correspondence between diversity of scenes. A user study also shows that our method outperforms state-of-the-art photographic style transfer methods.

Partial correspondence of 3D shapes using properties of the nearest-neighbor field

This paper proposes an algorithm for finding correspondences between shapes in 3D. The method is designed to address three challenging cases: large deformations, partiality of the shapes, and topological noise. At the core of the method lies a novel, yet simple, similarity measure that analyzes statistical properties of the nearest-neighbor field from the source surface to the target. This information is shown to be powerful, compared to minimizing some function of distances. In particular, the proposed similarity function analyzes the diversity of the nearest-neighbor field and its preservation of distances. Empirical evaluation on partial matching benchmarks shows that our method outperforms state-of-the-art techniques, both quantitatively and qualitatively.

Hierarchical coherency sensitive hashing and interpolation with RANSAC for large displacement optical flow

Nearest Neighbor Field (NNF) has shown excellent performance for large displacement optical flow estimation recently. However, it contains much noise and lacks of global constraint. In this paper we present an effective approach, named HCSH (Hierarchical Coherency Sensitive Hashing), which combines the coarse-to-fine scheme and random search strategy, to enable NNF to enjoy the inherent smooth of coarse-to-fine framework. Then besides the forward–backward check for NNF, we also use auto-correlation to remove the unreliable correspondences in flat regions, where the NNF is often considered ambiguous and the motion can be naturally recovered by the latter interpolation. Inspired by EpicFlow, we propose edge-aware interpolation (EAI-Flow) to filling the gaps by removing correspondence. RANSAC is introduced to improve the locality-weighted affine transformation estimation, with neighbor propagation of affine model to reduce required iterations and speed up the computation. Experimental validation shows that our approach outperforms the state-of-the-art with more accurate optical flow.

Coarse-to-Fine PatchMatch for Dense Correspondence

Although the matching technique has been studied in various areas of computer vision for decades, efficient dense correspondence remains an open problem. In this paper, we present a simple but powerful matching method that works in a coarse-to-fine scheme for optical flow and stereo matching. Inspired by the nearest neighbor field (NNF) algorithms, our approach, called coarse-to-fine PatchMatch, blends an efficient random search strategy with the coarse-to-fine scheme for efficient dense correspondence. Unlike existing NNF techniques, which are efficient but yield results that are often too noisy because of a lack of global regularization, we propose a propagation step involving a constrained random search radius between adjacent levels of a hierarchical architecture. The resulting correspondence has a built-in smoothing effect, making it more suited to dense correspondence than the NNF techniques. Furthermore, our approach can also capture tiny structures with large motions, which is a problem for traditional coarse-to-fine methods. Interpolated using an edge-preserving interpolation method, our method outperforms the state-of-the-art optical flow methods on the MPI-Sintel and KITTI data sets and is much faster than competing methods.

Photo Realistic Image Completion via Dense Correspondence

In this paper, we propose an image completion algorithm based on dense correspondence between the input image and an exemplar image retrieved from the Internet. Contrary to traditional methods which register two images according to sparse correspondence, in this paper, we propose a hierarchical PatchMatch method that progressively estimates a dense correspondence, which is able to capture small deformations between images. The estimated dense correspondence has usually large occlusion areas that correspond to the regions to be completed. A nearest neighbor field (NNF) interpolation algorithm interpolates a smooth and accurate NNF over the occluded region. Given the calculated NNF, the correct image content from the exemplar image is transferred to the input image. Finally, as there could be a color difference between the completed content and the input image, a color correction algorithm is applied to remove the visual artifacts. Numerical results show that our proposed image completion method can achieve photo realistic image completion results.

Fast Optical Flow Estimation Without Parallel Architectures

According to recent results on Middlebury, MPI Sintel, and KITTI benchmarks, the accuracy of optical flow estimation algorithms has been significantly improved. The speed of them, however, has been too slow to meet the requirement of real-time applications. As a result, some parallel architectures (such as FPGA or GPU) have to be used for accelerating. Therefore, reducing the computational cost of optical flow estimation makes a lot of sense. To overcome the above issues, this paper proposes a fast local method based on 3D-gradients and approximate nearest-neighbor field (NNF), which is different from the widely used global model. In our method, NNF is used to provide initial optical flow field, and the proposed fast 3D-gradients-based local operator is used to propagate flow from coarse level to finer level in the coarse-to-fine refinement. We implement two versions of our method (with/without NNF initialization). Experimental results show that our method has a significant advantage for speed over other methods, where the fast version is capable of processing $Urban$ sequence ( $640\times 480$ ) at $\approx 10$ frames/s without parallel architectures. Meanwhile, our accuracy is also within acceptable levels on both small and large motion for some real-time applications.

Visual attribute transfer through deep image analogy

We propose a new technique for visual attribute transfer across images that may have very different appearance but have perceptually similar semantic structure. By visual attribute transfer, we mean transfer of visual information (such as color, tone, texture, and style) from one image to another. For example, one image could be that of a painting or a sketch while the other is a photo of a real scene, and both depict the same type of scene. Our technique finds semantically-meaningful dense correspondences between two input images. To accomplish this, it adapts the notion of "image analogy" [Hertzmann et al. 2001] with features extracted from a Deep Convolutional Neutral Network for matching; we call our technique deep image analogy. A coarse-to-fine strategy is used to compute the nearest-neighbor field for generating the results. We validate the effectiveness of our proposed method in a variety of cases, including style/texture transfer, color/style swap, sketch/painting to photo, and time lapse.

Abrupt motion tracking via nearest neighbor field driven stochastic sampling

Stochastic sampling based trackers have shown good performance for abrupt motion tracking so that they have gained popularity in recent years. However, conventional methods tend to use a two-stage sampling paradigm in which the search space needs to be uniformly explored with an inefficient preliminary sampling phase. In this paper, we propose a novel sampling-based method in the Bayesian filtering framework to address the problem. Within the framework, nearest neighbor field estimation is utilized to compute the importance proposal probabilities, which guide the Markov chain search towards promising regions and thus enhance the sampling efficiency; given the motion priors, a smoothing stochastic sampling Monte Carlo algorithm is proposed to approximate the posterior distribution through a smoothing weight-updating scheme. Moreover, to track the abrupt and the smooth motions simultaneously, we develop an abrupt-motion detection scheme which can discover the presence of abrupt motions during online tracking. Extensive experiments on challenging image sequences demonstrate the effectiveness and the robustness of our algorithm in handling the abrupt motions.

Fast edge-preserving PatchMatch for large displacement optical flow.

The speed of optical flow algorithm is crucial for many video editing tasks such as slow motion synthesis, selection propagation, tone adjustment propagation, and so on. Variational coarse-to-fine optical flow algorithms can generally produce high-quality results but cannot fulfil the speed requirement of many practical applications. Besides, large motions in real-world videos also pose a difficult problem to coarse-to-fine variational approaches. We, in this paper, present a fast optical flow algorithm that can handle large displacement motions. Our algorithm is inspired by recent successes of local methods in visual correspondence searching as well as approximate nearest neighbor field algorithms. The main novelty is a fast randomized edge-preserving approximate nearest neighbor field algorithm, which propagates self-similarity patterns in addition to offsets. Experimental results on public optical flow benchmarks show that our method is significantly faster than state-of-the-art methods without compromising on quality, especially when scenes contain large motions. Finally, we show some demo applications by applying our technique into real-world video editing tasks.

Non-rigid dense correspondence with applications for image enhancement

This paper presents a new efficient method for recovering reliable local sets of dense correspondences between two images with some shared content. Our method is designed for pairs of images depicting similar regions acquired by different cameras and lenses, under non-rigid transformations, under different lighting, and over different backgrounds. We utilize a new coarse-to-fine scheme in which nearest-neighbor field computations using Generalized PatchMatch [Barnes et al. 2010] are interleaved with fitting a global non-linear parametric color model and aggregating consistent matching regions using locally adaptive constraints. Compared to previous correspondence approaches, our method combines the best of two worlds: It is dense, like optical flow and stereo reconstruction methods, and it is also robust to geometric and photometric variations, like sparse feature matching. We demonstrate the usefulness of our method using three applications for automatic example-based photograph enhancement: adjusting the tonal characteristics of a source image to match a reference, transferring a known mask to a new image, and kernel estimation for image deblurring.

The Anisotropy in the Galaxy Velocity Field Originated from the Gravitational Pancaking Effect

We analyze the Millennium run semi-analytic galaxy catalog to explore quantitatively the gravitational pancaking effect on the orientation of galaxy velocity field. We first calculate the probability density distribution of the cosine of the angle between the velocity of a field galaxy and the direction normal to a local pancake plane which is determined using two nearest neighbor field galaxies. A clear signal of alignment is detected for the case that the pancake scale is in the range of $5-8h^{-1}$ Mpc. The tendency of the velocity-pancake alignment is found to still exist when the pancakes are determined using three neighbor galaxies, indicating that it has a spatial coherence. The degree of the velocity-pancake alignment is shown to increase with the velocity magnitude and the local density, while it decreases with the separation distance from the galaxy to the pancake and disappears when the pancake has a filamentary shape. A final conclusion is that our work may provide another clue to understanding the large-scale structure in the universe.