Dense Correspondence Research Articles

Structured-Patch Optimization for Dense Correspondence

This paper presents a new method to compute the dense correspondences between two images by using the energy optimization and the structured patches. In terms of the property of the sparse feature and the principle that nearest sub-scenes and neighbors are much more similar, we design a new energy optimization to guide the dense matching process and find the reliable correspondences. The sparse features are also employed to design a new structure to describe the patches. Both transformation and deformation with the structured patches are considered and incorporated into an energy optimization framework. Thus, our algorithm can match the objects robustly in complicated scenes. Finally, a local refinement technique is proposed to solve the perturbation of the matched patches. Experimental results demonstrate that our method outperforms the state-of-the-art matching algorithms.

Synchronous phase demodulation algorithm for conic carrier Hartmann topographer

In this paper, a theoretical analysis of a new algorithm for measuring phase objects (PO) is presented. The algorithm analyses images captured with a modified Hartmann plate. The proposed method demodulates a signal with a conic-wavefront carrier by using synchronous interferometric techniques. Typically a Hartmann plate is a mask with an array of holes, the proposal is to replace those holes with a series of circular concentric rings created by a conic carrier, named Conic Hartmann plate mask (CHM). The proposed algorithm computes a dense correspondence between the pattern source and its interferogram on the PO. This dense correspondence is computed as the phase change of the refracted waveform with respect to the reference one. This approach is more robust than standard techniques that computes the center of each spot in order to estimate its position; which involves pattern recognition and segmentation tasks, something normal in a typical Hartmann test. We show in this work that the new method overcomes different problems such as irregular background illumination, spots overlapping with pupil border and spots matching due to rapid changes on the surface slopes. This work shows a general procedure for radial derivatives integration based on base functions. As far as the authors know, the here presented demonstration has not been reported before. In particular, we show that Zernike polynomials are suitable for PO. Numerical experiments with real and simulated data demonstrate the algorithm performance.

Let's face it--complex traits are just not that simple.

Let's face it--complex traits are just not that simple.

Establishing a multidisciplinary context for modeling 3D facial shape from DNA.

Establishing a multidisciplinary context for modeling 3D facial shape from DNA.

3D facial landmarks: Inter-operator variability of manual annotation.

BackgroundManual annotation of landmarks is a known source of variance, which exist in all fields of medical imaging, influencing the accuracy and interpretation of the results. However, the variability of human facial landmarks is only sparsely addressed in the current literature as opposed to e.g. the research fields of orthodontics and cephalometrics. We present a full facial 3D annotation procedure and a sparse set of manually annotated landmarks, in effort to reduce operator time and minimize the variance.MethodFacial scans from 36 voluntary unrelated blood donors from the Danish Blood Donor Study was randomly chosen. Six operators twice manually annotated 73 anatomical and pseudo-landmarks, using a three-step scheme producing a dense point correspondence map. We analyzed both the intra- and inter-operator variability, using mixed-model ANOVA. We then compared four sparse sets of landmarks in order to construct a dense correspondence map of the 3D scans with a minimum point variance.ResultsThe anatomical landmarks of the eye were associated with the lowest variance, particularly the center of the pupils. Whereas points of the jaw and eyebrows have the highest variation. We see marginal variability in regards to intra-operator and portraits. Using a sparse set of landmarks (n=14), that capture the whole face, the dense point mean variance was reduced from 1.92 to 0.54 mm.ConclusionThe inter-operator variability was primarily associated with particular landmarks, where more leniently landmarks had the highest variability. The variables embedded in the portray and the reliability of a trained operator did only have marginal influence on the variability. Further, using 14 of the annotated landmarks we were able to reduced the variability and create a dense correspondences mesh to capture all facial features.

Multiple Shape Correspondence by Dynamic Programming

AbstractWe present a multiple shape correspondence method based on dynamic programming, that computes consistent bijective maps between all shape pairs in a given collection of initially unmatched shapes. As a fundamental distinction from previous work, our method aims to explicitly minimize the overall distortion, i.e., the average isometric distortion of the resulting maps over all shape pairs. We cast the problem as optimal path finding on a graph structure where vertices are maps between shape extremities. We exploit as much context information as possible using a dynamic programming based algorithm to approximate the optimal solution. Our method generates coarse multiple correspondences between shape extremities, as well as denser correspondences as by‐product. We assess the performance on various mesh sequences of (nearly) isometric shapes. Our experiments show that, for isometric shape collections with non‐uniform triangulation and noise, our method can compute relatively dense correspondences reasonably fast and outperform state of the art in terms of accuracy.

Guest Editorial: Tracking, Detection and Segmentation

This special issue contains papers selected by representatives of the British Machine Vision Association (BMVA) describing recent advances made against major outstanding challenges in computer vision: detection; tracking and pose estimation; activity recognition; video segmentation; dense correspondence and video synthesis. Establishing a dense correspondence between images remains an open problem; recent solutions based on randomised search have yielded promising advances in accuracy and efficiency. Our first paper, “PMBP: PatchMatch Belief Propagation for Correspondence Field Estimation” presents a unified framework in which the Patch Match algorithm is first described as a variant of belief propagation, and then extended using features of the latter to yield results that are both more accurate than the original Patch Match formulation and orders of magnitude faster than previous belief propagation approaches. ‘Mixture ofTrees ProbabilisticGraphicalModel forVideo Segmentation” by Badrinarayanan, Budvytis and Cipolla, continues the theme of video analysis; proposing a new spatio-temporal representation, the Mixture of Trees for temporally coherent semantic video segmentation. This

An Efficient Approach to Correspondences between Multiple Non‐Rigid Parts

AbstractIdentifying multiple deformable parts on meshes and establishing dense correspondences between them are tasks of fundamental importance to computer graphics, with applications to e.g. geometric edit propagation and texture transfer. Much research has considered establishing correspondences between non‐rigid surfaces, but little work can both identify similar multiple deformable parts and handle partial shape correspondences. This paper addresses two related problems, treating them as a whole: (i) identifying similar deformable parts on a mesh, related by a non‐rigid transformation to a given query part, and (ii) establishing dense point correspondences automatically between such parts. We show that simple and efficient techniques can be developed if we make the assumption that these parts locally undergo isometric deformation. Our insight is that similar deformable parts are suggested by large clusters of point correspondences that are isometrically consistent. Once such parts are identified, dense point correspondences can be obtained by an iterative propagation process. Our techniques are applicable to models with arbitrary topology. Various examples demonstrate the effectiveness of our techniques.

THE ITALIAN ARBERESH AND ALBANIAN NATIONAL CASE

This scientific research through the accumulation of facts, historical resources and their interpretation aims to give a full panorama of the whole contribution of the Italian arberesh to the Albanian case at the end of the XIX-th century until the first decade of the XX-th century. Exactly by beginning at the end of the 19th century in the south of Italy it will begin to loom an Albanian literature with an authentic national content. The well known representatives of such a literature would be important figures in the rows of the Arberesh Italian – Albanian community who had achieved to conserve a clear Albanian language and traditions. Also the Italian-Albanian periodic press like “Fiamuri Eve”, “The Albanian Nation” that were created during this period would become the forums for the disclosure of their political thoughts and ideas about the fate and the future of the Albanians that lived under the rule of the Otoman Empire at the end of the 19th century. Among the initiators of such a movement at this period would be Jeronin De Rada, Dhimiter Kamarda, Zef Skiroi etc. Their activity would be displayed through their wide multi-aspect creativity which is evidenced in the literature, scientific, cultural, publicist political and patriotic aspect. Of a particular importance would be considered their ideas and projects in the funtion of creating an autonomous Albanian Vilayet. All their activities will serve a single purpose and will developed in order to support the solution of the Albanian nation political case and will find the full support not only of the Italian-Albanian society but also of the other well known figures and personalities of the Italian politics. The Italian Arberesh used all their possibilities to give an international dimension to the Albanian case as a factor of stability in the Balkan region and beyond. In order to express the obove problematic we have exploited a rich archival literature represented by the dense correspondence of De Rada with other known personalities of that time such as the Romanian princess with Albanian origin Dora D’istria (Elena Gjika), we have used the political Testament of De Rada, by viewing it in a comparative context with the studies of other Albanians and foreign authors. DOI: 10.5901/mjss.2014.v5n19p651

Three-dimensional lunar surface reconstruction based on remote sensing imagery from Chang E-2

In this paper, we propose a complete solution for effective, automatic and accurate reconstruction of the lunar surface based on the linear array push-broom imagery from Chang E-2 (CE-2). First, with the sparse ephemeris data, an approach for estimating the corresponding areas between forward (F) imagery and backward (B) imagery is proposed through exploiting the imaging characteristics of linear push-broom cameras. Second, feature based matching (FBM) is conducted between F and B imagery, followed by Area Based Matching (ABM) for dense correspondence. Third, the extrinsic parameters of every scan line are estimated according to the supplied sparse ephemeris, and the orientation matrix is derived. Finally, the digital ortho map (DOM) and digital elevation model (DEM) of one orbit are generated automatically. The results show that the relative and absolute accuracies of triangulation are relatively high and generally acceptable. The visualised 3D realistic scene and the mosaicked DOM and DEM of lunar surface validate the feasibility of the proposed solution.

Efficient hybrid tree-based stereo matching with applications to postcapture image refocusing.

Estimating dense correspondence or depth information from a pair of stereoscopic images is a fundamental problem in computer vision, which finds a range of important applications. Despite intensive past research efforts in this topic, it still remains challenging to recover the depth information both reliably and efficiently, especially when the input images contain weakly textured regions or are captured under uncontrolled, real-life conditions. Striking a desired balance between computational efficiency and estimation quality, a hybrid minimum spanning tree-based stereo matching method is proposed in this paper. Our method performs efficient nonlocal cost aggregation at pixel-level and region-level, and then adaptively fuses the resulting costs together to leverage their respective strength in handling large textureless regions and fine depth discontinuities. Experiments on the standard Middlebury stereo benchmark show that the proposed stereo method outperforms all prior local and nonlocal aggregation-based methods, achieving particularly noticeable improvements for low texture regions. To further demonstrate the effectiveness of the proposed stereo method, also motivated by the increasing desire to generate expressive depth-induced photo effects, this paper is tasked next to address the emerging application of interactive depth-of-field rendering given a real-world stereo image pair. To this end, we propose an accurate thin-lens model for synthetic depth-of-field rendering, which considers the user-stroke placement and camera-specific parameters and performs the pixel-adapted Gaussian blurring in a principled way. Taking ~1.5 s to process a pair of 640×360 images in the off-line step, our system named Scribble2focus allows users to interactively select in-focus regions by simple strokes using the touch screen and returns the synthetically refocused images instantly to the user.

Effective Volumetric Feature Modeling and Coarse Correspondence via Improved 3DSIFT and Spectral Matching

This paper presents a nonrigid coarse correspondence computation algorithm for volumetric images. Our matching algorithm first extracts then correlates image features based on a revised and improved 3DSIFT (I3DSIFT) algorithm. With a scale-related keypoint reorientation and descriptor construction, this feature correlation is less sensitive to image rotation and scaling. Then, we present an improved spectral matching (ISM) algorithm on correlated features to obtain a one-to-one mapping between corresponded features. One can effectively extend this feature correspondence to dense correspondence between volume images. Our algorithm can benefit nonrigid volumetric image registration in many tasks such as motion modeling in medical image analysis and processing.

Inverse image editing

We study the problem of inverse image editing , which recovers a semantically-meaningful editing history from a source image and an edited copy. Our approach supports a wide range of commonly-used editing operations such as cropping, object insertion and removal, linear and non-linear color transformations, and spatially-varying adjustment brushes. Given an input image pair, we first apply a dense correspondence method between them to match edited image regions with their sources. For each edited region, we determine geometric and semantic appearance operations that have been applied. Finally, we compute an optimal editing path from the region-level editing operations, based on predefined semantic constraints. The recovered history can be used in various applications such as image re-editing, edit transfer, and image revision control. A user study suggests that the editing histories generated from our system are semantically comparable to the ones generated by artists.

Three-dimensional panoramic terrain reconstruction from aerial imagery

A complete solution for effectively, automatically, and accurately reconstructing the three-dimensional (3-D) panoramic terrain from aerial imagery is presented. With enough premeasured and identified georeferences, we first estimate every camera’s accurate intrinsic and extrinsic parameters by implementing bundle adjustment, which is introduced in detail. Afterward, the adjacent relationship of imagery is acquired from the cameras’ position parameters. In addition, the formulas for corresponding area prediction and image rectification are derived according to the camera parameters. Subsequently, feature-based matching is conducted between adjacent image pairs to provide much more constraints for bundle adjustment. Area-based matching is applied to pairs of horizontal epipolar imagery for dense correspondence to produce dense spatial point cloud. Eventually, the mosaicked digital ortho map and digital elevation model of the whole imaging area are produced automatically by a series of steps including spatial intersection, Tin generation, differential correction, and color blending. Experimental results show that the root mean square (RMS) residual errors of check points in planimetry and altitude are, respectively, 0.039 and 0.170 m, demonstrating the high accuracy of camera orientation. The visualized panoramic 3-D realistic scene validates the feasibility and effectiveness of the proposed solution.

Shape Matching via Quotient Spaces

AbstractWe introduce a novel method for non‐rigid shape matching, designed to address the symmetric ambiguity problem present when matching shapes with intrinsic symmetries. Unlike the majority of existing methods which try to overcome this ambiguity by sampling a set of landmark correspondences, we address this problem directly by performing shape matching in an appropriate quotient space, where the symmetry has been identified and factored out. This allows us to both simplify the shape matching problem by matching between subspaces, and to return multiple solutions with equally good dense correspondences. Remarkably, both symmetry detection and shape matching are done without establishing any landmark correspondences between either points or parts of the shapes. This allows us to avoid an expensive combinatorial search present in most intrinsic symmetry detection and shape matching methods. We compare our technique with state‐of‐the‐art methods and show that superior performance can be achieved both when the symmetry on each shape is known and when it needs to be estimated.

Robust Simultaneous Registration and Segmentation with Sparse Error Reconstruction

We introduce a fast and efficient variational framework for Simultaneous Registration and Segmentation (SRS) applicable to a wide variety of image sequences. We demonstrate that a dense correspondence map (between consecutive frames) can be reconstructed correctly even in the presence of partial occlusion, shading, and reflections. The errors are efficiently handled by exploiting their sparse nature. In addition, the segmentation functional is reformulated using a dual Rudin-Osher-Fatemi (ROF) model for fast implementation. Moreover, nonparametric shape prior terms that are suited for this dual-ROF model are proposed. The efficacy of the proposed method is validated with extensive experiments on both indoor, outdoor natural and biological image sequences, demonstrating the higher accuracy and efficiency compared to various state-of-the-art methods.

Coarse‐to‐Fine Isometric Shape Correspondence by Tracking Symmetric Flips

AbstractWe address the symmetric flip problem that is inherent to multi‐resolution isometric shape matching algorithms. To this effect, we extend our previous work which handles the dense isometric correspondence problem in the original 3D Euclidean space via coarse‐to‐fine combinatorial matching. The key idea is based on keeping track of all optimal solutions, which may be more than one due to symmetry especially at coarse levels, throughout denser levels of the shape matching process. We compare the resulting dense correspondence algorithm with state‐of‐the‐art techniques over several 3D shape benchmark datasets. The experiments show that our method, which is fast and scalable, is performance‐wise better than or on a par with the best performant algorithms existing in the literature for isometric (or nearly isometric) shape correspondence. Our key idea of tracking symmetric flips can be considered as a meta‐approach that can be applied to other multi‐resolution shape matching algorithms, as we also demonstrate by experiments.

Geodesic distances to landmarks for dense correspondence on ensembles of complex shapes.

Establishing correspondence points across a set of biomedical shapes is an important technology for a variety of applications that rely on statistical analysis of individual subjects and populations. The inherent complexity (e.g. cortical surface shapes) and variability (e.g. cardiac chambers) evident in many biomedical shapes introduce significant challenges in finding a useful set of dense correspondences. Application specific strategies, such as registration of simplified (e.g. inflated or smoothed) surfaces or relying on manually placed landmarks, provide some improvement but suffer from limitations including increased computational complexity and ambiguity in landmark placement. This paper proposes a method for dense point correspondence on shape ensembles using geodesic distances to a priori landmarks as features. A novel set of numerical techniques for fast computation of geodesic distances to point sets is used to extract these features. The proposed method minimizes the ensemble entropy based on these features, resulting in isometry invariant correspondences in a very general, flexible framework.

Matching Cost Filtering for Dense Stereo Correspondence

Dense stereo correspondence enabling reconstruction of depth information in a scene is of great importance in the field of computer vision. Recently, some local solutions based on matching cost filtering with an edge-preserving filter have been proved to be capable of achieving more accuracy than global approaches. Unfortunately, the computational complexity of these algorithms is quadratically related to the window size used to aggregate the matching costs. The recent trend has been to pursue higher accuracy with greater efficiency in execution. Therefore, this paper proposes a new cost-aggregation module to compute the matching responses for all the image pixels at a set of sampling points generated by a hierarchical clustering algorithm. The complexity of this implementation is linear both in the number of image pixels and the number of clusters. Experimental results demonstrate that the proposed algorithm outperforms state-of-the-art local methods in terms of both accuracy and speed. Moreover, performance tests indicate that parameters such as the height of the hierarchical binary tree and the spatial and range standard deviations have a significant influence on time consumption and the accuracy of disparity maps.

Robust Dense Registration of Partial Nonrigid Shapes

This paper presents a complete and robust solution for dense registration of partial nonrigid shapes. Its novel contributions are founded upon the newly proposed heat kernel coordinates (HKCs) that can accurately position points on the shape, and the priority-vicinity search that ensures geometric compatibility during the registration. HKCs index points by computing heat kernels from multiple sources, and their magnitudes serve as priorities of queuing points in registration. We start with shape features as the sources of heat kernels via feature detection and matching. Following the priority order of HKCs, the dense registration is progressively propagated from feature sources to all points. Our method has a superior indexing ability that can produce dense correspondences with fewer flips. The diffusion nature of HKCs, which can be interpreted as a random walk on a manifold, makes our method robust to noise and small holes avoiding surface surgery and repair. Our method searches correspondence only in a small vicinity of registered points, which significantly improves the time performance. Through comprehensive experiments, our new method has demonstrated its technical soundness and robustness by generating highly compatible dense correspondences.