Pairwise Image Matching Research Articles

Efficient and Accurate Hierarchical SfM Based on Adaptive Track Selection for Large-Scale Oblique Images

Image-based 3D modeling has been widely used in many areas. Structure from motion is the key to image-based reconstruction. However, the rapid growth of data poses challenges to current SfM solutions. A hierarchical SfM reconstruction methodology for large-scale oblique images is proposed. Firstly, match pairs are selected using positioning and orientation (POS) data and the terrain of the survey area. Then, images are divided to image groups by traversing the selected match pairs. After pairwise image matching, tracks are decimated using an adaptive track selection method. Thirdly, submaps are reconstructed from the image groups in parallel based on incremental SfM in the object space. A novel method based on statistics of the positional difference between common tracks is proposed to detect the outliers in submap merging. Finally, the reconstructed submaps are incrementally merged and optimized. The proposed methodology was used on a large oblique image set. The proposed methodology was compared with the state-of-the-art image-based reconstruction systems COLMAP and Metashape for SfM reconstruction. Experimental results show that the proposed methodology achieved the highest accuracy on the experimental dataset, i.e., about 22.37, and 3.52 times faster than COLMAP and Metashape, respectively. The experimental results demonstrate that the proposed hierarchical SfM methodology is accurate and efficient for large-scale oblique images.

Low-Cost and Efficient Indoor 3D Reconstruction through Annotated Hierarchical Structure-from-Motion

With the widespread application of location-based services, the appropriate representation of indoor spaces and efficient indoor 3D reconstruction have become essential tasks. Due to the complexity and closeness of indoor spaces, it is difficult to develop a versatile solution for large-scale indoor 3D scene reconstruction. In this paper, an annotated hierarchical Structure-from-Motion (SfM) method is proposed for low-cost and efficient indoor 3D reconstruction using unordered images collected with widely available smartphone or consumer-level cameras. Although the reconstruction of indoor models is often compromised by the indoor complexity, we make use of the availability of complex semantic objects to classify the scenes and construct a hierarchical scene tree to recover the indoor space. Starting with the semantic annotation of the images, images that share the same object were detected and classified utilizing visual words and the support vector machine (SVM) algorithm. The SfM method was then applied to hierarchically recover the atomic 3D point cloud model of each object, with the semantic information from the images attached. Finally, an improved random sample consensus (RANSAC) generalized Procrustes analysis (RGPA) method was employed to register and optimize the partial models into a complete indoor scene. The proposed approach incorporates image classification in the hierarchical SfM based indoor reconstruction task, which explores the semantic propagation from images to points. It also reduces the computational complexity of the traditional SfM by avoiding exhausting pair-wise image matching. The applicability and accuracy of the proposed method was verified on two different image datasets collected with smartphone and consumer cameras. The results demonstrate that the proposed method is able to efficiently and robustly produce semantically and geometrically correct indoor 3D point models.

Optimal Illumination and Color Consistency for Optical Remote-Sensing Image Mosaicking

Illumination and color consistency are very important for optical remote-sensing image mosaicking. In this letter, we propose a simple but effective technique that simultaneously performs image illumination and color correction for multiview images. In this framework, we first present an uneven illumination removal algorithm based on bright channel prior, which guarantees the illumination consistency inside a single image. We then adapt a pairwise color-correction method to coarsely align the color tone between source and reference images. In this stage, we give a new single-image quality metric which combines brightness deviation, color cast, and entropy together for automatic reference-image selection. Finally, we perform a least-squares adjustment (LSA) procedure to obtain optimal illumination and color consistency among multiview images. In detail, we first perform a pairwise image matching by using SIFT algorithm; once sparse local patch correspondences obtained, the illumination and color relationship between images can be established based on a global gamma correction model; the illumination and color errors can then be minimized by LSA. Extensive experiments on both challenging synthetic and real optical remote-sensing image data sets show that it significantly outperforms the compared state-of-the-art approaches. All the source code and data sets used in this letter are made public. 1 1 https://sites.google.com/site/jiayuanli2016whu/home

Beyond Pairwise Matching: Person Reidentification via High-Order Relevance Learning.

Person reidentification has attracted extensive research efforts in recent years. It is challenging due to the varied visual appearance from illumination, view angle, background, and possible occlusions, leading to the difficulties when measuring the relevance, i.e., similarities, between probe and gallery images. Existing methods mainly focus on pairwise distance metric learning for person reidentification. In practice, pairwise image matching may limit the data for comparison (just the probe and one gallery subject) and yet lead to suboptimal results. The correlation among gallery data can be also helpful for the person reidentification task. In this paper, we propose to investigate the high-order correlation among the probe and gallery data, not the pairwise matching, to jointly learn the relevance of gallery data to the probe. Recalling recent progresses on feature representation in person reidentification, it is difficult to select the best feature and each type of feature can benefit person description from different aspects. Under such circumstances, we propose a multihypergraph joint learning algorithm to learn the relevance in corporation with multiple features of the imaging data. More specifically, one hypergraph is constructed using one type of feature and multiple hypergraphs can be generated accordingly. Then, the learning process is conducted on the multihypergraph structure, and the identity of a probe is determined by its relevance to each gallery data. The merit of the proposed scheme is twofold. First, different from pairwise image matching, the proposed method jointly explores the relationships among different images. Second, multimodal data, i.e., different features, can be formulated in the multihypergraph structure, which can convey more information in the learning process and can be easily extended. We note that the proposed method is a general framework to incorporate with any combination of features, and thus is flexible in practice. Experimental results and comparisons with the state-of-the-art methods on three public benchmarking data sets demonstrate the superiority of the proposed method.

Exploiting the Redundancy of Multiple Overlapping Aerial Images for Dense Image Matching Based Digital Surface Model Generation

In recent years, significant development in the domain of dense image matching (DIM) can be observed. Meanwhile, in most countries, aerial images are acquired countrywide on a regular basis with decreasing time intervals and increasing image overlaps. Therefore, aerial images represent a growing potential for digital surface model (DSM) acquisition and updating. Surface reconstruction by image matching, in most cases, requires dealing with the redundancy caused by multiple overlapping images. Many approaches considering this redundancy in the surface reconstruction process have been developed. However, there is no commonly accepted procedure for this task. From the experience of the author, it can be stated that currently applied methods show some limitations regarding DSM generation from aerial images. Therefore, it is claimed that there is room for the development of new algorithms for integration of dense image matching results from multiple stereo pairs. Methods dedicated to aerial image based DSM generation that would exploit the specificity of this task are desirable. In this paper, an approach to compute the DSM elevations from redundant elevation hypotheses derived by pairwise dense image matching is presented. The proposed approach takes into account the base-to-height (b/h) ratio of stereo pairs, the distribution of elevation hypotheses from multiple stereo pairs and the neighboring elevations. An algorithm of selection of the elevation hypotheses used for the calculation of the final DSM elevation for each grid cell was developed. The algorithm was used to generate the DSM based on two sets of aerial images having significantly different acquisition parameters. The results were compared to the models obtained from several commonly used software packages for image based DSM generation. The quality assessment was carried out by visual inspection of terrain profiles and shaded surface display as well as by the planarity control of flat parts of the terrain. The assessment of the results showed that the application of the proposed algorithm can bring some advantages and it can contribute to improving the quality of the DSM.

Bundling centre for landmark image discovery

This paper introduces a novel method to efficiently discover landmark images in large image collections. Each cluster is considered a combination of several sub-clusters, which are composed of images taken from different viewpoints of an identical landmark. For each sub-cluster, we find its local centre represented by a group of similar images and define it as the bundling centre (BC). Therefore, we start image discovery by identifying the BCs and accomplish the task by efficiently growing and merging those sub-clusters represented by different BCs. In our proposed method, we use a min-Hash-based method to build a sparse graph to avoid time-consuming, full-scale, exhaustive pairwise image matching. Based on the information provided by the sparse graph, BCs are identified as local dense neighbours sharing high intra-similarity. We have also proposed a weighted voting method to grow these BCs with high accuracy. More importantly, the fixed local centres ensure that each sub-cluster contains identical landmarks and generates results with high precision. In addition, compared to a single representative (iconic) image, the group of similar images obtained by each BC can provide more comprehensive cluster information and, thus, overcome the problem of low recall caused by information lost during visual word quantisation. We present the experimental results of three datasets and show that, without query expansion, our method can boost the landmark image discovery performances of current techniques.

Fast Feature‐Oriented Visual Connection for Large Image Collections

AbstractDeriving the visual connectivity across large image collections is a computationally expensive task. Different from current image‐oriented match graph construction methods which build on pairwise image matching, we present a novel and scalable feature‐oriented image matching algorithm for large collections. Our method improves the match graph construction procedure in three ways. First, instead of building trees repeatedly, we put the feature points of the input image collection into a single kd‐tree and select the leaves as our anchor points. Then we construct an anchor graph from which each feature can intelligently find a small portion of related candidates to match. Finally, we design a new form of adjacency matrix for fast feature similarity measuring, and return all the matches in different photos across the whole dataset directly. Experiments show that our feature‐oriented correspondence algorithm can explore visual connectivity between images with significant improvement in speed.

Object Detection Via Structural Feature Selection and Shape Model

In this paper, we propose an approach for object detection via structural feature selection and part-based shape model. It automatically learns a shape model from cluttered training images without need to explicitly use bounding boxes on objects. Our approach first builds a class-specific codebook of local contour features, and then generates structural feature descriptors by combining context shape information. These descriptors are robust to both within-class variations and scale changes. Through exploring pairwise image matching using fast earth mover's distance, feature weights can be iteratively updated. Those discriminative foreground features are assigned high weights and then selected to build a part-based shape model. Finally, object detection is performed by matching each testing image with this model. Experiments show that the proposed method is very effective. It has achieved comparable performance to the state-of-the-art shape-based detection methods, but requires much less training information.