Wrong Matches Research Articles

OSSIM: An Object-Based Multiview Stereo Algorithm Using SSIM Index Matching Cost

Multiview stereo (MVS) is a crucial process in image-based automatic 3-D reconstruction and mapping applications. In a dense matching process, the matching cost is generally computed between image pairs, making the efficiency low due to the large number of stereo pairs. This paper presents a novel object-based MVS algorithm using structural similarity (SSIM) index matching cost in a coarse-to-fine workflow. As far as we know, this is the first time SSIM index is introduced to calculate the matching cost of MVS applications. In contrast to classical stereo methods, the proposed object-based structural similarity (OSSIM) method computes only a depth map for each image. Thus, the efficiency can be greatly improved when the overlap between images is large. To obtain an optimized depth map, the winner-take-all and semi-global matching strategies are implemented. Moreover, an object-based multiview consistency checking strategy is also proposed to eliminate wrong matches and perform pixelwise view selection. The proposed method was successfully applied on a close-range Fountain-P11 data set provided by EPFL and aerial data sets of Vaihingen and Zurich by the ISPRS. Experimental results demonstrate that the proposed method can deliver matches at high completeness and accuracy. For the Vaihingen data set, the correctness and completeness rate were 71.12% and 95.99% with an RMSE of 2.8 GSD. For the Foutain-P11 data set, the proposed method outperformed the other existing methods with the ratio of pixels less than 2 cm. Extensive comparison using Zurich data set shows that it can derive results comparable to the state-of-the-art software (PhotoScan, Pix4d, and Smart3D) in urban buildings areas.

40. Validation of a transit EPID device for a clinical use: application to iViewDose (Elekta)

Introduction IMRT/VMAT and stereotactic treatments require patient specific quality assurance before and during treatment. In that context, Electronic Portal Imaging Device (EPID) can be used for pre-treatment and in vivo verification. The aim of this study was to evaluate a new in vivo portal dosimetry tool iViewDose (Elekta) for a clinical use. Methods The study was performed on a VersaHD LINAC equipped with Agility collimator and EPID iView v.3.4.1 (Elekta). iViewDose v.1.0.1 (Elekta) software allows users to reconstruct delivered dose by the LINAC in 3D on the planning CT scan using EPID images. Treatment plans were calculated with Pinnacle v.9.10 (Philips) for VMAT technique. Agreement between calculation and measurement was evaluated with gamma index analysis in 3D ( γ 3D, criteria: 3% global/3 mm, threshold 50%). Detector validation was realized in two steps: (i) detector intrinsic characteristics evaluation (repeatability, reproducibility, linearity, dose rate dependence) and (ii) evaluation of error detectability for two treatment plan complexities (prostate and Head & Neck) delivered on an anthropomorphic phantom with: leaves shift from 1 to 5 mm, collimator rotation from 1 to 5° and MU increase from 1% to 5%. The in vivo agreement between calculation and measurement was investigated for nine patients (H&N, liver, esophagus, iliac bone) corresponding in a total of 86 fractions. According to the obtained results, a parallel analysis with CBCT images was carried out. Results Dose response of the detector was linear (R2 = 1). A x ¯ / σ of 0.06% was obtained for measurements performed with a dose rate varying from 70 to 500 MU/min. For a same irradiation, reconstructed dose was repeatable ( x ¯ / σ = 0.07%) and reproducible ( x ¯ / σ = 0.2%). For H&N treatment plan, iViewDose could detect a 1 mm bank shift ( γ 3D percentage decreased from 91.7% to 67.6%), a 2° collimator rotation ( γ 3D percentage decreased to 76.8%) and an increase of 1% MU ( γ 3D percentage decreased to 84.6%). A two bank shift in the same direction was not detected. Similar results were obtained for prostate treatment plans. In vivo calculation/measurement coherence for five patients (liver, esophagus, iliac bone) showed on average a γ 3D percentage of 90.9% (30 fractions). Those measurements showed punctually a wrong matching for a liver treatment case near diaphragm ( Fig. 1 b). For H&N treatment plans, in vivo calculation/measurement coherence is deteriorated during treatment: 84.4% of points have a gamma Fig. 1 a). Conclusions This study shows the iViewDose capability to detect LINAC and patient potential errors. With that kind of detector, it becomes necessary to correlate those results to a clinical impact and to define action thresholds justifying treatment re-planning. It could be very useful in an adaptive radiotherapy and would be more accurate with dose calculation on CBCT images of the day.

Study on Low Illumination Simultaneous Polarization Image Registration Based on Improved SURF Algorithm

Registration of simultaneous polarization images is the premise of subsequent image fusion operations. However, in the process of shooting all-weather, the polarized camera exposure time need to be kept unchanged, sometimes polarization images under low illumination conditions due to too dark result in SURF algorithm can not extract feature points, thus unable to complete the registration, therefore this paper proposes an improved SURF algorithm. Firstly, the luminance operator is used to improve overall brightness of low illumination image, and then create integral image, using Hession matrix to extract the points of interest to get the main direction of characteristic points, calculate Haar wavelet response in X and Y directions to get the SURF descriptor information, then use the RANSAC function to make precise matching, the function can eliminate wrong matching points and improve accuracy rate. And finally resume the brightness of the polarized image after registration, the effect of the polarized image is not affected. Results show that the improved SURF algorithm can be applied well under low illumination conditions.

Fundamental Principles on Learning New Features for Effective Dense Matching.

In dense matching (including stereo matching and optical flow), nearly all existing approaches are based on simple features, such as gray or RGB color, gradient or simple transformations like census, to calculate matching costs. These features do not perform well in complex scenes that may involve radiometric changes, noises, overexposure and/or textureless regions. Various problems may appear, such as wrong matching at the pixel or region level, flattening/breaking of edges and/or even entire structural collapse. In this paper, we propose two fundamental principles based on the consistency and the distinctiveness of features. We show that almost all existing problems in dense matching are caused by features that violate one or both of these principles. To systematically learn good features for dense matching, we develop a general multi-objective optimization based on these two principles and apply convolutional neural networks to find new features that lie on the Pareto frontier. By using two-frame optical flow and stereo matching as applications, our experimental results show that the features learned can significantly improve the performance of state-of-the-art approaches. Based on the KITTI benchmarks, our method ranks first on the two stereo benchmarks and is the best among existing two-frame optical-flow algorithms on flow benchmarks.

A MULTI-CORE PARALLEL MOSAIC ALORITHM FOR MULTI-VIEW UAV IMAGES

Abstract. As the spread of the error and accumulation often lead to distortion or failure of image mosaic during the multi-view UAV (Unmanned Aerial Vehicle) images stitching. In this paper, to solve the problem we propose a mosaic strategy to construct a mosaic ring and multi-level grouping parallel acceleration as an auxiliary. First, the input images will be divided into several groups, each group in the ring way to stitch. Then, use SIFT for matching, RANSAC to remove the wrong matching points. And then, calculate the perspective transformation matrix. Finally weaken the error by using the adjustment equation. All these steps run between different groups at the same time. By using real UAV images, the experiment results show that this method can effectively reduce the influence of accumulative error, improve the precision of mosaic and reduce the mosaic time by 60 %. The proposed method can be used as one of the effective ways to minimize the accumulative error.

3D Reconstruction of Space Objects from Multi-Views by a Visible Sensor.

In this paper, a novel 3D reconstruction framework is proposed to recover the 3D structural model of a space object from its multi-view images captured by a visible sensor. Given an image sequence, this framework first estimates the relative camera poses and recovers the depths of the surface points by the structure from motion (SFM) method, then the patch-based multi-view stereo (PMVS) algorithm is utilized to generate a dense 3D point cloud. To resolve the wrong matches arising from the symmetric structure and repeated textures of space objects, a new strategy is introduced, in which images are added to SFM in imaging order. Meanwhile, a refining process exploiting the structural prior knowledge that most sub-components of artificial space objects are composed of basic geometric shapes is proposed and applied to the recovered point cloud. The proposed reconstruction framework is tested on both simulated image datasets and real image datasets. Experimental results illustrate that the recovered point cloud models of space objects are accurate and have a complete coverage of the surface. Moreover, outliers and points with severe noise are effectively filtered out by the refinement, resulting in an distinct improvement of the structure and visualization of the recovered points.

ACCURATE ESTIMATION OF ORIENTATION PARAMETERS OF UAV IMAGES THROUGH IMAGE REGISTRATION WITH AERIAL OBLIQUE IMAGERY

Abstract. Unmanned Aerial Vehicles (UAVs) have gained popularity in acquiring geotagged, low cost and high resolution images. However, the images acquired by UAV-borne cameras often have poor georeferencing information, because of the low quality on-board Global Navigation Satellite System (GNSS) receiver. In addition, lightweight UAVs have a limited payload capacity to host a high quality on-board Inertial Measurement Unit (IMU). Thus, orientation parameters of images acquired by UAV-borne cameras may not be very accurate. Poorly georeferenced UAV images can be correctly oriented using accurately oriented airborne images capturing a similar scene by finding correspondences between the images. This is not a trivial task considering the image pairs have huge variations in scale, perspective and illumination conditions. This paper presents a procedure to successfully register UAV and aerial oblique imagery. The proposed procedure implements the use of the AKAZE interest operator for feature extraction in both images. Brute force is implemented to find putative correspondences and later on Lowe’s ratio test (Lowe, 2004) is used to discard a significant number of wrong matches. In order to filter out the remaining mismatches, the putative correspondences are used in the computation of multiple homographies, which aid in the reduction of outliers significantly. In order to increase the number and improve the quality of correspondences, the impact of pre-processing the images using the Wallis filter (Wallis, 1974) is investigated. This paper presents the test results of different scenarios and the respective accuracies compared to a manual registration of the finally computed fundamental and essential matrices that encode the orientation parameters of the UAV images with respect to the aerial images.

Bidirectional scale-invariant feature transform feature matching algorithms based on priority k-d tree search

In this article, a bidirectional feature matching algorithm and two extended algorithms based on the priority k-d tree search are presented for the image registration using scale-invariant feature transform features. When matching precision of image registration is below 50%, the discarding wrong match performance of many robust fitting methods like Random Sample Consensus (RANSAC) is poor. Therefore, improving matching precision is a significant work. Generally, a feature matching algorithm is used once in the image registration system. We propose a bidirectional algorithm that utilizes the priority k-d tree search twice to improve matching precision. There are two key steps in the bidirectional algorithm. According to the case of adopting the ratio restriction of distances in the two key steps, we further propose two extended bidirectional algorithms. Experiments demonstrate that there are some special properties of these three bidirectional algorithms, and the two extended algorithms can achieve higher precisions than previous feature matching algorithms.

Person Reidentification via Ranking Aggregation of Similarity Pulling and Dissimilarity Pushing

Person reidentification is a key technique to match different persons observed in nonoverlapping camera views. Many researchers treat it as a special object-retrieval problem, where ranking optimization plays an important role. Existing ranking optimization methods mainly utilize the similarity relationship between the probe and gallery images to optimize the original ranking list, but seldom consider the important dissimilarity relationship. In this paper, we propose to use both similarity and dissimilarity cues in a ranking optimization framework for person reidentification. Its core idea is that the true match should not only be similar to those strongly similar galleries of the probe, but also be dissimilar to those strongly dissimilar galleries of the probe. Furthermore, motivated by the philosophy of multiview verification, a ranking aggregation algorithm is proposed to enhance the detection of similarity and dissimilarity based on the following assumption: the true match should be similar to the probe in different baseline methods. In other words, if a gallery blue image is strongly similar to the probe in one method, while simultaneously strongly dissimilar to the probe in another method, it will probably be a wrong match of the probe. Extensive experiments conducted on public benchmark datasets and comparisons with different baseline methods have shown the great superiority of the proposed ranking optimization method.

Video error concealment scheme based on tensor model

Since compressed video sequences may be corrupted or lost when transmitted over error-prone networks, error concealment techniques are very important for video communication. As a video sequence is a group of high-dimensional data, it can be considered as a big 3 r d -order tensor. The methodologies that matricize high-dimensional data and then apply matrix-based method for further analysis often cause a loss of the internal structure information. Therefore, we built a tensor model to process such data, in order to preserve the natural multilinear structure. The key idea of our tensor model includes two parts. The first part is to construct a small tensor consist of the corrupted block and its several reference blocks. This part could be accomplished by block matching, but the traditional block matching might give a wrong match when the corrupted area is large and continuous. In order to overcome such situation, we proposed a flexible block matching scheme (FBM). The second part is to figure out the data in the corrupted part by tensor low rank approximation. Unlike the traditional low rank approximation, we did not fix the rank of core tensor as a constant number. Instead, the rank is flexible in our method, which is adaptive to the situation. Compared with some other error concealment method in the experiments, our method is able to achieve significantly higher PSNR (Peak Signal to Noise Ratio) as well as better visual quality.

Hierarchical Filtering Strategy for Registration of Remote Sensing Images of Coral Reefs

Registration of remote sensing images of coral reefs is basis for detection and analysis of changes to coral reefs, which is difficult due to inadequate and unstable texture information. Affine invariant features matching (AIFM) method, combining maximally stable extremal region (MSER) detector and scale invariant feature transformation (SIFT) descriptor, followed by optimizing using RANdom SAmple Consensus (RANSAC), still cannot lead to satisfactory results of image registration. To address this problem, we propose a hierarchical filtering strategy for image registration, which is composed of two stages. Filter 1, constructed based on a geometric transformation model determined by the corresponding contours of coral reefs, is employed to remove the wrong matching pairs which obviously dissatisfy spatial distributions of matching features. Based on the filtering results of Filters 1 and 2, with an appropriate threshold value determined by overlapping ratio of the corresponding coral reefs, can lead to further optimized results for image registration. This threshold is used to filter all erroneous matching pairs, while retaining as many correct matching pairs as possible. To test the effect of these filters, we design two experiments to compare the four approaches: AIFM without filtering, AIFM with Filter 1, AIFM with Filter 2, and AIFM with hierarchical filtering ( ${\bf{Filter}}\;{\bf{1}} + {\bf{Filter}}\;{\bf{2}}$ ). The experimental results demonstrate that the approach with hierarchical filtering performs much better than the other approaches.

Register Multimodal Images of Large Scene Depth Variation with Global Information

This paper addresses the problem of registering multimodal images of scene depth variation. The existing methods typically build matches of keypoints with descriptors and then apply consensus/consistency check to rule out incorrect matches. However, the consistency check often fails to work when there are a large number of wrong matches. Given a set of initial matches built with descriptors, we seek to search the best or correctly matched keypoints. To this end, this work employs the global information over entire images to assess the quality of keypoint matches. Since the image content has depth variation, projection transformations are needed to account for the misalignment and hence quadruples of keypoint matches are considered. In order to search the correctly matched keypoints, an iterative process is used that considers all preserved quadruples passing the spatial coherence constraint. Extensive experimental results on various image data show that the proposed method outperforms the state-of-the-art methods.

基于角点特征的遥感图像快速配准

针对遥感图像传统配准算法匹配速度较慢、不满足实时性要求等问题，本文提出了一种结合改进AGAST与FREAK算法的遥感图像快速配准方法。首先，利用改进AGAST检测算法分别快速检测参考图像和待配准图像中的特征点；然后用FREAK算法获取二进制描述符串，利用级联匹配计算特征向量之间的汉明距离，获得特征点匹配对；最后利用改进的相似三角形剔除方法去掉错误的匹配对，并结合最小二乘法，估算出空间几何变换参数，实现两幅图像的配准。实验结果表明，本文方法在保证遥感图像配准精度的同时，配准速度相比于传统配准方法得到较大提升。 In this paper, because the traditional registration algorithms for remote sensing image are slower and don’t meet the requirements of real-time problem, a new method is proposed based on the combination of improved AGAST and FREAK for fast remote sensing images registration. Firstly, the improved AGAST is used to detect the feature points between reference image and image that is to be registered; Secondly, FREAK algorithm is used to obtain a binary string descriptor, and hamming distance between features vector is computed by using a cascade match to get matching feature points; Finally, wrong match pairs are eliminated by using the improved similar triangle method, and the optimal spatial geometric transform parameters are estimated using the least square method to accomplish the two images registration. Experimental results show that the pro- posed method improves the registration rate compared to the traditional registration methods, and ensures accuracy at the same time.

A TSR Visual Servoing System Based on a Novel Dynamic Template Matching Method.

The so-called Tethered Space Robot (TSR) is a novel active space debris removal system. To solve its problem of non-cooperative target recognition during short-distance rendezvous events, this paper presents a framework for a real-time visual servoing system using non-calibrated monocular-CMOS (Complementary Metal Oxide Semiconductor). When a small template is used for matching with a large scene, it always leads to mismatches, so a novel template matching algorithm to solve the problem is presented. Firstly, the novel matching algorithm uses a hollow annulus structure according to a FAST (Features from Accelerated Segment) algorithm and makes the method be rotation-invariant. Furthermore, the accumulative deviation can be decreased by the hollow structure. The matching function is composed of grey and gradient differences between template and object image, which help it reduce the effects of illumination and noises. Then, a dynamic template update strategy is designed to avoid tracking failures brought about by wrong matching or occlusion. Finally, the system synthesizes the least square integrated predictor, realizing tracking online in complex circumstances. The results of ground experiments show that the proposed algorithm can decrease the need for sophisticated computation and improves matching accuracy.

Online learning of dynamic multi-view gallery for person Re-identification

Person re-identification receives increasing attentions in computer vision due to its potential applications in video surveillance. In order to alleviate wrong matches caused by misalignment or missing features among cameras, we propose to learn a multi-view gallery of frequently appearing objects in a relatively closed environment. The gallery contains appearance models of these objects from different cameras and viewpoints. The strength of the learned appearance models lies in that they are invariant to viewpoint and illumination changes. To automatically estimate the number of frequently appearing objects in the environment and update their appearance models online, we propose a dynamic gallery learning algorithm. We specifically build up two datasets to validate the effectiveness of our approach in realistic scenarios. Comparisons with benchmark methods demonstrate promising performance in accuracy and efficiency of re-identification.

结合Harris与SIFT算子的图像快速配准算法

A new method for fast image registration based on improved Harris-Sift algorithm is proposed. Firstly,classic Harris algorithm is improved by building Gaussian scale space to extract scale invariant Harris corners and they are refined to sub-pixel corners using Forsnter algorithm. Then the SIFT descriptor is utilized to characterize those feature points and the matching procedure is carried out via randomized kd trees. At last,RANSAC is used to remove wrong matches and the optimal transform parameters are estimated using the least square method to accomplish the image registration process. The experimental results demonstrate that compared with the classic SIFT algorithm the proposed method decreases the cost time of the registration procedure mostly by 64% while almost keeping the same performance.

Feature matching of fancy weft knitted stitch based on modified SURF algorithm

In this paper, based on Hessian invariant moments, a new approach for classification of fancy weft knitted stitch was proposed. The speeded up robust features (SURF) algorithm was used to identify fancy weft knitted samples including cable stitch and tuck stitch. Weft knitted stitch images generally contain many repeated features, wrong matches are easily encountered when the SURF algorithm is applied to match images. To resolve this problem, the inflection point of misjudgment was found out. The general regularity of faults and the key threshold were concluded by analyzing a number of classification experiments. The results show that wrong matches can be removed by locking the inflection point and feature matching of fancy weft knitted stitch is proved to be feasible for assigning an unknown image to one of a set of known texture classes.

Digital Image Stabilization Based on Improved Scale Invariant Feature Transform

A novel digital image stabilization approach using Harris and Scale Invariant Feature Transform (SIFT) was presented in this article. Using SIFT in digital image stabilization, too many feature points and matches were extracted, but some of them were not so stable. Using these feature points and matches can not only increase the computational effort, but also enhance the wrong matching probability. We proposed to use SIFT to detect feature points and incorporate the Harris criterion to select the most stable feature points in the video sequence where image motion was happened due to vehicle or platform vibration. With these feature points, we use general feature descriptor and matching algorithm to achieve the image stabilization. Experimental results show that the proposed algorithm can not only bring down the probability of wrong matching and get more accurate matches, but also reduce the computation burden than SIFT effectively.

Registration of Brain Medical Images Based on SURF Algorithm and R-RANSAC Algorithm

This paper proposes a matching method for medical image registration, which combined with SURF(Speeded up Robust Features)algorithm and the improved R-RANSAC(the Randomized of Random Sample Consensus) algorithm. Firstly, this algorithm extracts featured points with SURF algorithm from images and matches similar featured points with Euclidean distance. Secondly, the R-RANSAC algorithm is used to eliminate wrong matches and the SPRT (Sequential Probability Ratios Test) is used to minimize R-RANSAC runtime. Finally, the image registration process is accomplished by estimating space geometric varied parameters according to least square method. The algorithm combines robustness and high efficiency of SURF and high-accuracy of R-RANSAC algorithm. Experimental results show that in the condition of images with noise, non-uniform intensity and large scope of the initial misalignment, the proposed algorithm achieves better robustness and higher speed while maintaining good registration accuracy compared with the conventional area-based and feature-based registration methods. DOI : http://dx.doi.org/10.11591/telkomnika.v12i3.4500 Full Text: PDF

Automatic Registration of Coastal Remotely Sensed Imagery by Affine Invariant Feature Matching with Shoreline Constraint

A new approach based on Affine Invariant Feature Matching (AIFM) with a filtering technique is proposed for automatic registration of remotely sensed image in coastal areas. The novelty of this approach is an automatic filtering technique using RANdom SAmple Consensus (RANSAC) with shoreline constraint for AIFM to remove all wrong matches and simultaneously keep as many correct matches as possible. To implement it, a progressive threshold strategy (from small value to large value) is presented to determine an appropriate RANSAC threshold, in which the progressive process is guided by shoreline constraint. The proposed approach (with filtering) is compared with standard AIFM (without filtering) using two typical image pairs in coastal areas. The experimental results indicate that the proposed approach can always provide much better matching results than standard AIFM.