Affine Invariant Feature Research Articles

Automatic Production of Deep Learning Benchmark Dataset for Affine-Invariant Feature Matching

To promote the development of deep learning for feature matching, image registration, and three-dimensional reconstruction, we propose a method of constructing a deep learning benchmark dataset for affine-invariant feature matching. Existing images often have large viewpoint differences and areas with weak texture, which may cause difficulties for image matching, with respect to few matches, uneven distribution, and single matching texture. To solve this problem, we designed an algorithm for the automatic production of a benchmark dataset for affine-invariant feature matching. It combined two complementary algorithms, ASIFT (Affine-SIFT) and LoFTR (Local Feature Transformer), to significantly increase the types of matching patches and the number of matching features and generate quasi-dense matches. Optimized matches with uniform spatial distribution were obtained by the hybrid constraints of the neighborhood distance threshold and maximum information entropy. We applied this algorithm to the automatic construction of a dataset containing 20,000 images: 10,000 ground-based close-range images, 6000 satellite images, and 4000 aerial images. Each image had a resolution of 1024 × 1024 pixels and was composed of 128 pairs of corresponding patches, each with 64 × 64 pixels. Finally, we trained and tested the affine-invariant deep learning model, AffNet, separately on our dataset and the Brown dataset. The experimental results showed that the AffNet trained on our dataset had advantages, with respect to the number of matching points, match correct rate, and matching spatial distribution on stereo images with large viewpoint differences and weak texture. The results verified the effectiveness of the proposed algorithm and the superiority of our dataset. In the future, our dataset will continue to expand, and it is intended to become the most widely used benchmark dataset internationally for the deep learning of wide-baseline image matching.

Effects of Cognitive Distraction on Upper Limb Movement Decoding From EEG Signals.

Hand movement decoding from electroencephalograms (EEG) signals is vital to the rehabilitation and assistance of upper limb-impaired patients. Few existing studies on hand movement decoding from EEG signals consider any distractions. However, in practice, patients can be distracted while using the hand movement decoding systems in real life. In this paper, we aim to investigate the effects of cognitive distraction on movement decoding performance. We first propose a robust decoding method of hand movement directions to cognitive distraction from EEG signals by using the Riemannian Manifold to extract affine invariant features and Gaussian Naive Bayes classifier (named RM-GNBC). Then, we use the experimental and simulated EEG data under conditions without and with distraction to compare the decoding performance of three decoding methods (including the proposed method, tangent space linear discriminant analysis (TSLDA), and baseline method)). The simulation and experimental results show that the Riemannian-based methods (i.e., RM-GNBC and TSLDA) have higher accuracy under the conditions without and with cognitive distraction and smaller decreases in decoding accuracy between the conditions without and with cognitive distraction than the baseline method. Furthermore, the RM-GNBC method has 6% (paired t-test, p = 0.026) and 5% (paired t-test, p = 0.137) higher accuracies than the TSLDA method under the conditions without and with cognitive distraction, respectively. The results show that the Riemannian-based methods have higher robustness to cognitive distraction. This work contributes to developing a brain-computer interface (BCI) to improve the rehabilitation and assistance of hand-impaired patients in real life and open an avenue to the studies on the effects of distraction on other BCI paradigms.

Recognition and Tracking of Objects in a Clustered Remote Scene Environment

Object recognition and tracking are two of the most dynamic research sub-areas that belong to the field of Computer Vision. Computer vision is one of the most active research fields that lies at the intersection of deep learning and machine vision. This paper presents an efficient ensemble algorithm for the recognition and tracking of fixed shape moving objects while accommodating the shift and scale invariances that the object may encounter. The first part uses the Maximum Average Correlation Height (MACH) filter for object recognition and determines the bounding box coordinates. In case the correlation based MACH filter fails, the algorithms switches to a much reliable but computationally complex feature based object recognition technique i.e., affine scale invariant feature transform (ASIFT). ASIFT is used to accommodate object shift and scale object variations. ASIFT extracts certain features from the object of interest, providing invariance in up to six affine parameters, namely translation (two parameters), zoom, rotation and two camera axis orientations. However, in this paper, only the shift and scale invariances are used. The second part of the algorithm demonstrates the use of particle filters based Approximate Proximal Gradient (APG) technique to periodically update the coordinates of the object encapsulated in the bounding box. At the end, a comparison of the proposed algorithm with other state-of-the-art tracking algorithms has been presented, which demonstrates the effectiveness of the proposed algorithm with respect to the minimization of tracking errors.

A computationally efficient affine-invariant feature for image matching with wide viewing angles

This paper proposes an efficient affine-invariant feature image matching method, especially when there are wide viewing angles. In previous studies, all the AKAZE-based algorithms are built on nonlinear scale-space by using Fast Explicit Diffusion (FED) scheme, which has the advantage of high computational efficiency. But all of them do not have affine invariance and cannot favor the image matching when the image views are too different. Unlike AKAZE features, AFREAK features are extracted from the simulated images obtained by sampling the latitude and longitude angles of an original image, which have good robustness for viewpoint changes. Therefore, we combine the idea of simulating an affine transformation with the construction of a nonlinear scale-space to build an affine-invariant nonlinear scale-space. Then, in order to reduce the redundancy information, we optimize the sampling of longitude angles and simplify the FREAK descriptor. Moreover, a coarse-to-fine matching strategy based on Vector Field Consistency (VFC) and Progressive Sampling Consensus (PROSAC) is developed to further improve the rapidity and correct matching rate. Experimental results on various benchmark image datasets demonstrate that the proposed method is computationally efficient for image matching while has the fully affine invariance.

Matching Large Baseline Oblique Stereo Images Using an End-to-End Convolutional Neural Network

The available stereo matching algorithms produce large number of false positive matches or only produce a few true-positives across oblique stereo images with large baseline. This undesired result happens due to the complex perspective deformation and radiometric distortion across the images. To address this problem, we propose a novel affine invariant feature matching algorithm with subpixel accuracy based on an end-to-end convolutional neural network (CNN). In our method, we adopt and modify a Hessian affine network, which we refer to as IHesAffNet, to obtain affine invariant Hessian regions using deep learning framework. To improve the correlation between corresponding features, we introduce an empirical weighted loss function (EWLF) based on the negative samples using K nearest neighbors, and then generate deep learning-based descriptors with high discrimination that is realized with our multiple hard network structure (MTHardNets). Following this step, the conjugate features are produced by using the Euclidean distance ratio as the matching metric, and the accuracy of matches are optimized through the deep learning transform based least square matching (DLT-LSM). Finally, experiments on Large baseline oblique stereo images acquired by ground close-range and unmanned aerial vehicle (UAV) verify the effectiveness of the proposed approach, and comprehensive comparisons demonstrate that our matching algorithm outperforms the state-of-art methods in terms of accuracy, distribution and correct ratio. The main contributions of this article are: (i) our proposed MTHardNets can generate high quality descriptors; and (ii) the IHesAffNet can produce substantial affine invariant corresponding features with reliable transform parameters.

Formulation of Fractional Affine Feature Detector for TerraSAR-X (TSX) Images

The performance of affine invariant feature detector-Hessain Affine on Terra SAR-X images decreases in the presence of speckle noise due to false feature point detections and matches. The owing inherent characteristics of Terra SAR-X images like broad dynamic range and the multiplicative nature of speckle noise, the gradient magnitude is strengthened on homogeneous regions. Hence the false feature point detections on high contrast regions are not suppressed. In order to make feature detection robust towards the speckle noise, the use of improved version of Grunwald-Letnikov (G-L) fractional differential operator in the development of feature detection is investigated. This paved the way for a new affine invariant fractional order feature detector. This detector allows a stable and fine feature point selection without fine tuning parameters of the Hessian Affine feature detector. The algorithm proposed is tested on TerraSAR-X (TSX) images with different look angles. The results show that the proposed Fractional Affine algorithm improves the feature matching performance compared to Hessian Affine algorithm in terms of number of true correspondences and registration accuracy.

Quasi Fourier-Mellin Transform for Affine Invariant Features.

Fourier-Mellin transform (FMT) has been widely used for the extraction of rotation- and scale-invariant features. However, affine transform is a more reasonable approximation model for real viewpoint change. Due to shearing, the integral along the angular direction in the calculation of FMT cannot be used to extract the inherent features of an image undergoing affine transform. To eliminate the effect of shearing, whitening transform should be conducted on the integral along the radial direction. FMT can hardly be modified by conventional whitening-based methods with low computational cost due to additional processes. In this paper, two factors are constructed and embedded into FMT. Quasi Fourier-Mellin transform (QFMT) is proposed. The embedding of these factors is equivalent to whitening transform and can eliminate the effect of shearing in the affine transform. In particular, QFMT can also be calculated by integrating along the radial direction followed by integrating along the angular direction, as in FMT. Based on QFMT, the quasi Fourier-Mellin descriptor (QFMD) is constructed for the extraction of affine invariant features. Some experiments have also been conducted to test the performance of the proposed method.

A Grassmannian Graph Approach to Affine Invariant Feature Matching.

In this work, we present a novel, theoretical approach to address one of the longstanding problems in computer vision: 2D and 3D affine invariant feature matching. Our proposed Grassmannian Graph (GrassGraph) framework employs a two stage procedure that is capable of robustly recovering correspondences between two unorganized, affinely related feature (point) sets. In the ideal case, the first stage maps the feature sets to an affine invariant Grassmannian representation, where the features are mapped into the same subspace. It turns out that coordinate representations extracted from the Grassmannian differ by an arbitrary orthonormal matrix. In the second stage, by approximating the Laplace-Beltrami operator (LBO) on these coordinates, this extra orthonormal factor is nullified, providing true affine invariant coordinates which we then utilize to recover correspondences via simple mutual nearest neighbor relations. Our validation benchmarks use large number of experimental trials performed on 2D and 3D datasets. Experimental results show that the proposed GrassGraph method successfully recovers large affine transformations.

Automatic target recognition algorithm based on affine invariant feature of line grouping

In order to improve the accuracy of the multi-sensor automatic target recognition algorithm,based on line group matching, an image matching algorithm using linear group geometric primitives was proposed. Line group was used to represent the target. Every three lines in line group can compose a triangle. An affine invariant, which consisted of the ratio of these triangles' area, was considered as a distinct feature to describe the line group. The best matching line group was found among the candidate line groups in real-time infrared image. The corresponding points between target and real-time image can be obtained through the matching lines. Then the location of the target can be computed through affine transformation. Finally, the position of the target in the real-time map was detected by the same-named point; in addition, a method of linear purification in the real-time infrared image was proposed. The extracted primary lines provided a condition which ensured the effectiveness and efficiency of the proposed recognition algorithm. Experimental result shows that the proposed algorithm has a high matching accuracy in multi-sensor target recognition.

Registrating Oblique SAR Images Based on Complementary Integrated Filtering and Multilevel Matching

This paper presents a novel registration method for oblique synthetic aperture radar (SAR) images based on complementary integrated filtering (CIF) and multilevel matching. Our algorithm is divided into three steps. First, we considered different type of noises and employed the CIF to increase the signal-to-noise ratio of SAR images. Second, complementary affine invariant features, namely maximally stable extremal regions and Harris-affine features, were extracted simultaneously from image pairs, and then the initial matches were obtained based on the scale invariant feature transform (SIFT) descriptor and Euclidean distance. Therefore, the fundamental and homography matrixes could be calculated between image pairs, and then more matches were obtained under quasi-affine invariant SIFT (QAISIFT) and the hybrid geometric constraints. We further implemented the least square matching (LSM) based on the second-polynomial geometric model (SPGM), and thus the matching error of each corresponding point can be compensated according to the optimal SPGM. Third, the precise registration was achieved based on the matches of the second step. Experiments on four groups of oblique SAR images demonstrated the effectiveness of the proposed method. The contribution of this paper includes three aspects. One is that the proposed CIF can remove SAR image noise as much as possible; another is that the proposed QAISIFT can achieve near affine invariance across viewpoint change images; the third is that the advanced LSM can notably improve the accuracy of feature matches.

Aircraft Detection in Aerial Image Based on BOVW of Affine Invariance Features

Aircraft detection is one of the applications of high resolution remote-sensing images. Conventional method of aircraft detection use deep and complicated network, which needs highly computing cost. In this letter, an efficient and effective aircraft detection framework based on BOVW features and cascade AdaBoost classifier has been proposed. A variety of affine invariant features, which represent the complicate structure of aircraft, are extracted from sliding windows, and the direction estimation of aircraft is introduced to align the aircraft before detection. Besides, an accurate NMS algorithm is designed to make the target location more accurate. The proposed method is evaluated with recent advanced detection methods. The superior experimental outcome indicates that our framework achieves better accuracy as well as efficiency.

Affine invariant fusion feature extraction based on geometry descriptor and BIT for object recognition

It is difficult to recognise an image with affine transformation due to viewing angle and distance variations. Therefore, affine invariant feature extraction is a valuable technology in the field of image recognition. Inspired by bio-visual mechanism, an affine invariant for object recognition method based on a fusion feature framework is proposed in this study, which employs geometry descriptor and double biologically inspired transformation (DBIT). First, a shape feature of interest detector is adopted to detect contour features. Then, the area estimation of affine region detector is utilised to construct area ratio feature vectors. Second, an orientation edge detector is built to highlight the edges of different directions. On this basis, local space frequency detector is adopted to measure the spatial frequency at each direction and interval, which converts the output map into DBIT feature vectors. A weighted fusion strategy is performed based on Pearson correlation distance to fuse the geometry feature and DBIT feature. Some tests for Alphanumeric, Coil-100 MPEG-7, Mixed National Institute of Standards and Technology (MNIST) and Olivetti Research Laboratory face images database (ORL) database remain highly stable recognition accuracy, even when the shear factor is between -0.5 and + 0.5. The experiment results show the authors' proposed approach has a nice performance in feature invariance, selectivity and recognition accuracy.

Extreme Feature Regions Detection and Accurate Quality Assessment for Point-Cloud 3D Reconstruction

The 3-D reconstruction methods based on structure from motion (SfM) pipeline mainly use the traditional scale invariant feature detecting methods for feature matching. This type of methods suffers from the accuracy with affine matching in the image-based modeling system. In this paper, we propose an affine-invariant feature detection and matching method which is accurate and fast based on the three types of critical points in Morse theory called precise extreme feature region (PEFR). We also exploit the evaluation method for the reconstruction results without the ground truth. We propose the average descending rate to quantitatively evaluate the accuracy of the reconstruction model. Using PEFR in an SfM pipeline to build 3-D point-cloud, our method presents better results and less runtime compared to the state-of-the-art approaches. In addition, our quality assessment method presents a practical comparison of the reconstruction result based on the geometric accuracy of the reconstruction models. The experimental results prove that our PEFR method is faster and more accurate and works well on the reconstruction pipeline. The assessment method is fit for the quality assessment on 3-D reconstruction.

Evaluation of Local Feature Detectors for the Comparison of Thermal and Visual Low Altitude Aerial Images

Local features are key regions of an image suitable for applications such as image matching, and fusion. Detection of targets under varying atmospheric conditions, via aerial images is a typical defence application where multi spectral correlation is essential. Focuses on local features for the comparison of thermal and visual aerial images in this study. The state of the art differential and intensity comparison based features are evaluated over the dataset. An improved affine invariant feature is proposed with a new saliency measure. The performances of the existing and the proposed features are measured with a ground truth transformation estimated for each of the image pairs. Among the state of the art local features, Speeded Up Robust Feature exhibited the highest average repeatability of 57 per cent. The proposed detector produces features with average repeatability of 64 per cent. Future works include design of techniques for retrieval of corresponding regions.

PRELIMINARY INVESTIGATION OF THE ROBUSTNESS OF MAXIMALLY STABLE EXTREMAL REGIONS (MSER) MODEL FOR THE AUTOMATIC REGISTRATION OF OVERLAPPING IMAGES

Various researchers in Digital Image processing have developed keen interest in the automation of object detection, description and extraction process used for various applications and this has led to the development of series of Feature detection and extraction models one of which is the Maximally Stable Extremal Regions Feature Algorithm (MSER). This paper investigates the robustness of MSER algorithm (a blob-like and affine-invariant feature detector) for the detection and extraction of corresponding features used for the automatic registration of series of overlapping images. The robustness investigation was carried out in three different registration campaigns using overlapping images extracted from google earth and image pair acquired from an Unmanned Aerial Vehicle (UAV). Sum of Square Difference (SSD) and Bilinear interpolation models were used to establish the similarity measure between the images to be registered, resampling of the pixel-values and computation of non-integer coordinates respectively while Random Sampling Consensus (RANSAC) algorithm was used to exclude the outliers and to compute the transformation matrix using affine transformation function. The results obtained from this preliminary investigation shows that the processing speed of MSER is quite high for Automatic Image Registration with a relatively high accuracy. While an accuracy of 61.54% was obtained from the first campaign with a processing time of 11.92 seconds, the second campaign gave an accuracy of 52.02% with a processing time of 11.20 seconds and the third campaign produced an accuracy of 55.62% with a processing time of 3.27 seconds. The obtained speed and accuracy shows that MSER is a very robust model and as such, can be deployed as the feature detection and extraction model in the development of an automatic image registration scheme.

AIFD Based 2D Image Registration to Multi-View Stereo Mapped 3D Models

Multi-view stereo (MVS) map based 3D range reconstruction is to generate 3D ranges by analyzing the surrounding snapshots from different perspectives. Different to the traditional method which employing the expensive and difficult maintaining laser range devices to calibrate the range of the real 3D objects, MVS has achieved its success by seeking the geometrical correlations between the correspondences from the snapshot of different perspectives. The concerning of MVS keeps rising thanks to the fast development of digital maps and 3D printing. Several algorithms with regard to MVS has been well developed and achieved their success with regard to reconstruction of 3D ranges. Meanwhile, most of the algorithms were mainly focusing on the fusion and merging of different scenes and surface refinement. Less capability of the feature matching algorithms on the affine invariant images renders the current MVS algorithms need huge amount of images with tiny perspective differences. In this paper, we will propose a new MVS algorithm, deploying our previous published Affine Invariant Feature Descriptor (AIFD) to detect and match the correspondences from different perspectives and applying Homograph matrix and segmentation to define the planes of the objects. Thanks to the AIFD and Homograph based projection model, our proposed MVS algorithm outperform other MVS algorithms in terms of speed and efficiency.

Illumination invariant feature point matching for high-resolution planetary remote sensing images

Despite its success with regular close-range and remote-sensing images, the scale-invariant feature transform (SIFT) algorithm is essentially not invariant to illumination differences due to the use of gradients for feature description. In planetary remote sensing imagery, which normally lacks sufficient textural information, salient regions are generally triggered by the shadow effects of keypoints, reducing the matching performance of classical SIFT. Based on the observation of dual peaks in a histogram of the dominant orientations of SIFT keypoints, this paper proposes an illumination-invariant SIFT matching method for high-resolution planetary remote sensing images. First, as the peaks in the orientation histogram are generally aligned closely with the sub-solar azimuth angle at the time of image collection, an adaptive suppression Gaussian function is tuned to level the histogram and thereby alleviate the differences in illumination caused by a changing solar angle. Next, the suppression function is incorporated into the original SIFT procedure for obtaining feature descriptors, which are used for initial image matching. Finally, as the distribution of feature descriptors changes after anisotropic suppression, and the ratio check used for matching and outlier removal in classical SIFT may produce inferior results, this paper proposes an improved matching procedure based on cross-checking and template image matching. The experimental results for several high-resolution remote sensing images from both the Moon and Mars, with illumination differences of 20°–180°, reveal that the proposed method retrieves about 40%–60% more matches than the classical SIFT method. The proposed method is of significance for matching or co-registration of planetary remote sensing images for their synergistic use in various applications. It also has the potential to be useful for flyby and rover images by integrating with the affine invariant feature detectors.

A smart computing algorithm for finger vein matching with affine invariant features using fuzzy image retrieval

Finger vein recognition is a recently explored biometric system, renowned for its accuracy and reliability. Even though finger vein biometric system possesses exceptional qualities against other biometrics, it is quite challenging to extract enough number of reliable and consistent features for matching. Due to light scattering and other irregularities in the imaging procedures, finger vein images often possess random distortions and affine issues. In this paper, efficiency of affine invariant feature matching is investigated in the framework of finger vein biometrics. In order to reduce the time complexity of the affine invariant features, a size reduction of the database is attained through fuzzy based image retrieval. The experiment results shows better performance after the retrieval and much lower error rates in comparison with the existing feature matching algorithms.

Robust Harris Corner Matching Based on the Quasi-Homography Transform and Self-Adaptive Window for Wide-Baseline Stereo Images

Corner feature matching remains a difficult task for wide-baseline images because of viewpoint distortion, surface discontinuities, and partial occlusions. In this paper, we propose a robust Harris corner matching method based on the quasi-homography transform (QHT) and self-adaptive window. Our method is divided into three steps. First, high-quality Harris corners were extracted from stereo images using optimal detecting, and initial matches were simultaneously acquired by integrating complementary affine-invariant features and the scale-invariant feature transform descriptor. Second, the pair of fundamental matrices was estimated based on the initial matches and improved random sample consensus algorithm. Subsequently, the global QHT was produced by duplicate epipolar geometries. Third, conjugate Harris corners were obtained by combining QHT and normalized cross correlation, and the accuracy of the corresponding points was further improved based on self-adaptive least-squares matching (SALSM). Experiments on six groups of wide-baseline images demonstrate the effectiveness of the proposed method, and a comprehensive comparison with the existing corner matching algorithms indicates that our method has notable superiority in terms of accuracy and distribution. The main contribution of this paper is that the proposed global QHT can reduce the search range effectively for candidates, and the proposed SALSM can notably improve the accuracy of the corresponding corners.

Clustering of near duplicate images using bundled features

Clustering the images is generally based on the image’s visual features. Selection of relevant features is the most essential task. A clustering approach based on the bundled features is presented in this paper. Bundling of affine scale invariant feature transform (ASIFT) feature helps to cluster the near duplicates. When the local features are combined with the ASIFT features, the clustering efficiency is increased. Clustering the results from the web image search engines is very essential to help users narrow their search. We applied our idea of clustering with bundled features over Google Image search results. The results obtained show that the presented approach outperforms compared to the clustering done only with local features.