Affine Scale Invariant Feature Transform Research Articles

Automatic Production of Deep Learning Benchmark Dataset for Affine-Invariant Feature Matching

To promote the development of deep learning for feature matching, image registration, and three-dimensional reconstruction, we propose a method of constructing a deep learning benchmark dataset for affine-invariant feature matching. Existing images often have large viewpoint differences and areas with weak texture, which may cause difficulties for image matching, with respect to few matches, uneven distribution, and single matching texture. To solve this problem, we designed an algorithm for the automatic production of a benchmark dataset for affine-invariant feature matching. It combined two complementary algorithms, ASIFT (Affine-SIFT) and LoFTR (Local Feature Transformer), to significantly increase the types of matching patches and the number of matching features and generate quasi-dense matches. Optimized matches with uniform spatial distribution were obtained by the hybrid constraints of the neighborhood distance threshold and maximum information entropy. We applied this algorithm to the automatic construction of a dataset containing 20,000 images: 10,000 ground-based close-range images, 6000 satellite images, and 4000 aerial images. Each image had a resolution of 1024 × 1024 pixels and was composed of 128 pairs of corresponding patches, each with 64 × 64 pixels. Finally, we trained and tested the affine-invariant deep learning model, AffNet, separately on our dataset and the Brown dataset. The experimental results showed that the AffNet trained on our dataset had advantages, with respect to the number of matching points, match correct rate, and matching spatial distribution on stereo images with large viewpoint differences and weak texture. The results verified the effectiveness of the proposed algorithm and the superiority of our dataset. In the future, our dataset will continue to expand, and it is intended to become the most widely used benchmark dataset internationally for the deep learning of wide-baseline image matching.

An Improved ASIFT Image Feature Matching Algorithm Based on POS Information.

The affine scale-invariant feature transform (ASIFT) algorithm is a feature extraction algorithm with affinity and scale invariance, which is suitable for image feature matching using unmanned aerial vehicles (UAVs). However, there are many problems in the matching process, such as the low efficiency and mismatching. In order to improve the matching efficiency, this algorithm firstly simulates image distortion based on the position and orientation system (POS) information from real-time UAV measurements to reduce the number of simulated images. Then, the scale-invariant feature transform (SIFT) algorithm is used for feature point detection, and the extracted feature points are combined with the binary robust invariant scalable keypoints (BRISK) descriptor to generate the binary feature descriptor, which is matched using the Hamming distance. Finally, in order to improve the matching accuracy of the UAV images, based on the random sample consensus (RANSAC) a false matching eliminated algorithm is proposed. Through four groups of experiments, the proposed algorithm is compared with the SIFT and ASIFT. The results show that the algorithm can optimize the matching effect and improve the matching speed.

SIRA: Scale illumination rotation affine invariant mask R-CNN for pedestrian detection.

In this paper, we resolve the challenging obstacle of detecting pedestrians with the ubiquity of irregularities in scale, rotation, and the illumination of the natural scene images natively. Pedestrian instances with such obstacles exhibit significantly unique characteristics. Thus, it strongly influences the performance of pedestrian detection techniques. We propose the new robust Scale Illumination Rotation and Affine invariant Mask R-CNN (SIRA M-RCNN) framework for overcoming the predecessor’s difficulties. The first phase of the proposed system deals with illumination variation by histogram analysis. Further, we use the contourlet transformation, and the directional filter bank for the generation of the rotational invariant features. Finally, we use Affine Scale Invariant Feature Transform (ASIFT) to find points that are translation and scale-invariant. Extensive evaluation of the benchmark database will prove the effectiveness of SIRA M-RCNN. The experimental results achieve state-of-the-art performance and show a significant performance improvement in pedestrian detection.

Recognition and Tracking of Objects in a Clustered Remote Scene Environment

Object recognition and tracking are two of the most dynamic research sub-areas that belong to the field of Computer Vision. Computer vision is one of the most active research fields that lies at the intersection of deep learning and machine vision. This paper presents an efficient ensemble algorithm for the recognition and tracking of fixed shape moving objects while accommodating the shift and scale invariances that the object may encounter. The first part uses the Maximum Average Correlation Height (MACH) filter for object recognition and determines the bounding box coordinates. In case the correlation based MACH filter fails, the algorithms switches to a much reliable but computationally complex feature based object recognition technique i.e., affine scale invariant feature transform (ASIFT). ASIFT is used to accommodate object shift and scale object variations. ASIFT extracts certain features from the object of interest, providing invariance in up to six affine parameters, namely translation (two parameters), zoom, rotation and two camera axis orientations. However, in this paper, only the shift and scale invariances are used. The second part of the algorithm demonstrates the use of particle filters based Approximate Proximal Gradient (APG) technique to periodically update the coordinates of the object encapsulated in the bounding box. At the end, a comparison of the proposed algorithm with other state-of-the-art tracking algorithms has been presented, which demonstrates the effectiveness of the proposed algorithm with respect to the minimization of tracking errors.

Multimodal Registration of Optical Images and Digital Elevation Models Using Terrain Features

Registration of multimodal images such as optical data and digital elevation models (DEM) is a challenging task due to significant non-linear differences between these images. To address the problem, this paper proposes a new robust approach to integrate optical images and DEMs by using terrain features. In order to detect these features, a simulated image is generated based on the DEM by illuminating it in the geometry of the optical image acquisition, such that typical textures induced by topography are well present as those in the optical image. Hence, the multimodal registration is performed on the optical data and the simulated image to maximize the accuracy of feature matching. The Affine Scale-Invariant Feature Transform (ASIFT) is used to detect keypoints from the two images. Correspondences are matched through the nearest neighbor distance ratio, and outliers are removed using a two-step elimination procedure. The proposed method has been tested on five pairs of optical-DEM images with various spatial resolutions. Experimental results have shown that this method can provide robust registration for optical-DEM images with high accuracies of sub pixel level. The novelty of proposed method is to make use of terrain features for registration, providing a new perspective for the integration of multimodal images.

Establishing a large amount of point correspondences using patch-based affine-scale invariant feature transform for fisheye images

Affine-scale invariant feature transform (ASIFT) has performed very well for perspective images, but the intrinsic nonlinear distortion makes it difficult to utilize ASIFT directly for fisheye images to find a large amount of correspondences. We reuse ASIFT and propose a new feature matching method for uncorrected fisheye images. First, we employ the Gaussian hemi-image to divide fisheye images into patches. In this process, we introduce Plücker coordinates and side operator to establish the correspondences between the hemispherical projection model, Gaussian hemi-image, and fisheye images. Simultaneously, maximum stable extreme region algorithm is used to detect the target regions. After that, each patch in these regions is simulated at different orientation parameters in ASIFT, and SIFT algorithm is applied to all simulated patches. Experiments on real-world images show that the proposed method can achieve good performance: the numbers of the point correspondences increase greatly with satisfactory accuracy, reliability, and efficiency.

Feature Matching Combining Radiometric and Geometric Characteristics of Images, Applied to Oblique- and Nadir-Looking Visible and TIR Sensors of UAV Imagery.

A large amount of information needs to be identified and produced during the process of promoting projects of interest. Thermal infrared (TIR) images are extensively used because they can provide information that cannot be extracted from visible images. In particular, TIR oblique images facilitate the acquisition of information of a building’s facade that is challenging to obtain from a nadir image. When a TIR oblique image and the 3D information acquired from conventional visible nadir imagery are combined, a great synergy for identifying surface information can be created. However, it is an onerous task to match common points in the images. In this study, a robust matching method of image pairs combined with different wavelengths and geometries (i.e., visible nadir-looking vs. TIR oblique, and visible oblique vs. TIR nadir-looking) is proposed. Three main processes of phase congruency, histogram matching, and Image Matching by Affine Simulation (IMAS) were adjusted to accommodate the radiometric and geometric differences of matched image pairs. The method was applied to Unmanned Aerial Vehicle (UAV) images of building and non-building areas. The results were compared with frequently used matching techniques, such as scale-invariant feature transform (SIFT), speeded-up robust features (SURF), synthetic aperture radar–SIFT (SAR–SIFT), and Affine SIFT (ASIFT). The method outperforms other matching methods in root mean square error (RMSE) and matching performance (matched and not matched). The proposed method is believed to be a reliable solution for pinpointing surface information through image matching with different geometries obtained via TIR and visible sensors.

Enhanced Facial Emotion Recognition by Optimal Descriptor Selection with Neural Network

Facial Emotion Recognition (FER) is the approach of detecting emotions of humans from facial expressions. Emotions are detected automatically by the human brain and hence, many computer aided techniques are implemented for recognizing emotions. Recognition of natural emotions is an exciting area with a broad range of real time applications like automated tutoring models, smart environments computer–human interaction, driver warning systems, and video and image retrieval. The major goal of this paper is to implement an intelligent model for FER. The proposed model involves few steps: (a) Face extraction (b) Image filtering, (c) Extraction of facial components, (d) Descriptor selection, and (d) Classification. Initially, the face is extracted from the input image by the Viola–Jones method, which is generally employed for the detection of object. Further, the noise of the image is filtered by the Gabor Filtering. Further, the facial components are extracted as features via the Affine-Scale-Invariant Feature Transform (ASIFT), which is the modified version of SIFT method. Since the length of the descriptors generated from ASIFT is high; the number of descriptors is reduced by the optimal descriptor selection approach with the aid of hybrid meta-heuristic algorithm termed as MV-WOA. The extracted descriptors are subjected to Neural Network (NN). As a modification to the existing machine learning algorithms, the number of hidden neurons in the NN is optimized by the proposed WOA+MVO algorithm. By conducting experiments, the results demonstrate that the developed system can perform better the recent traditional approaches for emotion recognition for classifying seven emotions like normal, smile, sad, surprise, anger, fear, and disgust.

Nature Scene Signs Recognition by Affine Scale-Invariant Feature Transform

Nature Scene Signs Recognition by Affine Scale-Invariant Feature Transform

Feature Matching and Heat Flow in Centro-Affine Geometry

In this paper, we study the differential invariants and the invariant heat flow in centro-affine geometry, proving that the latter is equivalent to the inviscid Burgers' equation. Furthermore, we apply the centro-affine invariants to develop an invariant algorithm to match features of objects appearing in images. We show that the resulting algorithm compares favorably with the widely applied scale-invariant feature transform (SIFT), speeded up robust features (SURF), and affine-SIFT (ASIFT) methods.

Image retrieval based on ASIFT features in a Hadoop clustered system

For image matching, the scale invariant feature transform (SIFT) algorithm is a commonly used one. They are invariant to image rotation, scale zooming, and partially invariant to change in illumination and 3D camera viewpoint. Affine SIFT (ASIFT) is an extension of SIFT, which solves the problem when images are captured at different angles. However, ASIFT has higher computational complexity than SIFT, due to a huge amount of features in the images. Therefore, in this study, a Hadoop-based image retrieval system is proposed to solve the ASIFT shortcomings of high computation by the MapReduce technology. The system uses a combination of the Bag-of-Words method and support vector machine. Finally, the experimental results verify that the proposed method is more effective than the other state-of-the-art methods for a variety of datasets.

Geometrical Matching of SAR and Optical Images Utilizing ASIFT Features for SAR-based Navigation Aided Systems.

This article presents a new approach to the estimation of shift and rotation between two images from different kinds of imaging sensors. The first of the image is an orthophotomap that is created using optical sensors with georeference information. The second one is created utilizing a Synthetic Aperture Radar (SAR) sensor.The proposed solution can be mounted on a flying platform, and, during the flight, the obtained SAR images are compared with the reference optical images, and thus it is possible to calculate the shift and rotation between these two images and then the direct georeferencing error. Since both images have georeference information, it is possible to calculate the navigation correction in cases when the drift of the calculated trajectory is expected. The method can be used in platforms where there is no satellite navigation signal and the trajectory is calculated on the basis of an inertial navigation system, which is characterized by a significant error. The proposed method of estimating the navigation error utilizing Affine Scale-Invariant Feature Transform (ASIFT) and Structure from Motion (SfM) is described, and techniques for improving the quality of SAR imaging using despeckling filters are presented. The methodology was tested and verified using real-life SAR images. Differences between the results obtained for a few selected despeckling methods were compared and commented on. Deep investigation of the nature of the SAR imaging technique and noise creation character allows new algorithms to be developed, which can be implemented on flying platforms to support existing navigation systems in which trajectory error occurs.

Artwork painting identification method for panorama based on adaptive rectilinear projection and optimized ASIFT

In the paper, the authors present an artwork painting identification method for panorama based on adaptive rectilinear projection and optimized ASIFT (Affine Scale-Invariant Feature Transform). Firstly, the authors use the panorama dataset to train the artwork painting detection network to obtain the location information of artwork paintings. Secondly, the authors use the adaptive rectilinear projection to map the artwork painting into a square image with a fixed size. Then the authors use the image enhancement method to improve the image quality. Finally, the authors use the optimized ASIFT for features extraction and image matching. Several contrast experiments were conducted on the artwork paintings panorama dataset for artwork paintings identification. The results show that the proposed method can achieve 96% identification accuracy on average for the whole test artwork paintings panorama dataset. The proposed adaptive rectilinear based-method can improve at least 20% of the recognition accuracy. The proposed optimized ASIFT can improve at least 30% of the identification accuracy than SIFT. The authors also study other factors such as the size of the original artwork image, the image matching threshold, whether using image enhancement or not. The results show the size of the original artwork has little influence on the artwork identification in the panorama. The image matching threshold with 2.0 is better than 3.0. Furthermore, using the image enhancement method can improve about 2% of the identification accuracy.

Affine invariant image watermarking scheme based on ASIFT and Delaunay tessellation

Watermarking algorithms that resist geometric combination attacks is currently a trend of research in this field. This paper proposed a geometrically invariant watermarking scheme based on the feature points of resistance to affine transformation. The scheme used Affine Scale Invariant Feature Transform (ASIFT) to detect the feature points of the carrier image and filtrated the established feature points group depending on anti-affine properties. The obtained feature points set obtained better robustness against multiple attacks. As for robustness, we utilized the feature points set to construct the Delaunay tessellation and embedded the watermark information in the histogram of each triangle. Because of the stability of Delaunay tessellation and the basic characteristics of affine transformation, the geometric robustness of the proposed scheme achieved a good effect as was verified in experiments.

High-precision co-registration of orbiter imagery and digital elevation model constrained by both geometric and photometric information

The alignment of images to a reference digital elevation model (DEM) has many applications, especially in planetary exploration. In this paper, we propose a novel high-precision co-registration method for pixel-based matching between an image and a reference DEM to rectify the ever-increasing number of orbiter images to DEMs in a unified reference frame automatically. First, the DEM is converted to a simulated image using a hillshading technique based on a photometric model and the illumination conditions of the image. Then, an initial matching between the simulated and input images is performed based on affine scale-invariant feature transform (ASIFT). The rational function model (RFM) of the image is established and used as a geometric constraint. Then, with tie points (TPs) generated by the initial ASIFT matching, the RFM geometric model of the image is refined, and the gross errors in the TP set are iteratively eliminated. Finally, a high-precision co-registration by pixel-based Least-Squares (LS) image matching is performed using the refined geometric model. Lunar Reconnaissance Orbiter (LRO) Narrow Angle Camera (NAC) and SLDEM2015 (a combined product of Lunar Reconnaissance Orbiter Laser Altimeter (LOLA) and a DEM generated by the Japanese Selenological and Engineering Explorer (SELENE) terrain camera) images are used in our experiment. Results using 17 NAC images at different latitudes demonstrate that the proposed co-registration method is effective and can attain an accuracy of 18.3 pixels in the image space and 25.17 m (0.5 pixel of the reference DEM) in the object space. The method offers an automatic and high-precision matching of LRO NAC images to SLDEM2015. It is also applicable to the co-registration of other orbital images to a reference DEM.

Unmanned aerial vehicle oblique image registration using an ASIFT-based matching method

Low-altitude unmanned aerial vehicles (UAV) are widely used to acquire aerial photographs, some of which are oblique and have a large angle of view. Precise, automatic registration of such images is a challenge for conventional image processing methods. We present an affine scale-invariant feature transform (ASIFT)-based method that can register UAV oblique images at a subpixel level. First, we used the ASIFT algorithm to collect initial feature points. Positions of the feature points on corresponding local images were then corrected using the weighted least square matching (WLSM) method. Mismatching points were discarded and a local transform model was estimated using the adaptive normalized cross correlation algorithm, which also provides initial parameters for WLSM. Experiments show that sufficient feature points are collected to successfully register, to the subpixel level, UAV and other images with large angle-of-view variations and strong affine distortions. The proposed method improves the matching accuracy of previous UAV image registration methods.

Robust Image Watermarking Algorithm Based on ASIFT against Geometric Attacks

Image processing technology has been developed rapidly in recent years, and altering the content of an image is easy for everyone, but may be illegal for an attacker. Thus, it is urgent and necessary to overcome this problem to protect the integrity and authenticity of images. Watermarking is a powerful technique proposed to solve this problem. This paper introduces a robust image watermarking algorithm working in the wavelet domain, embedding the watermark information into the third level low frequency coefficients after the three-level discrete wavelet transform (DWT) and singular value decomposition (SVD). Additionally, to improve the robustness to geometric attacks, the affine-scale-invariant feature transform (ASIFT) is applied to obtain feature points which are invariant to geometric attacks. Then, features of acquired points between the watermarked image and the received image are used to realize the resynchronization to improve the robustness. Experimental results show that the proposed algorithm achieves great balance between robustness and imperceptibility, and is robust against geometric attacks, JPEG compression, noise addition, cropping, median filters, and so on.

Comparing the performance and accuracy of algorithms applied to tattoos images identification

This article presents results of the simulation of SIFT based algorithms in the context of the identification of tattoos. The algorithms studied are the SIFT - Scale Invariant Feature Transform, ASIFT - Affine SIFT, BOV - Bag of Visual Words and FV - Fisher Vector. The use of the OPF - Optimum-Path Forest and SVM - Support Vector Machine classifiers is exploited in conjunction with SIFT and ASIFT algorithms as well as BOV and FV. The present study uses the National Institute of Standards and Technology (NIST) Tatt-C dataset in a reduced and complete version. This work uses runtime and accuracy to compare the results of the simulations.

Clustering of near duplicate images using bundled features

Clustering the images is generally based on the image’s visual features. Selection of relevant features is the most essential task. A clustering approach based on the bundled features is presented in this paper. Bundling of affine scale invariant feature transform (ASIFT) feature helps to cluster the near duplicates. When the local features are combined with the ASIFT features, the clustering efficiency is increased. Clustering the results from the web image search engines is very essential to help users narrow their search. We applied our idea of clustering with bundled features over Google Image search results. The results obtained show that the presented approach outperforms compared to the clustering done only with local features.

Automatic Matching of Multi-Source Satellite Images: A Case Study on ZY-1-02C and ETM+

The ever-growing number of applications for satellites is being compromised by their poor direct positioning precision. Existing orthoimages, such as enhanced thematic mapper (ETM+) orthoimages, can provide georeferences or improve the geo-referencing accuracy of satellite images, such ZY-1-02C images that have unsatisfactory positioning precision, thus enhancing their processing efficiency and application. In this paper, a feasible image matching approach using multi-source satellite images is proposed on the basis of an experiment carried out with ZY-1-02C Level 1 images and ETM+ orthoimages. The proposed approach overcame differences in rotation angle, scale, and translation between images. The rotation and scale variances were evaluated on the basis of rational polynomial coefficients. The translation vectors were generated after blocking the overall phase correlation. Then, normalized cross-correlation and least-squares matching were applied for matching. Finally, the gross errors of the corresponding points were eliminated by local statistic vectors in a TIN structure. Experimental results showed a matching precision of less than two pixels (root-mean-square error), and comparison results indicated that the proposed method outperforms Scale-Invariant Feature Transform (SIFT), Speeded Up Robust Features (SURF), and Affine-Scale Invariant Feature Transform (A-SIFT) in terms of reliability and efficiency.