Improved Feature Matching Research Articles

Binocular Visual Measurement Method Based on Feature Matching.

To address the issues of low measurement accuracy and unstable results when using binocular cameras to detect objects with sparse surface textures, weak surface textures, occluded surfaces, low-contrast surfaces, and surfaces with intense lighting variations, a three-dimensional measurement method based on an improved feature matching algorithm is proposed. Initially, features are extracted from the left and right images obtained by the binocular camera. The extracted feature points serve as seed points, and a one-dimensional search space is established accurately based on the disparity continuity and epipolar constraints. The optimal search range and seed point quantity are obtained using the particle swarm optimization algorithm. The zero-mean normalized cross-correlation coefficient is employed as a similarity measure function for region growing. Subsequently, the left and right images are matched based on the grayscale information of the feature regions, and seed point matching is performed within each matching region. Finally, the obtained matching pairs are used to calculate the three-dimensional information of the target object using the triangulation formula. The proposed algorithm significantly enhances matching accuracy while reducing algorithm complexity. Experimental results on the Middlebury dataset show an average relative error of 0.75% and an average measurement time of 0.82 s. The error matching rate of the proposed image matching algorithm is 2.02%, and the PSNR is 34 dB. The algorithm improves the measurement accuracy for objects with sparse or weak textures, demonstrating robustness against brightness variations and noise interference.

Solving photogrammetric cold cases using AI-based image matching: New potential for monitoring the past with historical aerial images

With the ongoing digitization in archives, an increasing number of historical data becomes available for research. This includes historical aerial images which provide detailed information about the depicted area. Among the applications enabled by these images are change detection of land use, land cover, glaciers, and coastal environments as well as the observation of land degradation, and natural hazards. Studying the depicted areas and occurring 3D deformations requires the generation of a digital surface model (DSM) which is usually obtained via photogrammetric Structure-from-Motion (SfM). However, conventional SfM workflows often fail in registering historical aerial images due to their radiometric characteristics introduced by digitization, original image quality, or vast temporal changes between epochs. We demonstrate that the feature matching step in the Structure from Motion (SfM) pipeline is particularly crucial. To address this issue, we apply the two synergetic neural network methods SuperGlue and DISK, improving feature matching for historical aerial images. This requires several modifications to enable rotational invariance and leveraging the high resolution of aerial images. In contrast to other studies our workflow does not require any prior information such as DSMs, flight height, focal lengths, or scan resolution which are often no more extent in archives. It is shown that our methods using adapted parameter settings are even able to deal with quasi texture-less images. This enables the simultaneous processing of various kind of mono-temporal and multi-temporal data handled in a single workflow from data preparation over feature matching through to camera parameter estimation and the generation of a sparse point cloud. It outperforms conventional strategies in the number of correct feature matches, number of registered images and calculated 3D points and allows the generation of multi-temporal DSMs with high quality.With the flexibility of the method, it enables the automatic processing of formerly unusable or only to be interactively processed data, e.g. aerial images where the flight route is unknown, or with difficult radiometric properties. This makes it possible to go back even further in time, where the data quality usually decreases, and enables a holistic monitoring and comparison of environments of high interest. The code is made publicly available at https://github.com/tudipffmgt/HAI-SFM.

Study on Optimization Method of Visual Odometry Based on Feature Matching

The mismatching of image features affects the calculation of the fundamental matrix and then leads to poor estimation accuracy of SLAM visual odometry. Aiming at the above problems, a visual odometry optimization method based on feature matching is proposed. Firstly, the initial matching set is roughly filtered by the minimum distance threshold method, and then, the relative transformation relationship between images is calculated by the RANSAC algorithm. If it conforms to the transformation relationship, it is an interior point. The iteration result with most interior points is the correct matching result. Then, the homography transformation between images is calculated, and the fundamental matrix is calculated by it. The interior points are determined by epipolar geometric constraints, and the fundamental matrix with the most interior points is obtained. Finally, the effects of the visual odometry optimization algorithm are verified by the TUM data set from two aspects: feature matching and fundamental matrix calculation. The experimental results show that an improved feature matching algorithm can effectively remove mismatched feature points while improving the operation efficiency. At the same time, the accuracy of feature point matching is increased by 15.8%. The fundamental matrix estimation algorithm not only improves the calculation accuracy of the fundamental matrix but also increases the interior point rate by 11.9%. A theoretical basis for improving the accuracy estimation of visual odometry will be provided.

Real‐time structural displacement estimation by fusing asynchronous acceleration and computer vision measurements

AbstractAlthough displacement measurement is essential for many civil infrastructure applications, the precise estimation of structural displacement remains a challenge. In this study, a structural displacement estimation technique was developed by fusing asynchronous acceleration and computer vision measurements using a Kalman filter. First, the scale factor, which converts translation from vision measurements (in pixel units) into displacement (in length units), is automatically computed using a natural target (i.e., without any artificial target or any prior knowledge of the target size). Second, an improved feature matching algorithm was developed to better trace the natural target within the computer vision. Third, an adaptive multirate Kalman filter was formulated such that asynchronous computer vision and acceleration measurements with different sampling rates could be seamlessly combined to improve displacement estimation. The feasibility and effectiveness of the proposed displacement estimation technique were validated by performing shaking table, four‐story building model, and steel box girder pedestrian bridge tests. In all tests, the proposed technique was able to accurately estimate displacements with root mean square errors of less than 3 mm.

An improved feature matching ORB-SLAM algorithm

To solve the problems of low accuracy and poor real-time performance in traditional mobile robot vision simultaneous positioning and map construction (SLAM), the original algorithm was improved. First, the ORB features of adjacent images are extracted, and the PROSAC algorithm is used to achieve feature point matching. At the same time, the PROSAC algorithm is improved and optimized, and the execution time of the optimized PROSAC algorithm is significantly reduced; finally, based on the graph optimization model, a global Bundle Adjustment algorithm based on the largest common-view weight frame is proposed to achieve dense and sparse map creation. The algorithm is verified by Tum data set, and the experiment shows that the root mean square error has dropped significantly. The results on the data set effectively prove the effectiveness of the algorithm in this paper.

Enhancing collaborative road scene reconstruction with unsupervised domain alignment

Scene reconstruction and visual localization in dynamic environments such as street scenes are a challenge due to the lack of distinctive, stable keypoints. While learned convolutional features have proven to be robust to changes in viewing conditions, handcrafted features still have advantages in distinctiveness and accuracy when applied to structure from motion. For collaborative reconstruction of road sections by a car fleet, we propose to use multimodal domain adaptation as a preprocessing step to align images in their appearance and enhance keypoint matching across viewing conditions while preserving the advantages of handcrafted features. Training a generative adversarial network for translations between different illumination and weather conditions, we evaluate qualitative and quantitative aspects of domain adaptation and its impact on feature correspondences. Combined with a multi-feature discriminator, the model is optimized for synthesis of images which do not only improve feature matching but also exhibit a high visual quality. Experiments with a challenging multi-domain dataset recorded in various road scenes on multiple test drives show that our approach outperforms other traditional and learning-based methods by improving completeness or accuracy of structure from motion with multimodal two-domain image collections in eight out of ten test scenes.

An RGB-NIR Image Fusion Method for Improving Feature Matching

The quality of RGB images can be degraded by poor weather or lighting conditions. Thus, to make computer vision techniques work correctly, images need to be enhanced first. This paper proposes an RGB image enhancement method for improving feature matching which is a core step in most computer vision techniques. The proposed method decomposes near-infrared (NIR) image into fine detail, medium detail, and base images by using weighted least squares filters (WLSF) and boosts the medium detail image. Then, the fine and boosted medium detail images are combined, and the combined NIR detail image replaces the luminance detail image of an RGB image. Experiments demonstrates that the proposed method can effectively enhance RGB image; hence more stable image features are extracted. In addition, the method can minimize the loss of the useful visual (or optical) information of the original RGB image that can be used for other vision tasks.

Image copy‐move forgery detection algorithm based on ORB and novel similarity metric

Image forgery poses a serious threat in electric power, medicine and other fields. Relevant departments need to pay a great price to identify the authenticity of the image. For traditional copy-move forgery image detection, the existing methods have at least two problems: low robustness and poor matching caused by a low number of feature points. Here, a novel similarity metric combining cosine and Jaccard is proposed to improve feature matching, which combines with oriented features from accelerated segment test and rotated binary robust independent elementary features (ORB) feature extraction to realise effective and fast image forgery detection. First, the image is divided into overlapping blocks, and ORB is used to extract the feature points of each image block to obtain the text information. Second, the novel similarity metric is used to calculate similarity and match the text. Finally, two image blocks with the highest similarity are located. The experimental results show that, on the one hand, ORB can greatly lessen detection time. On the other hand, the novel similarity metric can improve the poor matching caused by the small number of feature points. Combining the two methods can exhibit high robustness to translation, rotation, noise, illumination and JPEG compression.

Image feature extraction algorithm in big data environment

The purpose of this article is to explore effective image feature extraction algorithms in the context of big data, and to mine their potential information from complex image data. Based on the BRISK and SIFT algorithms, this paper proposes an image feature extraction and matching algorithm based on BRISK corner points. By combining the SIFT scale space and the BRISK algorithm, a new scale space construction method is proposed. The BRISK algorithm extracts the corner invariant features. Then, by using the improved feature matching method and eliminating the mismatching algorithm, the exact matching of the images is realized. A large number of experimental verifications were performed in the standard test Mikolajczyk image database and aerial image database. The experimental results show that the improved algorithm in this paper is an effective image matching algorithm. The highest accuracy of actual aerial image matching can reach 85.19%, and it can realize the actual aerial image matching that BRISK and SIFT algorithms cannot complete. The improved algorithm in this paper has the advantages of higher matching accuracy and strong robustness.

Research and Improvement of Content-Based Image Retrieval Framework

This paper proposed a high-performance image retrieval framework, which combines the improved feature extraction algorithm SIFT (Scale Invariant Feature Transform), improved feature matching, improved feature coding Fisher and improved Gaussian Mixture Model (GMM) for image retrieval. Aiming at the problem of slow convergence of traditional GMM algorithm, an improved GMM is proposed. This algorithm initializes the GMM by using on-line [Formula: see text]-means clustering method, which improves the convergence speed of the algorithm. At the same time, when the model is updated, the storage space is saved through the improvement of the criteria for matching rules and generating new Gaussian distributions. Aiming at the problem that the dimension of SIFT (Scale Invariant Feature Transform) algorithm is too high, the matching speed is too slow and the matching rate is low, an improved SIFT algorithm is proposed, which preserves the advantages of SIFT algorithm in fuzzy, compression, rotation and scaling invariance advantages, and improves the matching speed, the correct match rate is increased by an average of 40% to 55%. Experiments on a recently released VOC 2012 database and a database of 20 category objects containing 230,800 images showed that the framework had high precision and recall rates and less query time. Compared with the standard image retrieval framework, the improved image retrieval framework can detect the moving target quickly and effectively and has better robustness.

Feature Extracting from Video Encoded for Searching Using Improved Methods

There is a need for the transportation of visual content from a particular node to a computation unit, on a network limited by bandwidth for distributed visual analysis applications, like mobile visual search or Visual Sensor Networks (VSNs). The proposed system uses the video compression technique to transfer information over a communication network. The system is used for key point detection, descriptor calculation and feature matching. The proposed system provides key points to the server as side data to retrieve only the relevant points from the encoded video. In the proposed encoding approach, the User chooses the video from the dataset, extracts frames from the video and then extracts the key point of the selected frame and encodes the string or the key point on the extracted frame to then reconstruct or compress the video and upload it on the server. For decoding, whenever the user enters the search query on the browser, the server receives the search query. The server then selects the video, which is stored on the server, and extracts frames from the video, then key point or encoded data from the selected frame using decoding, and then compares the search query and the decoded data. If it is true, the video is sent to the user. The results show that the proposed approach offers significantly improved feature matching and image retrieval performance at given bitrates.

Keypoint-based copy-move detection scheme by adopting MSCRs and improved feature matching

Copy-move detection is to find the existence of duplicated regions in an image. In this paper, an effective method based on region features is proposed to detect copy-move forgeries, especially when the image is multiple copied or with multiple copy-move groups. Firstly, maximally stable color region detector is applied to extract features, and these features are represented by Zernike moments. Then an improved matching strategy considering n best-matching features is applied to deal with the multiple-copied problem. Moreover, a hierarchical cluster algorithm is developed to estimate transformation matrices and confirm the existence of forgery. Based on these matrices, the duplicated regions can be located at pixel level. Experimental results indicate that the proposed scheme outperforms other similar state-of-the-art techniques.

Improved Feature Matching for Mobile Devices with IMU.

Thanks to the recent diffusion of low-cost high-resolution digital cameras and to the development of mostly automated procedures for image-based 3D reconstruction, the popularity of photogrammetry for environment surveys is constantly increasing in the last years. Automatic feature matching is an important step in order to successfully complete the photogrammetric 3D reconstruction: this step is the fundamental basis for the subsequent estimation of the geometry of the scene. This paper reconsiders the feature matching problem when dealing with smart mobile devices (e.g., when using the standard camera embedded in a smartphone as imaging sensor). More specifically, this paper aims at exploiting the information on camera movements provided by the inertial navigation system (INS) in order to make the feature matching step more robust and, possibly, computationally more efficient. First, a revised version of the affine scale-invariant feature transform (ASIFT) is considered: this version reduces the computational complexity of the original ASIFT, while still ensuring an increase of correct feature matches with respect to the SIFT. Furthermore, a new two-step procedure for the estimation of the essential matrix E (and the camera pose) is proposed in order to increase its estimation robustness and computational efficiency.

Fast Matching of Binary Descriptors for Large-Scale Applications in Robot Vision

The introduction of computationally efficient binary feature descriptors has raised new opportunities for real-world robot vision applications. However, brute force feature matching of binary descriptors is only practical for smaller datasets. In the literature, there has therefore been an increasing interest in representing and matching binary descriptors more efficiently. In this article, we follow this trend and present a method for efficiently and dynamically quantizing binary descriptors through a summarized frequency count into compact representations (called fsum) for improved feature matching of binary point-features. With the motivation that real-world robot applications must adapt to a changing environment, we further present an overview of the field of algorithms, which concerns the efficient matching of binary descriptors and which are able to incorporate changes over time, such as clustered search trees and bag-of-features improved by vocabulary adaptation. The focus for this article is on evaluation, particularly large scale evaluation, compared to alternatives that exist within the field. Throughout this evaluation it is shown that the fsum approach is both efficient in terms of computational cost and memory requirements, while retaining adequate retrieval accuracy. It is further shown that the presented algorithm is equally suited to binary descriptors of arbitrary type and that the algorithm is therefore a valid option for several types of vision applications.

Keypoint Encoding for Improved Feature Extraction From Compressed Video at Low Bitrates

In many mobile visual analysis applications, compressed video is transmitted over a communication network and analyzed by a server. Typical processing steps performed at the server include keypoint detection, descriptor calculation, and feature matching. Video compression has been shown to have an adverse effect on feature-matching performance. The negative impact of compression can be reduced by using the keypoints extracted from the uncompressed video to calculate descriptors from the compressed video. Based on this observation, we propose to provide these keypoints to the server as side information and to extract only the descriptors from the compressed video. First, we introduce four different frame types for keypoint encoding to address different types of changes in video content. These frame types represent a new scene, the same scene, a slowly changing scene, or a rapidly moving scene, and are determined by comparing features between successive video frames. Then, we propose Intra, Skip, and Inter modes of encoding the keypoints for different frame types. For example, keypoints for new scenes are encoded using the Intra mode, and keypoints for unchanged scenes are skipped. As a result, the bitrate of the side information related to keypoint encoding is significantly reduced. Finally, we present pairwise matching and image retrieval experiments conducted to evaluate the performance of the proposed approach using the Stanford mobile augmented reality dataset and 720p format videos. The results show that the proposed approach offers significantly improved feature matching and image retrieval performance at a given bitrate.

On the use of INS to improve Feature Matching

Abstract. The continuous technological improvement of mobile devices opens the frontiers of Mobile Mapping systems to very compact systems, i.e. a smartphone or a tablet. This motivates the development of efficient 3D reconstruction techniques based on the sensors typically embedded in such devices, i.e. imaging sensors, GPS and Inertial Navigation System (INS). Such methods usually exploits photogrammetry techniques (structure from motion) to provide an estimation of the geometry of the scene. Actually, 3D reconstruction techniques (e.g. structure from motion) rely on use of features properly matched in different images to compute the 3D positions of objects by means of triangulation. Hence, correct feature matching is of fundamental importance to ensure good quality 3D reconstructions. Matching methods are based on the appearance of features, that can change as a consequence of variations of camera position and orientation, and environment illumination. For this reason, several methods have been developed in recent years in order to provide feature descriptors robust (ideally invariant) to such variations, e.g. Scale-Invariant Feature Transform (SIFT), Affine SIFT, Hessian affine and Harris affine detectors, Maximally Stable Extremal Regions (MSER). This work deals with the integration of information provided by the INS in the feature matching procedure: a previously developed navigation algorithm is used to constantly estimate the device position and orientation. Then, such information is exploited to estimate the transformation of feature regions between two camera views. This allows to compare regions from different images but associated to the same feature as seen by the same point of view, hence significantly easing the comparison of feature characteristics and, consequently, improving matching. SIFT-like descriptors are used in order to ensure good matching results in presence of illumination variations and to compensate the approximations related to the estimation process.

The Remote Sensing Image Matching Algorithm Based on the Normalized Cross-Correlation and SIFT

SIFT (scale invariant feature transform) is one of the most robust and widely used image matching algorithms based on local features. However, its computational complexity is high. In order to reduce the matching time, an improved feature matching algorithm is proposed in this paper under the premise of stable registration accuracy. This paper proposed a normalized cross-correlation with SIFT combination of remote sensing image matching algorithm. The basic idea of the algorithm is performing the space geometry transformation of the input image with reference to the base image. Then the normalized cross-correlation captures the relevant part of the remote sensing images. By this way, we can reduce the matching range. So some unnecessary calculations are properly omitted. By utilizing the SIFT algorithm, we match the preprocessed remote sensing images, and get the registration points. This can shorten the matching time and improve the matching accuracy. Its robustness is increased correspondingly. The experimental results show that the proposed Normalized cross-correlation plus SIFT algorithm is more rapid than the standard SIFT algorithm while the performance is favorably compared to the standard SIFT algorithm when matching among structured scene images. The experiment results confirm the feasibility of our methods.

A CNN BASED HYBRID APPROACH TOWARDS AUTOMATIC IMAGE REGISTRATION

Image registration is a key component of spatial analyses that involve different data sets of the same area. Automatic approaches in this domain have witnessed the application of several intelligent methodologies over the past decade; however accuracy of these approaches have been limited due to the inability to properly model shape as well as contextual information. In this paper, we investigate the possibility of an evolutionary computing based framework towards automatic image registration. Cellular Neural Network has been found to be effective in improving feature matching as well as resampling stages of registration, and complexity of the approach has been considerably reduced using corset optimization. CNN-prolog based approach has been adopted to dynamically use spectral and spatial information for representing contextual knowledge. The salient features of this work are feature point optimisation, adaptive resampling and intelligent object modelling. Investigations over various satellite images revealed that considerable success has been achieved with the procedure. Methodology also illustrated to be effective in providing intelligent interpretation and adaptive resampling.

Cellular neural network–based hybrid approach toward automatic image registration

Image registration is a key component of various image processing operations that involve the analysis of different image data sets. Automatic image registration domains have witnessed the application of many intelligent methodologies over the past decade; however, inability to properly model object shape as well as contextual information has limited the attainable accuracy. A framework for accurate feature shape modeling and adaptive resampling using advanced techniques such as vector machines, cellular neural network (CNN), scale invariant feature transform (SIFT), coreset, and cellular automata is proposed. CNN has been found to be effective in improving feature matching as well as resampling stages of registration and complexity of the approach has been considerably reduced using coreset optimization. The salient features of this work are cellular neural network approach–based SIFT feature point optimization, adaptive resampling, and intelligent object modelling. Developed methodology has been compared with contemporary methods using different statistical measures. Investigations over various satellite images revealed that considerable success was achieved with the approach. This system has dynamically used spectral and spatial information for representing contextual knowledge using CNN-prolog approach. This methodology is also illustrated to be effective in providing intelligent interpretation and adaptive resampling.

A CNN based Hybrid approach towards automatic image registration

Abstract Image registration is a key component of various image processing operations which involve the analysis of different image data sets. Automatic image registration domains have witnessed the application of many intelligent methodologies over the past decade; however inability to properly model object shape as well as contextual information had limited the attainable accuracy. In this paper, we propose a framework for accurate feature shape modeling and adaptive resampling using advanced techniques such as Vector Machines, Cellular Neural Network (CNN), SIFT, coreset, and Cellular Automata. CNN has found to be effective in improving feature matching as well as resampling stages of registration and complexity of the approach has been considerably reduced using corset optimization The salient features of this work are cellular neural network approach based SIFT feature point optimisation, adaptive resampling and intelligent object modelling. Developed methodology has been compared with contemporary methods using different statistical measures. Investigations over various satellite images revealed that considerable success was achieved with the approach. System has dynamically used spectral and spatial information for representing contextual knowledge using CNN-prolog approach. Methodology also illustrated to be effective in providing intelligent interpretation and adaptive resampling.