Feature Description Algorithm Research Articles

Research on Image Stitching Technology for Road Surface Cracks

Image stitching refers to the combination of multiple images with a certain over lappingarea rate acquired by the image acquisition device into one image, which has a full picture of each image. Image stitching technology is an important part in the field of digital image processing. With the increasing requirements for image quality and the limited resolution of ordinary cameras, it is impossible to obtain large angle of view and clear image at the same time. In this paper, the key technologies involved in pavement crack image stitching are intensively studied, including image feature point detection and matching, internal and external camera parameter estimation and optimization, and image fusion. The work done in this paper and the main innovations include: (1)Two main feature detection and description algorithms, SIFT algorithm and ORBalgorithm, are studied. In view of the fact that the SIFT algorithm has redundancy when the Gaussian pyramid is established, this paper appropriately reduces the number of groups and maintains the number and stability of feature point detection. Combined with the advantages and disadvantages of the improved SIFT algorithm and ORB algorithm, and the performance of the two algorithms in the detection of pavement cracks, the improvedSIF'T algorithm is used as the feature point detection algorithm, and rBRlEF algorithm for describing the feature points in the ORB algorithm is used as the feature point description algorithm. The description algorithm is called the BSIF'T algorithm. Experiments show that the robustness of BSIFT algorithm is similar to that of SIFT algorithm, which is better than ORB algorithm and faster than SIF'T algorithm. (2)Image fiusion algorithms are studied, including best seam-line search and image fusion. Aiming at the ghost phenomenon that may occur in overlapping regions, this paper proposes a best seam-line search algorithm based on distance transform.

Identifying Historic Buildings over Time through Image Matching

The buildings in a city are of great importance. Certain historic buildings are landmarks and indicate the city’s architecture and culture. The buildings over time undergo changes because of various factors, such as structural changes, natural disaster damages, and aesthetic interventions. The form of buildings in each period is perceived and understood by people of each generation, through photography. Nevertheless, each photograph has its own characteristics depending on the camera (analog or digital) used for capturing it. Any photo, even depicting the same object, is impossible to capture in the same way in terms of illumination, viewing angle, and scale. Hence, to study two or more photographs depicting the same object, first they should be identified and then properly matched. Nowadays, computer vision contributes to this process by providing useful tools. In particular, for this purpose, several feature detection and description algorithms of homologous points have been developed. In this study, the identification of historic buildings over time through feature correspondence techniques and methods is investigated. Especially, photographs from landmarks of Drama city, in Greece, on different dates and conditions (weather, light, rotation, scale, etc.), were gathered and experiments on 2D pairs of images, implementing traditional feature detectors and descriptors algorithms, such as SIFT, ORB, and BRISK, were carried out. This study aims to evaluate the feature matching procedure focusing on both the algorithms’ performance (accuracy, efficiency, and robustness) and the identification of the buildings. SIFT and BRISK are the most accurate algorithms while ORB and BRISK are the most efficient.

RI-MFM: A Novel Infrared and Visible Image Registration with Rotation Invariance and Multilevel Feature Matching

In the past ten years, multimodal image registration technology has been continuously developed, and a large number of researchers have paid attention to the problem of infrared and visible image registration. Due to the differences in grayscale distribution, resolution and viewpoint between two images, most of the existing infrared and visible image registration methods are still insufficient in accuracy. To solve such problems, we propose a new robust and accurate infrared and visible image registration method. For the purpose of generating more robust feature descriptors, we propose to generate feature descriptors using a concentric-circle-based feature-description algorithm. The method enhances the description of the main direction of feature points by introducing centroids, and, at the same time, uses concentric circles to ensure the rotation invariance of feature descriptors. To match feature points quickly and accurately, we propose a multi-level feature-matching algorithm using improved offset consistency for matching feature points. We redesigned the matching algorithm based on the offset consistency principle. The comparison experiments with several other state-of-the-art registration methods in CVC and homemade datasets show that our proposed method has significant advantages in both feature-point localization accuracy and correct matching rate.

Research and improvement of feature detection algorithm based on FAST

At present, in the application of feature-based medical 3D reconstruction technology, there are still problems such as low matching accuracy of feature points in endoscope images and slow processing speed of image data. Therefore, the feature-based 3D reconstruction theory is of great research value and has great application value. This paper proposed a new feature detection method to improve the problems. This paper divides feature detection into two parts for further improvements: feature extraction and feature description. For feature extraction, the FAST algorithm shows a poor classification effect, so this paper adds the decision tree based on the C4.5 algorithm into the traditional FAST. The original data are divided into two decision trees to make the feature extraction performance more stable and feature point extraction more efficient. For the feature description part, the FREAK descriptor is used, combined with this paper's improved feature extraction algorithm. The feature points are extracted in scale space. The second-order function fitting is carried out according to the feature points' response scores in different scales. The scale-invariant descriptor of sub-pixel precision is obtained. The experimental results on the endoscope image show that the feature extraction method has a higher extraction accuracy and faster extraction speed. In addition, the feature description algorithm has higher calculation efficiency.

Patch-SIFT: Enhanced feature descriptor to learn human facial emotions using an Ensemble approach

Background: Having experienced more than a year of pandemic, a variety of applications such as online classrooms, virtual office meetings, conferences, online games, Social media & Networks, Mobile applications, and many other infotainment areas have made humans live with gadgets and respond to them. However, all these applications have an impact on human behavioral transformation. It is very significant for employers to understand the emotions of their employees in the era of online office & work from home concept to increase productivity. Learning and identifying emotions from the human face has its application in all online portals when physical contact could not be achieved. Ojbective: Human Facial emotions can be learned using enormous feature descriptors that extract image features. While local feature descriptors retrieve pixel-level information, global feature descriptors extract the overall image information. Both of the feature descriptors quantify the image information, however, they don’t provide complete and relevant information. Hence, this research work aims to improve the existing local feature descriptor to perform globally for emotion recognition. Method: Our proposed feature descriptor, Patch-SIFT collects features from multiple patches within an image. This strategy is evolved to globally apply the local feature descriptor as a hybridization paradigm. The extracted features are trained and tested on an ensemble model. Findings: The Proposed Feature descriptor (Patch-SIFT) performance with ensemble model is found to produce an improved accuracy of 98% compared with existing feature descriptors and Machine learning classifiers. Novelty: This research work tries to evolve a new Feature descriptor algorithm based on SIFT algorithm for an efficient emotion recognition system that works without the need for any additional GPU or huge dataset. Keywords Classification, Ensemble, Feature descriptor, Patch­SIFT

Multidimensional Local Binary Pattern for Hyperspectral Image Classification

For the large amount of spatial and spectral information contained in hyperspectral image (HSI), feature description of HSI has attracted widespread concern in recent years. Existing deep learning-based HSI feature description algorithms require a large number of training samples and have poor interpretability. Therefore, it is necessary to develop an efficient HSI features description algorithm with interpretability based on machine learning. Local binary pattern (LBP) is a classical descriptor used to extract the local spatial texture features of images, which has been widely applied to image feature description and matching. However, the existing LBP algorithms for HSI are based on the single-dimensional description, which leads to the limitations on the expression of spatial–spectral information. Therefore, a multidimensional LBP (MDLBP) based on Clifford algebra for HSI is proposed in this article, which is able to extract spatial–spectral feature from multiple dimensions. First, with the theory of the Clifford algebra, a new representation of HSI including spatial and spectral information is built. Second, the geometric relationship between the local geometry of HSI in Clifford algebra space is calculated to realize the local multidimensional description of the local spatial–spectral information. Finally, a novel LBP coding algorithm for HSI is implemented based on the local multidimensional description to calculate the feature descriptor of HSI. The experimental results on HSI classification show that our proposed MDLBP algorithm can achieve higher accuracy than the representative spatial–spectral features and the existing LBP algorithms, especially in the scenery of small-scale training samples.

2D object recognition: a comparative analysis of SIFT, SURF and ORB feature descriptors

Object recognition is a key research area in the field of image processing and computer vision, which recognizes the object in an image and provides a proper label. In the paper, three popular feature descriptor algorithms that are Scale Invariant Feature Transform (SIFT), Speeded Up Robust Feature (SURF) and Oriented Fast and Rotated BRIEF (ORB) are used for experimental work of an object recognition system. A comparison among these three descriptors is exhibited in the paper by determining them individually and with different combinations of these three methodologies. The amount of the features extracted using these feature extraction methods are further reduced using a feature selection (k-means clustering) and a dimensionality reduction method (Locality Preserving Projection). Various classifiers i.e. K-Nearest Neighbor, Naive Bayes, Decision Tree, and Random Forest are used to classify objects based on their similarity. The focus of this article is to present a study of the performance comparison among these three feature extraction methods, particularly when their combination derives in recognizing the object more efficiently. In this paper, the authors have presented a comparative analysis view among various feature descriptors algorithms and classification models for 2D object recognition. The Caltech-101 public dataset is considered in this article for experimental work. The experiment reveals that a hybridization of SIFT, SURF and ORB method with Random Forest classification model accomplishes the best results as compared to other state-of-the-art work. The comparative analysis has been presented in terms of recognition accuracy, True Positive Rate (TPR), False Positive Rate (FPR), and Area Under Curve (AUC) parameters.

Image Stitching using AKAZE Features

Accelerated KAZE (AKAZE) is a multi-scale 2D feature detection and description algorithm in nonlinear scale spaces proposed recently. This paper presents an image stitching algorithm which uses a feature detection and description algorithm; AKAZE and an image blending algorithm; weighted average blending. The whole process is divided into the following steps: First of all, detect feature points in the image and then get feature descriptors of detected points using AKAZE. Next, obtain corresponding matching pairs by using K-NN (K nearest neighbors) algorithm and remove the false matched points by MSAC (M-estimator SAmple Consensus) algorithm. MSAC is a variant of the RANSAC (Random Sample Consensus) algorithm and more accurate than RANSAC. Thereafter, calculate the homography matrix from correct matches. At last, blend the images by using weighted average blending algorithm. Comparison of proposed AKAZE-based algorithm with SIFT-, SURF- and ORB-based algorithms is also presented. According to the experiments, the proposed AKAZE-based image stitching algorithm minimizes stitching seam and generates a perfect stitched image, and also this algorithm is faster than previous algorithms.

DEEP LEARNING BASED FEATURE MATCHING AND ITS APPLICATION IN IMAGE ORIENTATION

Abstract. Matching images containing large viewpoint and viewing direction changes, resulting in large perspective differences, still is a very challenging problem. Affine shape estimation, orientation assignment and feature description algorithms based on detected hand crafted features have shown to be error prone. In this paper, affine shape estimation, orientation assignment and description of local features is achieved through deep learning. Those three modules are trained based on loss functions optimizing the matching performance of input patch pairs. The trained descriptors are first evaluated on the Brown dataset (Brown et al., 2011), a standard descriptor performance benchmark. The whole pipeline is then tested on images of small blocks acquired with an aerial penta camera, to compute image orientation. The results show that learned features perform significantly better than alternatives based on hand crafted features.

A Method to Enhance Homogeneous Distribution of Matched Features for Image Matching

Features are distinctive landmarks of an image. There are various feature detection and description algorithms. Many computer vision algorithms require matching of features from two images. Large number of correct matches with homogeneous distribution in the images is needed for robustness of the image matching. The matches are generally obtained using a feature distance threshold and ambiguous matches are rejected using a ratio test. This paper proposes a method that can be added to image matching pipeline for enhancing homogeneous distribution and increasing the number of matched feature points. After successfully matching an image pair, spatially close feature points go through an elimination process which aims to decrease ambiguity at the second matching step. Then, a coarse geometric transformation between two images is calculated, through which the detected feature points in one image (i.e. the moving image) are projected to the other image (i.e. the fixed image). Then, feature points from the moving image are matched to neighboring feature points of the fixed image within a pre-determined spatial distance. This narrows down the possible candidates and enables less correct matches being rejected because of the ratio test. The effectiveness and feasibility of our method is demonstrated with experiments on images acquired from a drone camera during flight.

TSSLBP: tensor-based spatial–spectral local binary pattern

Local binary pattern (LBP) is a simple yet efficient image feature description algorithm that has been widely used in applications related to image classification. To more effectively make use of the local features of a hyperspectral image (HSI), a tensor-based spatial–spectral local binary pattern (TSSLBP) that can fully utilize the local spatial-spectral feature of an HSI is proposed. We specifically make efforts in three aspects. First, we construct a third-order tensor model for HSI to combine the local spatial and spectral features. Second, a dimension reduction method based on tensor decomposition is proposed to reduce redundant information of the HSI tensor model. Third, a joint spatial-spectral LBP coding algorithm is implemented based on the HSI tensor model to calculate the local spatial–spectral feature vector of HSI. To verify the effectiveness of the proposed TSSLBP, we apply it to the task of HSI classification and compare it with six existing algorithms. Experimental results on the three popular HSI datasets show that TSSLBP can fully utilize rich information in the local spatial–spectral domain and can significantly improve the classification accuracy of HSI, especially in the case of scarce training samples.

Virtual Reality Image Technology Assists Training Optimization of Motion Micro-Time Scale

Virtual reality image local features are highly repetitive and are not easily affected by target occlusion, so research on virtual reality image local feature extraction techniques has received increasing attention. For the purpose of global scale selection, this article considers the virtual reality image in the global scope from the theoretical perspectives of the continuity of virtual reality images, the repeatability of virtual reality image features, the scale of scene virtual reality images in the global scope, and the visual saliency. This paper gives a general framework of multi-scale geometric analysis and particle swarm optimization virtual reality image matching algorithm. For the problem of virtual reality image matching, the original particle swarm optimization algorithm is improved. A more reasonable inertia weight change formula, a method for selecting the optimal solution when two objectives are optimized at the same time, and a method for selecting a reasonable input are designed. In this paper, performance evaluation experiments are performed on several feature description algorithms. The experimental results show that the descriptors constructed by these feature description algorithms are significantly more robust than SIFT (Scale Invariant Feature Transform) descriptors. The improved multi-scale geometric analysis and particle swarm optimization are robust due to the introduction of multiple support domains. It has the best discriminative ability, but at the same time, the introduction of multiple support domains leads to the worst real-time algorithm; DGOH (Dual Gradient Orientation Histogram) algorithm is more robust than FRDOH (Fast Repre-sentation Based on a Double Orientation Histogram) and SIFT algorithm, and DGOH algorithm has the best real-time performance.

An Evolutionary Computing Approach to Solve Object Identification Problem in Image Processing Applications

In the given image identifying the existence of a required object is the concern of the object detection process. This is quite natural for Human without any effort, however making a machine to detect an object in image is tedious. To make machines to recognize the objects, the feature descriptor algorithms are to be implemented. The general object detection approaches use collection of local and global descriptors to represent an image. Difficulties arise during this process when there is variation in lightening, positioning, rotation, mirroring, occlusion, scaling etc., of the same object in different image scenes. To overcome these difficulties, we need combination of features that detects the object in the image scene. But there exist lot of descriptors that can be used. Hence, finding the required number of feature descriptors for object detection is a crucial task. The question that comes out here is how to select the optimum number of descriptors to achieve optimum accuracy? The answer for the question is an optimization algorithm, which can be employed to select the best combination of the descriptors with maximum detection accuracy. This paper proposing an Evolutionary Computation (EC) based approach with the Differential Evolution (DE) algorithm to find the optimal combination of feature descriptors to achieve optimal object detection accuracy. The proposed approach is implemented and its superiority is verified with four different images and results obtained are presented in this paper.

Local Feature Descriptor Indexing for Image Matching and Object Detection in Real-Time Applications

Local feature extraction and description algorithms can be used to address a large variety of computer vision problems, including pattern recognition, frame stitching and 3D reconstruction. One of the most computationally expensive and time-consuming stages of any method based on local features is keypoint matching. This paper discusses possible ways to optimize matching for frame alignment and object detection applications using Vantage-Point tree indexing to optimize pairwise keypoint matching, and image keypoint graph indexing to optimize pose estimation.

Performance Evaluation of Gradient Domain and Pyramid Blending Used in Image Stitching Process with ORB Binary Descriptor

Panorama development is the basically method of integrating multiple images captured of the same scene under consideration to get high resolution image. This process is useful for combining multiple images which are overlapped to obtain larger image. Usefulness of Image stitching is found in the field related to medical imaging, data from satellites, computer vision and automatic target recognition in military applications. The goal objective of this research paper is basically for developing an high improved resolution and its quality panorama having with high accuracy and minimum computation time. Initially we compared different image feature detectors and tested SIFT, SURF, ORB to find out the rate of detection of the corrected available key points along with processing time. Later on, testing is done with some common techniques of image blending or fusion for improving the mosaicing quality process. In this experimental results, it has been found out that ORB image feature detection and description algorithm is more accurate, fastest which gives a higher performance and Pyramid blending method gives the better stitching quality. Lastly panorama is developed based on combination of ORB binary descriptor method for finding out image features and pyramid blending method.

Hardware Friendly Robust Synthetic Basis Feature Descriptor

Finding corresponding image features between two images is often the first step for many computer vision algorithms. This paper introduces an improved synthetic basis feature descriptor algorithm that describes and compares image features in an efficient and discrete manner with rotation and scale invariance. It works by performing a number of similarity tests between the feature region surrounding the feature point and a predetermined number of synthetic basis images to generate a feature descriptor that uniquely describes the feature region. Features in two images are matched by comparing their descriptors. By only storing the similarity of the feature region to each synthetic basis image, the overall storage size is greatly reduced. In short, this new binary feature descriptor is designed to provide high feature matching accuracy with computational simplicity, relatively low resource usage, and a hardware friendly design for real-time vision applications. Experimental results show that our algorithm produces higher precision rates and larger number of correct matches than the original version and other mainstream algorithms and is a good alternative for common computer vision applications. Two applications that often have to cope with scaling and rotation variations are included in this work to demonstrate its performance.

SIFT and it’s Variants: An Overview

SIFT is a powerful feature description algorithm in the field of computer vision and pattern recognition. It finds its application in many real-world applications like object recognition, image retrieval, image matching,etc. due to its numerous robust properties. Since its inception, researchers are trying to improvise it in many ways and as a result, many variants of SIFT came into existence. In computer vision and pattern recognition literature, there are a good amount of reviews articles available on the earliest variants of SIFT like PCA-SIFT, SURF, etc. However,very limited literature is available that reviewed the latest variants. This paper comprehensively reviews some of the state of the art variants in SIFT family along with the popular old variants. It discusses the drawbacks in SIFT and how these drawbacks paved the way for the existence of different variants. A detailed description of each variant is presented and concluded with its pros and cons. The different variants reviewed in the paper are PCA-SIFT, SURF, GSIFT, CSIFT, ASIFT, VF-SIFT, and DSIFT.

GPU acceleration of the KAZE image feature extraction algorithm

The recently proposed, KAZE image feature detection and description algorithm (Alcantarilla et al. in Proceedings of the British machine vision conference. LNCS, vol 7577, no 6, pp 13.1–13.11, 2013) offers significantly improved robustness in comparison to conventional algorithms like SIFT (scale-invariant feature transform) and SURF (speeded-up robust features). The improved robustness comes at a significant computational cost, however, limiting its use for many applications. We report a GPU acceleration of the KAZE algorithm that is significantly faster than its CPU counterpart. Unlike previous reports, our acceleration does not resort to binary descriptors and can serve as a drop-in replacement for CPU-KAZE, SIFT, SURF etc. By achieving nearly tenfold speedup (for a 1920 by 1200 sized image, our Compute Unified Device Architecture (CUDA)-C implementation took around 245 ms on a single GPU in comparison to nearly 2400 ms for a 16-threaded CPU version) without degradation in feature extraction performance, our work expands the applicability of the KAZE algorithm. Additionally, the strategies described here could also prove useful for the GPU implementation of other nonlinear scale-space-based image processing algorithms.

A novel SIFT architecture and ASIC implementation for real time SOC application

Local feature detection and description algorithms such as scale invariant feature transform (SIFT) algorithm are among the most commonly used techniques in computer vision. They are used mainly to detect and extract high-level information from low-level (pixel) information in images. These algorithms are computationally intensive and its pure software implementations are far from reaching real-time performance especially on embedded systems with limited computational power. In this paper, an Application-Specific Integrated Circuit (ASIC) implementation of the SIFT algorithm is proposed that is suitable for real-time image processing. In the Gaussian scale space generation step, several techniques were used to substantially reduce the hardware complexity: the multiplierless multiple constant multiplication, the symmetrical property of Gaussian mask in addition to the common sub expression elimination algorithm. Through the re-arrangement of SIFT’s 128 values in a specific approach, a new multi-ported memory is implemented to reduce the memory size. Furthermore, a novel mechanism is applied to continuously monitor the active window in the input and the intermediate results. The proposed technique does not only improve the performance of the SIFT implementation, but it also keeps the features extraction accuracy exactly the same as the software implementation. In this work, for an image size of $$256 \times 256$$ a frame rate of 56 fps was achieved. Additionally, the results confirm that the proposed architecture has lower hardware and memory costs compared to other works in the literature (FPGA based implementations). Lastly this architecture was implemented using Standard Cell Library Based ASIC flow using 65 nm low power GF library, it occupied an area of 1.4 $${{{\mathrm{mm}}}^2}$$ and consumed a total power of 30.4604 mW which makes it very suitable for integrated System on Chips (SoC) applications.

Performance Evaluation of Feature Detectors and Descriptors Beyond the Visible

Feature detection and description algorithms represent an important milestone in most computer vision applications. They have been examined from various perspectives during the last decade. However, most studies focused on their performance when used on visible band imagery. This modality suffers considerably in poor lighting conditions and notably during night-time. Infrared cameras, which noticed a considerable proliferation in recent years, offer a viable alternative in such conditions. Understanding how the building blocks of computer vision applications behave in this modality would help the community accommodating them. For this reason, we carried out a performance analysis of the most commonly used feature detectors and descriptors beyond the visible. A dataset accounting for the various challenges on these algorithms has been generated. In addition, challenges inherent to the thermal modality have been considered, notably the non-uniformity noise. A comprehensive quantitative investigation into the performance of feature detectors and descriptors is therefore presented. This study would serve to filling the gap in the literature as most analyzes have been based on visible band imagery.