Keypoint Descriptors Research Articles

Application of multi-component data model for class descriptions in the image classification problem

The subject of research of the article is the methods of image classification according to the set of descriptors of key points in computer vision systems. The aim is to increase the efficiency of classification by introducing a multicomponent data model on a set of descriptors for the base of reference images. Applied methods: ORB detector and descriptors, apparatus of set theory and vector space, metric models for determining the relevance of sets of multidimensional vectors, elements of probability theory, software modeling. Results are obtained: a modified method of image classification based on the introduction of a multicomponent model for data analysis with a system of centers is developed, methods of constructing a set of data centers are identified, the most effective is the set medoid and centers based on it. The effectiveness of the modification significantly depends on the method of forming the centers, the applied classification model, as well as on the data itself. The best results were shown by the classification with the integrated indicator separately for each of the standards in the form of the sum of the values of the distributions for the set of centers; experimentally tested the effectiveness of the classification, confirmed the efficiency of the proposed method. The practical significance of the work is the construction of classification models in the transformed data space, confirmation of the efficiency of the proposed modifications on the examples of images, the creation of software for the implementation of developed classification methods in computer vision systems.

STATISTICAL DATA ANALYSIS TOOLS IN IMAGE CLASSIFICATION METHODS BASED ON THE DESCRIPTION AS A SET OF BINARY DESCRIPTORS OF KEY POINTS

Context. Modern computer vision systems require effective classification solutions based on the research of the the processed data nature. Statistical distributions are currently the perfect tool for representing and analyzing visual data in image recognition systems. If the description of a recognized object is represented by a set of vectors, the statistical apparatus becomes fundamental for making a classification decision. The study of data distributions in the feature blocks systems for key point descriptors has shown its effectiveness in terms of achieving the necessary quality of classification and processing speed. There is a need for in-depth study of the descriptor sets statistical properties in terms of the main aspect – the multidimensional data separation for classification. This task becomes especially important for constructing new effective feature spaces, for example, by aggregating a set of descriptors by their constituent components, including individual bits. To do this, it is natural to use the apparatus of statistical criteria designed to compare the parameters of the distribution of the studied samples. Despite the widespread use and applied effectiveness of the feature descriptors apparatus for image classification, the statistical basis of these methods in their implementation in aggregate visual data systems and the choice of effective means to assess their effectiveness for distinguishing real images in application databases remains insufficiently studied.
 Objective. Development of an effective images classification method by introducing aggregate statistical features for the description components.
 Method. A metric image classifier based on feature aggregation for a set of image descriptors using statistical criteria for assessing the classification decision significance is proposed.
 Results. The synthesis of the classification method on the basis of the introduction of aggregated statistical features for a set of image description descriptors is carried out. The efficiency and effectiveness of the developed classifier are confirmed. On examples of application of a method for system of real images features its efficiency is experimentally estimated.
 Conclusions. The study makes possible to evaluate the applied effectiveness of the key points descriptors apparatus and build on its basis an aggregate features system for the effective visual objects classification implementation. Our research has shown that the available information in the form of a bit descriptors representation is sufficient for a significant statistical distinction between visual objects descriptions. Analysis of pairs and other blocks for descriptor bits provides a promising opportunity to reduce processing time.
 The scientific novelty of the study is the development of a method of image classification based on an integrated statistical features system for structural description, confirmation of the effectiveness of the method and the importance of the created features classification system in the image database.
 The practical significance of the work is to confirm the efficiency of the proposed methods on the real image descriptions examples.

Identification of Inorganic Chemical Formulas based on Support Vector Machine and SURF Key Point Descriptor

Identification of Inorganic Chemical Formulas based on Support Vector Machine and SURF Key Point Descriptor

Research on numerical simulation of wear temperature field of vehicle brake disc based on SIFI feature

In order to overcome the problems of low simulation accuracy and long time in traditional numerical simulation method of vehicle brake disc wear temperature field, a numerical simulation method of vehicle brake disc wear temperature field based on SIFI characteristics is proposed. The temperature field simulation model of vehicle brake disc wear is built, and the temperature field estimation results are obtained. On this basis, after the Gaussian scale space is generated by using SIFI features, the key point descriptor is generated by determining the key extreme points and the direction of the key points, so as to extract the characteristics of the vehicle brake disc wear temperature field. According to the feature extraction results, the numerical simulation of the vehicle brake disc wear temperature field is carried out. The experimental results show that the simulation accuracy of this method is more than 92%, the average simulation time is 33.9 ms, and the comprehensive performance is better.

Keypoint Description Using Statistical Descriptor with Similarity-Invariant Regions

This article presents a method for the description of key points using simple statistics for regions controlled by neighboring key points to remedy the gap in existing descriptors. Usually, the existent descriptors such as speeded up robust features (SURF), Kaze, binary robust invariant scalable keypoints (BRISK), features from accelerated segment test (FAST), and oriented FAST and rotated BRIEF (ORB) can competently detect, describe, and match images in the presence of some artifacts such as blur, compression, and illumination. However, the performance and reliability of these descriptors decrease for some imaging variations such as point of view, zoom (scale), and rotation. The introduced description method improves image matching in the event of such distortions. It utilizes a contourlet-based detector to detect the strongest key points within a specified window size. The selected key points and their neighbors control the size and orientation of the surrounding regions, which are mapped on rectangular shapes using polar transformation. The resulting rectangular matrices are subjected to two-directional statistical operations that involve calculating the mean and standard deviation. Consequently, the descriptor obtained is invariant (translation, rotation, and scale) because of the two methods; the extraction of the region and the polar transformation techniques used in this paper. The description method introduced in this article is tested against well-established and well-known descriptors, such as SURF, Kaze, BRISK, FAST, and ORB, techniques using the standard OXFORD dataset. The presented methodology demonstrated its ability to improve the match between distorted images compared to other descriptors in the literature.

OGI-SLAM2: A Hybrid Map SLAM Framework Grounded in Inertial-Based SLAM

Indoor localization using mobile sensors has gained momentum lately. However, indoor pedestrian localization with low-cost IMU has not been practically applied, and the main challenge is that it is difficult for low-cost IMU to obtain high-precision localization performance over a long period of time. In this paper, we propose a multi-source fusion indoor pedestrian autonomous localization framework combining landmark maps and occupancy grid maps. Combining the ideas of optimization and occupancy grid map-based inertial SLAM algorithm(OGI-SLAM), we maintain the short- and medium-term localization accuracy with OGI-SLAM, and achieve long-term localization accuracy improvement with keypoint maps. Without prior knowledge of the map, we use user motion features and environmental features to extract keypoints in the trajectory and construct keypoint descriptors. Combining the uncertainty of system location and the similarity of environmental features, we propose a keypoint extraction and association method to solve the problem of inaccurate particle filtering loop closing with low-cost IMU. In addition, a two-step trajectory optimization method is proposed, which uses bundle adjustment to correct the positioning error of keypoints and models the error correction of common points between keypoints as affine transformation, avoiding the large matrix inversion operation in the optimization process. Simulations and field trial evaluations show that the system is able to achieve a maximum positioning accuracy of 1.64 meters with low-performance IMU on board, representing a 57% maximum improvement in positioning accuracy compared to OGI-SLAM. This highlights the promise of OGI-SLAM2 as an unconventional indoor positioning method.

A SIFT Description Approach for Non-Uniform Illumination and Other Invariants

The new improvement keypoint description technique of image-based recognition for rotation, viewpoint and non-uniform illumination situations is presented. The technique is relatively simple based on two procedures, i.e., the keypoint detection and the keypoint description procedure. The keypoint detection procedure is based on the SIFT approach, Top-Hat filtering, morphological operations and average filtering approach. Where this keypoint detection procedure can segment the targets from uneven illumination particle images. While the keypoint description procedures are described and implemented using the Hu moment invariants. Where the central moments are being unchanged under image translations. The sensitivity, accuracy and precision rate of data sets were evaluated and compared. The data set are provided by color image database with variants uniform and non-uniform illumination, viewpoint and rotation changes. The evaluative results show that the approach is superior to the other SIFTs in terms of uniform illumination, non-uniform illumination and other situations. Additionally, the paper demonstrates the high sensitivity of 100%, high accuracy of 83.33% and high precision rate of 80.00%. Comparisons to other SIFT approaches are also included.

The image description reduction in the set of descriptors on informativeness metric criteria base

The subject of the research is the methods of image classification on a set of key point descriptors in computer vision systems. The goal is to improve the performance of structural classification methods by introducing indexed hash structures on the set of the dataset reference images descriptors and a consistent chain combination of several stages of data analysis in the classification process. Applied methods: BRISK detector and descriptors, data hashing tools, search methods in large data arrays, metric models for the vector relevance estimation, software modeling. The obtained results: developed an effective method of image classification based on the introduction of high-speed search using indexed hash structures, that speeds up the calculation dozens of times; the gain in computing time increases with an increase of the number of reference images and descriptors in descriptions; the peculiarity of the classifier is that not an exact search is performed, but taking into account the permissible deviation of data from the reference; experimentally verified the effectiveness of the classification, which indicates the efficiency and effectiveness of the proposed method. The practical significance of the work is the construction of classification models in the transformed space of the hash data representation, the efficiency confirmation of the proposed classifiers modifications on image examples, development of applied software models implementing the proposed classification methods in computer vision systems.

Evaluation of Keypoint Descriptors for Flight Simulator Cockpit Elements: WrightBroS Database.

The goal of the WrightBroS project is to design a system supporting the training of pilots in a flight simulator. The desired software should work on smart glasses supplementing the visual information with augmented reality data, displaying, for instance, additional training information or descriptions of visible devices in real time. Therefore, the rapid recognition of observed objects and their exact positioning is crucial for successful deployment. The keypoint descriptor approach is a natural framework that is used for this purpose. For this to be applied, the thorough examination of specific keypoint location methods and types of keypoint descriptors is required first, as these are essential factors that affect the overall accuracy of the approach. In the presented research, we prepared a dedicated database presenting 27 various devices of flight simulator. Then, we used it to compare existing state-of-the-art techniques and verify their applicability. We investigated the time necessary for the computation of a keypoint position, the time needed for the preparation of a descriptor, and the classification accuracy of the considered approaches. In total, we compared the outcomes of 12 keypoint location methods and 10 keypoint descriptors. The best scores recorded for our database were almost 96% for a combination of the ORB method for keypoint localization followed by the BRISK approach as a descriptor.

Combination method of accelerated metric data search in image classification problems

The subject of research of the paper is the methods of image classification on a set of key point descriptors in computer vision systems. The goal is to improve the performance of structural classification methods by introducing indexed hash structures on the set of the dataset reference images descriptors and a consistent chain combination of several stages of data analysis in the classification process. Applied methods: BRISK detector and descriptors, data hashing tools, search methods in large data arrays, metric models for the vector relevance estimation, software modeling. The obtained results: developed an effective method of image classification based on the introduction of high-speed search using indexed hash structures, that speeds up the calculation dozens of times; the gain in computing time increases with an increase of the number of reference images and descriptors in descriptions; the peculiarity of the classifier is that not an exact search is performed, but taking into account the permissible deviation of data from the reference; experimentally verified the effectiveness of the classification, which indicates the efficiency and effectiveness of the proposed method. The practical significance of the work is the construction of classification models in the transformed space of the hash data representation, the efficiency confirmation of the proposed classifiers modifications on image examples, development of applied software models implementing the proposed classification methods in computer vision systems.

THE SPEED CALCULATING INCREASING METHOD OF THE MARKOV MODEL NETWORK NODE

The subject of this research is the image classification methods based on a set of key points descriptors. The goal is to increase the performance of classification methods, in particular, to improve the time characteristics of classification by introducing hashing tools for reference data representation. Methods used: ORB detector and descriptors, data hashing tools, search methods in data arrays, metrics-based apparatus for determining the relevance of vectors, software modeling. The obtained results: developed an effective method of image classification based on the introduction of high-speed search using hash structures, which speeds up the calculation dozens of times; the classification time for the considered experimental descriptions increases linearly with decreasing number of hashes; the minimum metric value limit choice on setting the class for object descriptors significantly affects the accuracy of classification; the choice of such limit can be optimized for fixed samples databases; the experimentally achieved accuracy of classification indicates the efficiency of the proposed method based on data hashing. The practical significance of the work is - the classification model’s synthesis in the hash data representations space, efficiency proof of the proposed classifiers modifications on image examples, development of applied software models implementing the proposed classification methods in computer vision systems.

Adaptive clustering-based approach for forgery detection in images containing similar appearing but authentic objects

Copy-move forgery is one of the well-known image forgery technique which exploits regions of the same image to create forged image by replicating or hiding authentic content of the original image. Original images can also contain similar looking but authentic objects. In such cases, identification of authentic and tampered images is a complicated task. To tackle this problem, we propose a method in which Stationary Wavelet Transform (SWT) and spatial-constrained edge preserving watershed segmentation are applied over input image in preprocessing step. Keypoint extraction and descriptor computation are performed using Cascaded Features from Accelerated Segment Test (Cascaded FAST) and Binary Robust Invariant Scalable Keypoint (BRISK) descriptor, respectively. Approximate nearest neighbor search is performed using Random Binary Search Tree (RBST) method. For keypoint clustering, Adaptive Density Peak Clustering (ADPC) technique is employed. Outlier removal is performed using Random Sample Consensus (RANSAC) technique. Further, forged regions are localized using correlation map generation. Experimental results display that the proposed approach can effectively distinguish between forged and original images containing similar appearing but authentic objects. It is also able to detect forged images sustaining different post-processing attacks. For COVERAGE dataset, proposed technique achieves high F-Measure = 86.901% and low False Positive Rate (FPR) = 15.241% in comparison to state-of-the-art techniques.

Learning 3D medical image keypoint descriptors with the triplet loss.

We propose to learn a 3D keypoint descriptor which we use to match keypoints extracted from full-body CT scans. Our methods are inspired by 2D keypoint descriptor learning, which was shown to outperform hand-crafted descriptors. Adapting these to 3D images is challenging because of the lack of labelled training data and high memory requirements. We generate semi-synthetic training data. For that, we first estimate the distribution of local affine inter-subject transformations using labelled anatomical landmarks on a small subset of the database. We then sample a large number of transformations and warp unlabelled CT scans, for which we can subsequently establish reliable keypoint correspondences using guided matching. These correspondences serve as training data for our descriptor, which we represent by a CNN and train using the triplet loss with online triplet mining. We carry out experiments on a synthetic data reliability benchmark and a registration task involving 20 CT volumes with anatomical landmarks used for evaluation purposes. Our learned descriptor outperforms the 3D-SURF descriptor on both benchmarks while having a similar runtime. We propose a new method to generate semi-synthetic data and a new learned 3D keypoint descriptor. Experiments show improvement compared to a hand-crafted descriptor. This is promising as literature has shown that jointly learning a detector and a descriptor gives further performance boost.

Segmentation of Melanocytic Lesion Images Using Gamma Correction with Clustering of Keypoint Descriptors.

The early detection of skin cancer, especially through the examination of lesions with malignant characteristics, has been reported to significantly decrease the potential fatalities. Segmentation of the regions that contain the actual lesions is one of the most widely used steps for achieving an automated diagnostic process of skin lesions. However, accurate segmentation of skin lesions has proven to be a challenging task in medical imaging because of the intrinsic factors such as the existence of undesirable artifacts and the complexity surrounding the seamless acquisition of lesion images. In this paper, we have introduced a novel algorithm based on gamma correction with clustering of keypoint descriptors for accurate segmentation of lesion areas in dermoscopy images. The algorithm was tested on dermoscopy images acquired from the publicly available dataset of Pedro Hispano hospital to achieve compelling equidistant sensitivity, specificity, and accuracy scores of 87.29%, 99.54%, and 96.02%, respectively. Moreover, the validation of the algorithm on a subset of heavily noised skin lesion images collected from the public dataset of International Skin Imaging Collaboration has yielded the equidistant sensitivity, specificity, and accuracy scores of 80.59%, 100.00%, and 94.98%, respectively. The performance results are propitious when compared to those obtained with existing modern algorithms using the same standard benchmark datasets and performance evaluation indices.

Evaluation of Multi-Stream Fusion for Multi-View Image Set Comparison

We consider the problem of image set comparison, i.e., to determine whether two image sets show the same unique object (approximately) from the same viewpoints. Our proposition is to solve it by a multi-stream fusion of several image recognition paths. Immediate applications of this method can be found in fraud detection, deduplication procedure, or visual searching. The contribution of this paper is a novel distance measure for similarity of image sets and the experimental evaluation of several streams for the considered problem of same-car image set recognition. To determine a similarity score of image sets (this score expresses the certainty level that both sets represent the same object visible from the same set of views), we adapted a measure commonly applied in blind signal separation (BSS) evaluation. This measure is independent of the number of images in a set and the order of views in it. Separate streams for object classification (where a class represents either a car type or a car model-and-view) and object-to-object similarity evaluation (based on object features obtained alternatively by the convolutional neural network (CNN) or image keypoint descriptors) were designed. A late fusion by a fully-connected neural network (NN) completes the solution. The implementation is of modular structure—for semantic segmentation we use a Mask-RCNN (Mask regions with CNN features) with ResNet 101 as a backbone network; image feature extraction is either based on the DeepRanking neural network or classic keypoint descriptors (e.g., scale-invariant feature transform (SIFT)) and object classification is performed by two Inception V3 deep networks trained for car type-and-view and car model-and-view classification (4 views, 9 car types, and 197 car models are considered). Experiments conducted on the Stanford Cars dataset led to selection of the best system configuration that overperforms a base approach, allowing for a 67.7% GAR (genuine acceptance rate) at 3% FAR (false acceptance rate).

Study of the effectiveness of image classifiers by statistical distributions for components of structural description

The subject of research is models for constructing image classifiers in the description space as a set of descriptors of key points in the recognition of visual objects in computer vision systems. The goal is to create and study the properties of the image classifier based on the construction of an ensemble of distributions for the components of the structural description using various models of classification decisions, which provides effective classification. Tasks: construction of classification models in the synthesized space of images of probability distributions, analysis of parameters influencing their efficiency, experimental evaluation of the effectiveness of classifiers by means of software modeling based on the results of processing the experimental image base. The applied methods are: ORB detector for formation of keypoint descriptors, data mining, mathematical statistics, means of determining relevance for sets of data vectors, software modeling. The obtained results: The developed method of classification confirms its efficiency and effectiveness for image classification. The effectiveness of the method can be enhanced by the introduction of a variety of types of metrics and measures of similarity between centers and descriptors, by the choice of method of forming centers for reference etalon descriptions, by the introduction of logical processing and compression of the structural description. The best results of the classification were shown by the model using the most important class by the distribution vector for each descriptor corresponding to the mode parameter. The use of a concentrated part of the description data makes it possible to improve its distinction from other descriptions. The use of the median as the center of description has an advantage over the mean. Conclusions. Scientific novelty is the development of an effective method of image classification based on the introduction of a system of probability distributions for data components, which contributes to in-depth analysis in the data space and increases in classification effectiveness. The classifier is implemented in the variants of comparing the integrated representation of distributions by classes and on the basis of mode analysis for the distributions of individual components. The practical importance of the work is the construction of classification models in the modified data space, confirmation of the efficiency of the proposed modifications of data analysis on examples of images, development of software models for implementation of the proposed classification methods in computer vision systems.

Application of data hashing tools to accelerate classification decisions in structural image recognition methods

The subject of this research is the image classification methods based on a set of key points descriptors. The goal is to increase the performance of classification methods, in particular, to improve the time characteristics of classification by introducing hashing tools for reference data representation. Methods used: ORB detector and descriptors, data hashing tools, search methods in data arrays, metrics-based apparatus for determining the relevance of vectors, software modeling. The obtained results: developed an effective method of image classification based on the introduction of high-speed search using hash structures, which speeds up the calculation dozens of times; the classification time for the considered experimental descriptions increases linearly with decreasing number of hashes; the minimum metric value limit choice on setting the class for object descriptors significantly affects the accuracy of classification; the choice of such limit can be optimized for fixed samples databases; the experimentally achieved accuracy of classification indicates the efficiency of the proposed method based on data hashing. The practical significance of the work is - the classification model’s synthesis in the hash data representations space, efficiency proof of the proposed classifiers modifications on image examples, development of applied software models implementing the proposed classification methods in computer vision systems.

A fast technique to detect copy-move image forgery with reflection and non-affine transformation attacks

Copy-move forgery is one of the most frequently utilized image tampering technique which uses the segment of the same image to produce manipulated image by duplicating or concealing image regions. To remove suspicious traces of forgery, various attacks are applied over the tampered image which make forgery detection process too complicated. We propose a forgery detection technique in which Center Surround Extrema (CenSurE) detector is applied for keypoint detection from images. To compute keypoint descriptors, Local Image Permutation Interval Descriptor (LIPID) is used. Keypoint matching is performed using k-Nearest Neighbor (k-NN) technique with utilization of k-d tree and Best-Bin-First (BBF) search method. Grouping over keypoints is performed using Fuzzy C-Means (FCM) clustering. We apply Random Sample Consensus (RANSAC) algorithm to remove outliers obtained during forgery detection process. Experimental results show that proposed technique can effectively detect forged images containing reflection and non-affine transformation with geometrical attacks. In addition, proposed approach also shows robustness against erosion, dilation, RGB color addition, zoom motion blur, JPEG compression, spread noise addition, and multiple copy-move attacks. Proposed scheme consumes least time in forgery detection as compared to state-of-the-art methods.

이미지 특징점 탐지 및 특징량 추출 알고리즘 추천 모델 개발

이미지 특징점 탐지 및 특징량 추출 알고리즘 추천 모델 개발

Goodpoint: unsupervised learning of key point detection and description

Subject of Research. The paper presents the study of algorithms for key point detection and description, widely used in computer vision. Typically, the corner detector acts as a key point detector, including neural key point detectors. For some types of images obtained in medicine, the application of such detectors is problematic due to the small number of detected key points. The paper considers a problem of a neural network key point detector training on unlabeled images. Method. We proposed the definition of key points not depending on specific visual features. A method was considered for training of a neural network model meant for detecting and describing key points on unlabeled data. The application of homographic image transformation was basic to the method. The neural network model was trained to detect the same key points on pairs of noisy images related to a homographic transformation. Only positive examples were used for detector training, just points correctly matched with features produced by the neural network model for key point description. Main Results. The unsupervised learning algorithm is used to train the neural network model. For the ease of comparison, the proposed model has a similar architecture and the same number of parameters as the supervised model. Model evaluation is performed on the three different datasets: natural images, synthetic images, and retinal photographs. The proposed model shows similar results to the supervised model on the natural images and better results on retinal photographs. Improvement of results is demonstrated after additional training of the proposed model on images from the target domain. This is an advantage over a model trained on a labeled dataset. For comparison, the harmonic average of such metrics is used as: the accuracy and the depth of matching by descriptors, reproducibility of key points and image coverage. Practical Relevance. The proposed algorithm makes it possible to train the neural network key point detector together with the feature extraction model on images from the target domain without costly dataset labeling and reduce labor costs for the development of the system that uses the detector.