Keypoint Descriptors Research Articles

A fully pipelined FPGA accelerator for scale invariant feature transform keypoint descriptor matching

The scale invariant feature transform (SIFT) algorithm is considered a classical feature extraction algorithm within the field of computer vision. SIFT keypoint descriptor matching is a computationally intensive process due to the amount of data consumed. In this work, we designed a novel fully pipelined hardware accelerator architecture for SIFT keypoint descriptor matching. The accelerator core was implemented and tested on a field programmable gate array (FPGA). The proposed hardware architecture is able to properly handle the memory bandwidth necessary for a fully-pipelined implementation and hits the roofline performance model, achieving the potential maximum throughput. The fully pipelined matching architecture was designed based on the consine angle distance method. Our architecture was optimized for 16-bit fixed-point operations and implemented on hardware using a Xilinx Zynq-based FPGA development board. Our proposed architecture shows a noticeable reduction of area resources compared with its counterparts in literature, while maintaining high throughput by alleviating memory bandwidth restrictions. The results show a reduction in consumed device resources of up to 91% in LUTs and 79% of BRAMs. Our hardware implementation is 15.7 × faster than the comparable software approach.

An Enhanced Image Matching Strategy Using Binary-Stream Feature Descriptors

This letter improves the computational efficiency and proposes a methodology to retrieve the missing feature pairs when utilizing binary-based features for image matching. The 64-byte feature descriptors of binary robust invariant scalable keypoints (BRISKs) are rearranged by combining human retina ganglion cells distribution and visual accommodation to speed up the image matching. In addition, an interactive two-sided matching is designed to determine the most probable keypoint pair when a feature point in the reference image is mapped to multiple candidates in the target image. Experimental results indicate that the proposed inverse sorting ring can reduce the processing time by more than 10% compared to the accelerated BRISK while maintaining the same reliability. Also, additional point-to-point feature pairs can be regained from the point-to-multicandidate cases by the proposed method in order to increase the number of matches.

Region-based image registration for remote sensing imagery

We propose an automatic region-based registration method for remote sensing imagery. In this method, we aim to register two images by matching region properties to address possible errors caused by local feature estimators. We apply automated image segmentation to identify the regions and calculate regional Fourier descriptors and standardized regional intensity descriptors for each region. We define a joint matching cost, as a linear combination of Euclidean distances, to establish and extract correspondences between regions. The segmentation technique utilizes kernel density estimators for edge localization, followed by morphological reconstruction and the watershed transform. We evaluated the registration performance of our method on synthetic and real datasets. We measured the registration accuracy by calculating the root-mean-squared error (RMSE) between the estimated transformation and the ground truth transformation. The results obtained using the joint intensity-Fourier descriptor were compared to the results obtained using Harris, Minimum eigenvalue, Features accelerated segment test (FAST), speeded-up robust features (SURF), binary robust invariant scalable keypoints (BRISK) and KAZE keypoint descriptors. The joint intensity-Fourier descriptor yielded average RMSE of 0.446±0.359 pixels and 1.152±0.488 pixels on two satellite imagery datasets consisting of 35 image pairs in total. These results indicate the capacity of the proposed technique for high accuracy. Our method also produces a lower registration error than the compared feature-based methods.

Copy-move forgery detection based on adaptive keypoints extraction and matching

Copy-move (region duplication) is one of the most common types of image forgeries, in which at least one part of an image is copied and pasted onto another area of the same image. The main aims of the copy-move forgery are to overemphasize a concept or conceal objects by duplicating some regions. Keypoint-based copy-move forgery detection (CMFD) schemes extract image keypoints and employ local image features to identify duplicated regions, which exhibits remarkable detection performance with respect to memory requirement, computational cost, and robustness. To enhance the performance of keypoint-based CMFD approaches, here are three issues that need to be solved: the non-uniform distribution of image keypoints, the low discriminatory power of local image descriptor, and the high computational cost and low matching efficiency of feature matching strategy. In order to overcome these issues, we propose a new copy-move forgery detection method based on adaptive keypoints extraction and matching in this paper. First, we extract the image keypoints using the adaptive uniform distribution threshold. Second, the binary robust invariant scalable keypoints (BRISK) descriptor is introduced to represent the local image feature of image keypoints. Afterwards, local BRISK descriptors are employed to match image keypoints by using embedded random ferns approach, which formulates the required matching as a discriminative classification problem. Finally, the falsely matched keypoints pairs are eliminated by utilizing the random sample consensus (RANSAC), and the fast mean-residual normalized intensity correlation (NNPROD) is employed to locate the tampering area. We evaluate the performance of the proposed CMFD method in detail by conducting several simulation experiments, and the experimental results have shown that the detection and localization accuracy of the proposed method is superior to that of the state-of-the-art approaches recently proposed in the literature, even in adverse conditions.

ЛОГІЧНИЙ АНАЛІЗ ТА ОБРОБЛЕННЯ ДАНИХ ЗАДЛЯ КЛАСИФІКАЦІЇ ЗОБРАЖЕНЬ НА ПІДСТАВІ ФОРМУВАННЯ СТАТИСТИЧНОГО ЦЕНТРУ ОПИСУ

Предметом досліджень є моделі для класифікації зображень у просторі описів як множини дескрипторів ключових точок при розпізнаванні візуальних об’єктів у системах комп’ютерного зору. Метою є розвинення структурного методу класифікації шляхом впровадження логічного оброблення даних із використанням ймовірнісного розподілу у вигляді статистичного центру. Завдання: розроблення математичних та програмних моделей для обчислення релевантності описів зображень із використанням логічного аналізу, вивчення властивостей, варіантів застосування, значень параметрів моделей, оцінювання результативності за наслідками оброблення експериментальної бази зображень. Застосовуваними методами є: детектор BRISK для формування дескрипторів ключових точок, інтелектуальний аналіз даних, математична статистика, засоби визначення релевантності для множин даних, програмне моделювання. Отримані результати: ефективність способу класифікації на основі логічного аналізу з використанням статистичних центрів залежить від відстаней між центрами еталонів бази. Застосування логічного аналізу спрощує оброблення і підвищує швидкодію класифікації. Найкращі результати щодо класифікації окремих дескрипторів показав підхід з використанням уточнених центрів. Використання концентрованої частки даних опису дає можливість ретельніше зосередитися на його відмінностях з іншими описами. Висновки. Наукова новизна – удосконалення методу класифікації зображень на основі впровадження логічного аналізу на підставі статистичного центру опису, що дає можливість модифікувати склад опису зі збереженням властивостей об’єктів в аспекті результативної класифікації. Практична значущість роботи полягає у досягненні прийнятого рівня ефективності класифікації за визначеною моделлю релевантності, підтвердженні працездатності запропонованих модифікацій оброблення даних на прикладах зображень, розробленні програмних моделей для впровадження описаних методів класифікації у системах комп’ютерного зору.

Ground target localization and recognition via descriptors fusion

Keypoint matching can be defined as locating the position of a particular point in two images precisely. Recently, keypoint descriptors have taken a great effect targeting to be powerfully invariant to rotation, scale and translation for improving target detection. The detection task is carried out by a reference-scene image matching to localize the desired target in the input scene. An innovative approach is proposed in this work to fuse the state-of-the-art feature descriptors ORB, BRISK for the sake of accurate ground target detection in two phases. Firstly, off-line phase, where the fused features are extracted from different perspective, azimuth angles of the desired target to build a comprehensive reference image representation. Secondly, on-line phase, where the fused features extraction task is carried out from the whole scene. Hence, it is matched with the stored reference one to find the keypoints correspondence. The outliers’ problem is eliminated using Random Sample Consensus (RANSAC) algorithm resulting in speeding up the matching procedure. The conducted comparative analysis has revealed the discriminative power of the fused features in localization and recognition tasks while keeping the proposed system works in real-time.

Copy move forgery detection based on keypoint and patch match

Copy move has become a simple and effective operation for image forgeries due to the advancement of image editing software, which is still challenging to be detected. In this paper, a novel method is proposed for copy move forgery detection based on Keypoint and Patch Match. Local Intensity Order Pattern (LIOP), a robust keypoint descriptor, is combined with SIFT to obtain reliable keypoints. After using g2NN to match the extracted keypoints, a new matched keypoint pair description model and a density grid-based filtering strategy are applied to removing the redundancy matched keypoint pairs. Finally an enhanced patch match approach is utilized to examine the matched keypoint pairs to accurately determine the existence of forgery. Compared with the state-of-the-art methods, the proposed method can detect copy move region more precisely according to the experimental result, even when detected objects are distorted by some processing such as rotation, scaling, JPEG compression and additional noise.

A comparison of local features for camera-based document image retrieval and spotting

This paper aims at comparing robustness of local features for camera-based document image retrieval and spotting system. We present a literature review of the state of the art of local features extraction that includes keypoint detectors and keypoint descriptors. We also present a dataset and evaluation protocol for camera-based document image retrieval and spotting systems. This dataset is composed of three subparts: The first dataset represents the images with textual content only; the second dataset represents images with graphical content mainly; the third dataset contains text plus graphical elements. Along with the datasets, we present the protocol that describes measurements to evaluate the accuracy and processing time of camera-based document image retrieval and spotting systems. The latter is employed for presenting a detailed evaluation of local features from the literature.

THE STUDY OF STATISTICAL PROPERTIES OF THE MODEL BLOCK REPRESENTATION FOR SET OF DESCRIPTORS OF KEY POINTS OF IMAGES

Context. The multidimensional nature of the processed data in modern computer vision systems requires new approaches to theconstruction of effective feature spaces that simplify processing by summarizing the available information. Structural methods ofimage recognition use descriptions of visual objects in the form of sets of key point descriptors as sets of numerical vectors of highdimension. The main tool for reducing the dimension is the presentation of data in the form of a system of their blocks and a statisticalstudy of such data structures, which in terms of recognition should display the generalized properties of an object as a set of itsfragments. In this connection, there is problem of studying the features of the applied application and the characteristics of the blockrepresentation model in the aspect of its use to determine the relevance of descriptions and classify data within the base of referenceimages.Objective. Perform statistical estimation of the importance of making classification decisions on the basis of the calculation ofthe relevance of object descriptions for the model of block representation of descriptors of key points of images.Method. Methods are proposed for distinguishing descriptions based on the application of the block representation model of thedata descriptors of key points of images using the criteria of mathematical statistics and information theory tools.Results. The main result of the article is the confirmation that the use of classical statistical criteria for analyzing empirical datain the form of structural descriptions of images makes it possible to determine the quality factor of the constructed feature space fordistinguishing visual objects when they are recognized in computer vision systems. The introduction of the block representationmodel and statistical analysis for the values of descriptors of key features of images contributes to the efficiency of the process ofrecognizing visual objects, which is confirmed by the improvement in the level of difference by increasing the fragment size in theconstructed description chain structure.Conclusions. The use of a variety of statistical criteria gave an identical effect on the significance of the differences in the empiricaldescriptions of visual objects in the constructed feature space, which underlines the objectivity of the study. The implementedmodel of block representation of data retains the distinguishing properties of a structural description with the effect of a significantimprovement in the speed of classification decision making.The scientific novelty of the study is the improvement and statistical substantiation of models for making decisions about theclass of visual objects based on the calculation of the relevance of their descriptions with references using a block representation ofdescriptors of key points of images.The practical significance of the work is confirming the advisability of introducing a block structure for the block description ofan object as an effective approach to solving the problem of recognition with examples of images for implementation in computervision systems.

Distributed image retrieval with colour and keypoint features

ABSTRACTContent-based image retrieval poses many problems to computer systems. The content of images has to be described by some feature extraction methods. As image databases are often very large, they are sometimes to complex to be processed by traditional computing methods. We have to use big data solutions to fast retrieve images. The paper presents a system for retrieving images in relational databases in a distributed environment. Content of the query image and images in the database is compared using global colour information and local image keypoints. Image keypoint descriptors are indexed by fuzzy sets directly in a relational database by our algorithm. The process is distributed to several machines thanks to the Apache Hadoop software framework with HDFS.

IRIS IMAGE KEY POINTS DESCRIPTORS BASED ON PHASE CONGRUENCY

Abstract. In this article the new method for iris image features extraction based on phase congruency is proposed. Iris image key points are calculated using the convolutions with Hermite transform functions. At each key point the feature vector characterizing this key point is obtained based on the phase congruency method. Iris key point descriptor contains phase congruency values at points located on concentric circles around the key point. To compare the key points, Euclidean metric between the key points descriptors is calculated. The distance between the iris images is equal to the number of matched iris key points. The proposed method was tested using the images from CASIA−IrisV4−Interval database and the value of EER = 0.226% was obtained.

ARCHITECTURAL OBJECTS RECOGNITION TECHNIQUE IN AUGMENTED REALITY TECHNOLOGIES BASED ON CREATING A SPECIALIZED MARKERS BASE

The paper proposes a method for recognizing architectural objects when creating augmented reality mobile applications based on building a database of specialized markers. The main method of augmented reality technology for the recognition of architectural objects was chosen - the technology based on special markers. The range of pattern recognition algorithms suitable for the task is highlighted. These are algorithms based on the selection of key points of images and their descriptors. The most important aim is the stable recognition of architectural objects upon mobile applications for augmented reality-type digital guide creation based on specialized markers. The scientific basis of the research is a systematic approach in the analysis of the considered markers recognition algorithms, machine learning for the development of a database of marker images and AO recognition are used. The technique consists of the following steps: processing images of architectural objects with the aim of identifying key points, obtaining descriptions of selected control points as descriptors, creating AR-metadata that correspond to architectural objects, organizing joint storage in the local database of descriptors and their corresponding metadata, visualizing the architectural object and AR metadata. To implement the stages of processing images of architectural objects and obtain descriptors of key points, algorithms for extracting key points on images, such as SIFT, MSER, SURF, RIFF, RF, are analyzed. It is shown that these algorithms are invariant to scaling, rotation, as well as resistant to changes in light, noise and viewing angle. A comprehensive use of them for processing architectural objects with the aim of obtaining descriptors of reference points was proposed. To ensure stable recognition of AO according to the developed methodology based on machine learning for processing architectural objects with the aim of obtaining descriptors of key points, it was proposed to create an additional module using an ordered stack. The launch sequence and the number of algorithms can be changed.

3D Object Recognition with Keypoint Based Algorithms

Object recognition is important in many practical applications of computer vision. Traditional 2D methods are negatively affected by illumination, shadowing and viewpoint. 3D methods have the potential to solve these problems, because 3D models include geometric properties of the objects. In this paper, 3D local feature based algorithms were used for 3D object recognition. The local feature was keypoint. This study aimed to research facilities of keypoints for 3D object recognition. Keypoint is feature of object that is detected by detector algorithms according to certain mathematical base. A recognition system was designed. For this purpose, a database that includes 3D model of objects was created. The algorithms were improved in MATLAB. The keypoints on the 3D models were detected using keypoint detectors. These keypoints were described by keypoints descriptors. The descriptor algorithms detect geometrical relation between each point of point cloud and create a histogram. In the third step, the keypoints in different point clouds are matched using the feature histograms obtained. Statistical methods are used to compare generated histograms. Thus, the two closest similar points between the different point clouds are matched. It is expected that the models with the most corresponding points belong to the same object. Euclidean distance between corresponding keypoints in the two point cloud is calculated. It has been accepted that the points are shorter than 10 mm. The positional accuracy of the matched points has been examined. Iterative Closest Point (ICP) was applied to the matching point clouds for this purpose. As a result, the graphics were generated that showed correct matching ratio and root mean square error. As a result, there are different approaches about 3D object recognition in literature. This study aimed to compare different keypoint detector and descriptor algorithms. Intrinsic Shape Signature (ISS) is keypoint detector algorithms. Point Feature Histograms (PFH) and Fast Point Feature Histograms (FPFH) are keypoint descriptor algorithms. The results of this study will provide guidance for future studies.

ДОСЛІДЖЕННЯ СИСТЕМИ ІЄРАРХІЧНИХ ОЗНАК ПРИ БЛОЧНОМУ ПОДАННІ ОПИСУ У СКЛАДІ МНОЖИНИ КЛЮЧОВИХ ТОЧОК ЗОБРАЖЕННЯ

Предметом досліджень статті є ієрархічні моделі для встановлення ступеня релевантності описів зображень при розпізнаванні візуальних об’єктів у системах комп’ютерного зору. Метою є розроблення модифікації методу структурного розпізнавання на підставі впровадження блокових моделей даних із інтегруванням ймовірнісних розподілів. Завдання: розроблення математичних та програмних моделей для ефективного за швидкодією ієрархічного оброблення даних при визначенні релевантності структурних описів, вивчення властивостей цих моделей, оцінювання результативності при обробленні зображень. Застосовуваними методами є: детектор ORB для формування дескрипторів ключових точок, інтелектуальний аналіз даних, методи побудови розподілів бітових даних, апарат метричного визначення релевантності, програмне моделювання. Отримані такі результати. Перехід від опису множин дескрипторів до розподілів фрагментів, побудова ієрархічних ознак забезпечують необхідну результативність розпізнавання. Оброблення та аналіз даних виконується у кілька разів швидше, ніж на підставі розподілів. Висновки. Наукова новизна дослідження полягає в удосконаленні методу структурного розпізнавання зображень на основі впровадження блочної структури опису із використанням інтегрованих значень розподілу для фрагментів множини дескрипторів. Практична значущість – досягнення суттєвого рівня підвищення швидкодії при обчисленні релевантності, підтвердження результативності запропонованих ієрархічних ознак на прикладах зображень, отримання прикладних програмних моделей для дослідження та впровадження методів класифікації системах комп’ютерного зору.

Coarse Alignment for Model Fitting of Point Clouds Using a Curvature-Based Descriptor

This paper presents a method for coarse alignment of point clouds by introducing a new descriptor based on the local curvature. The method is developed for model fitting a CAD model for use in robotic assembly. The method is based on selecting keypoints depending on shape factors calculated from the local covariance matrix of the surface. A descriptor is then calculated for each keypoint by fitting two spheres that describe the curvature of the surface. The spheres are calculated using conformal geometric algebra, which gives a convenient and efficient description of the geometry. The keypoint descriptors for the model and the observed point cloud are then compared to estimate the corresponding keypoints, which are used to calculate the displacement. The method is tested in several experiments. One experiment is for robotic assembly, where objects are placed on a table and their position and orientation is estimated using a 3-D CAD model. Note to Practitioners — 3-D cameras can be used in robotic assembly for recognizing objects, and for determining the position and orientation of parts to be assembled. In such applications, 3-D CAD models will be available for the objects, and point clouds representing each object can be generated for comparison with the observed point clouds from the 3-D camera. It is not straightforward to use existing descriptors in this paper, as the point cloud from the CAD model and the observed point cloud may differ due to different viewpoints and potential occlusions. The method proposed in this paper is intended to be easy to apply to industrial assembly problems where there is a need for a robust estimation of the displacement of an object, either as a coarse estimate for use in grasping or as an initial guess to use in fine registration for demanding assembly operations with close tolerances. The method exploits the curvature of the point clouds to accurately describe the surrounding surface of each point. This method serves as a basis for future industrial implementations.

Analysis of Tuberculosis in Chest Radiographs for Computerized Diagnosis using Bag of Keypoint Features.

Chest radiography is the most preferred non-invasive imaging technique for early diagnosis of Tuberculosis (TB). However, lack of radiological expertise in TB detection leads to indiscriminate chest radiograph (CXR) screening. A modest classification approach based on the local image description to detect subtle characteristics of TB using CXRs is highly recommended. In this work, an attempt has been made to classify normal and TB CXR images using Bag of Features (BoF) approach with Speeded-Up Robust Feature (SURF) descriptor. The images are obtained from a public database. Lung fields segmentation is performed using Distance Regularized Level Set (DRLS) formulation. The results of segmentation are validated against the ground truth images using similarity, overlap and area correlation measures. BoF approach with SURF keypoint descriptors is implemented to categorize the images using Multilayer Perceptron (MLP) classifier. The obtained results demonstrate that the DRLS method is able to delineate lung fields from CXR images. The BoF with SURF keypoint descriptor is able to characterize local attributes of normal and TB images. The segmentation results are found to be in high correlation with ground truth. MLP classifier is found to provide high Recall, Specificity (Spec), Accuracy, F-score and Area Under the Curve (AUC) values of 87.7%, 85.9%, 87.8%, 87.6% and 94% respectively between normal and abnormal images. The proposed computer aided diagnostic approach is found to perform better as compared to the existing methods. Thus, the study can be of significant assistance to physicians at the point of care in resource constrained regions.

Low-dimensional superpixel descriptor and its application in visual correspondence estimation

Establishing local visual correspondence between video frames is an important and challenging problem in many vision based applications. Local keypoint detection and description based pixel-level matching is a typical way for visual correspondence estimation. Unlike traditional local keypoint descriptor based methods, this paper proposes a comprehensive yet low-dimensional local feature descriptor based on superpixels generated by over segmentation. The proposed local feature descriptor extracts shape feature, texture feature, and color feature from superpixels by orientated center-boundary distance (OCBD), gray-level co-occurrence matrix (GLCM), and saturation histogram (SHIST), respectively. The types of features are more comprehensive than existing descriptors which extract only one specific kind of feature. Experimental results on the widely used Middlebury optical flow dataset prove that the proposed superpixel descriptor achieves triple accuracy compared with the state-of-the-art ORB descriptor which has the same dimension of features with the proposed one. In addition, since the dimension of the proposed superpixel descriptor is low, it is convenient for matching and memory-efficient for hardware implementation.

Hierarchical Graph based Segmentation and Consensus based Human Tracking Technique

Accurate detection, tracking and analysis of human movement using robots and other visual surveillance systems is still a challenge. Efforts are on to make the system robust against constraints such as variation in shape, size, pose and occlusion. Traditional methods of detection used the sliding window approach which involved scanning of various sizes of windows across an image. This paper concentrates on employing a stateof- the-art, hierarchical graph based method for segmentation. It has two stages: part level segmentation for color-consistent segments and object level segmentation for category-consistent regions. The tracking phase is achieved by employing SIFT keypoint descriptor based technique in a combined matching and tracking scheme with validation phase. Localization of human region in each frame is performed by keypoints by casting votes for the center of the human detected region. As it is difficult to avoid incorrect keypoints, a consensus-based framework is used to detect voting behavior. The designed methodology is tested on the video sequences having 3 to 4 persons.

Image Retrieval using Visual Phrases

Keypoint based descriptors are widely used for various computer vision applications. During this process, key-points are initially detected from the given images which are later represented by some robust and distinctive descriptors like scale-invariant feature transform (SIFT). Keypoint based image-to-image matching has gained significant accuracy for image retrieval type of applications like image copy detection, similar image retrieval and near duplicate detection. Local keypoint descriptors are quantized into visual words to reduce the feature space which makes image-to-image matching possible for large scale applications. Bag of visual word quantization makes it efficient at the cost of accuracy. In this paper, the bag of visual word model is extended to detect frequent pair of visual words which is known as frequent item-set in text processing, also called visual phrases. Visual phrases increase the accuracy of image retrieval without increasing the vocabulary size. Experiments are carried out on benchmark datasets that depict the effectiveness of proposed scheme.

Towards Accurate and Robust Multi-Modal Medical Image Registration Using Contrastive Metric Learning

Multi-modal medical image registration takes an essential role in image-based clinical diagnosis and surgical planning. It is not trivial due to appearance variations across different modalities. Rigidly aligning two images is used to register rigid body structure, and it is also usually the first step for deformable registration with a large discrepancy. In the field of computer vision, one well-established method for image alignment is to find corresponding points from two images, and image alignment is based on identified corresponding points. Our method lies in this category. Feature representation is crucial in finding corresponding points. However, conventional feature representation like SIFT does not take multi-modal information into account, and thus, it fails. In this paper, we propose a Convolution Neural Network Feature-based Registration (CNNFR) method for aligning the multi-modal medical image. The important component in this method is learning keypoint descriptors using contrastive metric learning, which minimizes the difference between two feature representations from two corresponding points and maximizes difference of two feature representation from two distant points. Also, a transfer learning-based CNNRF (TrCNNRF) is proposed to improve the generalization learning performance when the training data are insufficient. Experimental results demonstrate that the proposed methods can achieve superior performance regarding both accuracy and robustness, which can be used to rigidly register multi-modal images and provide an initial estimation for non-rigid registration in clinical practices.