Local Invariant Features Research Articles

URINet: Unsupervised point cloud rotation invariant representation learning via semantic and structural reasoning

In recent years, many rotation-invariant networks have been proposed to alleviate the interference caused by point cloud arbitrary rotations. These networks have demonstrated powerful representation learning capabilities. However, most of those methods rely on costly manually annotated supervision for model training. Moreover, they fail to reason the structural relations and lose global information. To address these issues, we present an unsupervised method for achieving comprehensive rotation invariant representations without human annotation. Specifically, we propose a novel encoder–decoder architecture named URINet, which learns a point cloud representation by combining local semantic and global structural information, and then reconstructs the input without rotation perturbation. In detail, the encoder is a two-branch network where the graph convolution based structural branch models the relationships among local regions to learn global structural knowledge and the semantic branch learns rotation invariant local semantic features. The two branches derive complementary information and explore the point clouds comprehensively. Furthermore, to avoid the self-reconstruction ambiguity brought by uncertain poses, a bidirectional alignment is proposed to measure the quality of reconstruction results without orientation knowledge. Extensive experiments on downstream tasks show that the proposed method significantly surpasses existing state-of-the-art methods on both synthetic and real-world datasets.

C-LFNet: Central-Local Feature 3D point cloud instance segmentation Network for robot bin-picking

We propose a novel instance segmentation algorithm called Central-Local Feature Network (C-LFNet) to address the low performance of point cloud segmentation in bin-picking. First, to address the issue of batch processing data, we propose a downsampling algorithm that can sample the number of target points while preserving the density distribution of points in space as much as possible both before and after sampling. Second, we design a dynamic graph convolution backbone to better extract the local geometric invariant features, referred to as Central-Local Feature Module (C-LFM). Finally, the results of each point estimate are clustered in order to enable instance segmentation of the point cloud. The proposed network is evaluated using the Fraunhofer IPA Bin-Picking dataset and compared to existing methods. Results demonstrate that the proposed network is effective and robust in bin-picking scenes to achieve the same category of point cloud instance segmentation, resulting in improved performance.

Convolutional Neural Networks for Valid and Efficient Causal Inference

Convolutional neural networks (CNN) have been successful in machine learning applications. Their success relies on their ability to consider space invariant local features. We consider the use of CNN to fit nuisance models in semiparametric estimation of the average causal effect of a treatment. In this setting, nuisance models are functions of pretreatment covariates that need to be controlled for. In an application where we want to estimate the effect of early retirement on a health outcome, we propose to use CNN to control for time-structured covariates. Thus, CNN is used when fitting nuisance models explaining the treatment and the outcome. These fits are then combined into an augmented inverse probability weighting estimator yielding efficient and uniformly valid inference. Theoretically, we contribute by providing rates of convergence for CNN equipped with the rectified linear unit activation function and compare it to an existing result for feedforward neural networks. We also show when those rates guarantee uniformly valid inference. A Monte Carlo study is provided where the performance of the proposed estimator is evaluated and compared with other strategies. Finally, we give results on a study of the effect of early retirement on hospitalization using data covering the whole Swedish population. Supplementary materials are available online at https://github.com/stat4reg/Causal/_CNN.

Robust Hashing via Global and Local Invariant Features for Image Copy Detection

Robust hashing is a powerful technique for processing large-scale images. Currently, many reported image hashing schemes do not perform well in balancing the performances of discrimination and robustness, and thus they cannot efficiently detect image copies, especially the image copies with multiple distortions. To address this, we exploit global and local invariant features to develop a novel robust hashing for image copy detection. A critical contribution is the global feature calculation by gray level co-occurrence moment learned from the saliency map determined by the phase spectrum of quaternion Fourier transform, which can significantly enhance discrimination without reducing robustness. Another essential contribution is the local invariant feature computation via Kernel Principal Component Analysis (KPCA) and vector distances. As KPCA can maintain the geometric relationships within image, the local invariant features learned with KPCA and vector distances can guarantee discrimination and compactness. Moreover, the global and local invariant features are encrypted to ensure security. Finally, the hash is produced via the ordinal measures of the encrypted features for making a short length of hash. Numerous experiments are conducted to show efficiency of our scheme. Compared with some well-known hashing schemes, our scheme demonstrates a preferable classification performance of discrimination and robustness. The experiments of detecting image copies with multiple distortions are tested and the results illustrate the effectiveness of our scheme.

An Optical Remote Sensing Image Matching Method Based on the Simple and Stable Feature Database

Satellite remote sensing has entered the era of big data due to the increase in the number of remote sensing satellites and imaging modes. This presents significant challenges for the processing of remote sensing systems and will result in extremely high real-time data processing requirements. The effective and reliable geometric positioning of remote sensing images is the foundation of remote sensing applications. In this paper, we propose an optical remote sensing image matching method based on a simple stable feature database. This method entails building the stable feature database, extracting local invariant features that are comparatively stable from remote sensing images using an iterative matching strategy, and storing useful information about the features. Without reference images, the feature database-based matching approach potentially saves storage space for reference data while increasing image processing speed. To evaluate the performance of the feature database matching method, we train the feature database with various local invariant feature algorithms on different time phases of Gaofen-2 (GF-2) images. Furthermore, we carried out matching comparison experiments with various satellite images to confirm the viability and stability of the feature database-based matching method. In comparison with direct matching using the classical feature algorithm, the feature database-based matching method in this paper can essentially improve the correct rate of feature point matching by more than 30% and reduce the matching time by more than 40%. This method improves the accuracy and timeliness of image matching, potentially solves the problem of large storage space occupied by the reference data, and has great potential for fast matching of optical remote sensing images.

Satellite image matching and registration using affine transformation and hybrid feature descriptors

Image registration (IR) is an image processing technique to determine geometrical transformation that gives the most accurate match between reference and floating images. In existing IR work, the inliers ratio and outlier's ratios are equal. This reduces the image registration accuracy. So, to avoid the outlier's ratio and increase the inliers ratio, hybrid invariant local features descriptor is proposed. This feature descriptor consists of binary-robust-invariant-scalable-key point (BRISK), speed-up-robust-feature (SURF), and feature from accelerated segment test feature (FAST). The hybrid feature descriptor extracts the relevant features from the image. Then, the feature matching step finds the correct correspondences from the two sets of features. Also, affine transformation avoids the false feature matching points. An experimental analysis shows the inliers ratio and repeatability evaluation metrics performance of individual feature descriptor, combined feature descriptor and proposed feature descriptor. The proposed hybrid feature descriptor achieves inliers ratio of 1.913 and repeatability is 0.121.

Ensembling Scale Invariant and Multiresolution Gabor Scores for Palm Vein Identification

Biometric recognition based on palm vein trait has the advantages of liveness detection and high level of security. An improved human palm vein identification system based on ensembling the scores computed from scale invariant features and multiresolution adaptive Gabor features is proposed. In the training phase, from the input palm vein images, the interested palm regions are segmented using 3-valley point maximal palm extraction strategy, an improved method that extracts the maximal region of interest (ROI) easily and properly. Extracted ROI is enhanced using contrast limited adaptive histogram equalization method. From the enhanced image, local invariant features are extracted by applying scale invariant feature transform (SIFT). The texture and multiresolution features are extracted by employing adaptive Gabor filter over the enhanced image. These two features, scale invariant and multiresolution Gabor features act as the templates. In the testing phase, for the test images, ROI extraction, image enhancement, and two different feature extractions are performed. Using cosine similarity and match count-based classification, the score, Ss is computed for the SIFT features. Another score, Sg is computed using the normalized Hamming distance measure for the Gabor features. Both these scores are ensembled using the weighted sum rule to produce the final score, SF for identifying the person. Experiments conducted with CASIA multispectral palmprint image database version 1.0 and VERA palm vein database show that, the proposed method achieves equal error rate of 0.026% and 0.0205% respectively. For these databases, recognition rate of 99.73% and 99.89% respectively are obtained which is superior to the state-of-the-art methods in authentication and identification. The proposed work is suitable for applications wherein the authenticated person should not be considered as imposter.

Improved deep generative adversarial network with illuminant invariant local binary pattern features for facial expression recognition

ABSTRACT Facial Expression Recognition (FER) is one of the prevailing as well as demanding tasks in social communication. In general, face expressions are usual and straight ways for individuals to converse their intentions as well as emotions. This paper proposes the Taylor-Chicken Swarm Optimization-based Deep Generative Adversarial Network (Taylor-CSO-based Deep GAN) for FER. Here, the facial expressions are predicted through a series of steps, such as video frame extraction, pre-processing, face detection, feature extraction, as well as FER. The major contribution of the proposed work lies in the last step, where Taylor-CSO based Deep GAN is employed for recognizing facial expressions. Initially, video frames are extracted from the input video as well as pre-processing is done to extract the Region of Interest (RoI). Then, the Viola Jones algorithm is employed to detect the face images, and Illuminant Invariant Local Binary Pattern (IILBP) features are extracted. Finally, the FER is performed by the proposed model. The performance of the developed model is analyzed using the CK+ dataset, RAVDESS dataset, and SAVEE dataset. Also, the developed model’s performance is evaluated with conventional FER methods. The performance analysis exhibits that the adopted model is precise and valuable.

Matching wide-baseline stereo images with weak texture using the perspective invariant local feature transformer

The development of remote sensing sensor techniques allows us to now readily capture many types of indoor and outdoor scene images, which often include many weak texture regions with notable geometric distortions. Obtaining qualified matches from these difficult stereo images using existing methods is challenging. The recent achievements of deep-learning models have shown that the convolutional neural network (CNN) is adept at the image matching task. However, in practical applications, the following challenges remain: first, it is difficult to detect features in the weak texture regions of an image, and existing CNNs fail to extract discriminative image information from the quantized features of weak texture; second, as a result of the complex distortion across wide-baseline stereo images, it is difficult to match feature primitives detected in the image pair. To solve these problems, we propose the perspective invariant local feature transformer (PILFT) algorithm. Our method includes four main steps. (1) The affine scale-invariant feature transform is proposed to automatically extract the corresponding features from images, and then the perspective of the matched image is corrected to eliminate as much geometric deformation as possible. (2) The residual network is used to extract potential features from stereo images to obtain coarse and fine feature maps at different scales. (3) Using an attention mechanism, location and context information are added to the coarse level features, which are predicted by a dual-softmax function layer. (4) The features are precisely predicted on the fine feature map using the coarse reference, and the final matching results are determined by calculating the matching probability. A large number of experiments on wide-baseline weak texture images demonstrate that the proposed method has advantages over the existing algorithms in the number of matches, correct match rate, and matching accuracy. The pseudocodes of PILFT are available at https://github.com/KiltAB/PILFT.

A Scattering Wavelet Network-based Approach to Fingerprint Classification

In a large-scale automatic fingerprint identification system (AFIS), fingerprint classification is an essential indexing step to reduce the search time in a large database for accurate matching. Fingerprint classification is still a challenging machine learning problem due to large intra-class and small inter-class variability. Nonlinear elastic deformation is one of the main sources of intra-class variability, which occurs due to the non-uniform pressure applied during fingerprint acquisition and the elastic nature of the fingerprint itself. This paper proposes a novel approach to fingerprint classification based on a scattering wavelet network to extract translation and small deformation invariant local features. The resulting sparse invariant feature vectors are used as input to a simple generative PCA affine classifier for the classification. The experiments evaluated with two different protocols on a benchmark NIST SD-4 database show the effectiveness and robustness of the proposed fingerprint classification algorithm in terms of classification accuracy.

Fusion of Novelty Detectors Using Deep and Local Invariant Visual Features for Inspection Task

In this study, a novel framework using multiple novelty detection filters is developed to learn a model of the normality of a robot’s visual perception, which is called multichannel novelty detection. Subsequently, the acquired model was used to highlight dissimilar perceptions when the robot explored an environment. The main purpose of fusing multiple novelty filters is that each novelty filter performs well in detecting specific types of novelties; therefore, a new framework is proposed that demonstrates a new way to combine multiple different purposed novelty detection filters together in order to yield an overall more robust novelty status on the visual features. To develop a multichannel novelty detection system, expectation- and appearance-based novelty detection models were used in this study. To become experts in detecting different types of novelty using these models, different features from the input image were extracted as inputs for the models. The expectation-based novelty detection model uses the MobileNetV2 deep network to extract the deep features of the input image, which is subsequently used to learn a sequential and temporal model of normality to detect novelty. By contrast, the appearance-based novelty detection model uses speeded up robust features (SURF), which provide more region-focused features within the input image, to identify whether a specific region of the image is novel. The proposed multichannel novelty detection system is a completely online and real-time approach that is very important for mobile robotics applications. The proposed framework was tested in three novel environments, and it was reported that the proposed multichannel novelty detection system performs better than expectation-based and appearance-based novelty filters separately. Statistically, in the three novel environments, the Matthews correlation coefficients are reported to be 0.94, 0.97, and 0.93, and <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">F</i> <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sub> scores are reported to be 0.95, 0.97, and 0.93, respectively, which proves that and can be concluded as almost perfect statistically.

Efficient photometric and geometric stitching approach for remote sensing images based on wavelet transform and local invariant

Image stitching is an active research area in many photogrammetric and remote sensing applications. The main contribution of our work is to regard stitching as a multi-image matching problem that exploits advantages of invariant local features and homographic transformation to overcome photometric, geometric, and perspective distortions. Our proposed approach, referred to as Haar Invariant Feature with PROSAC, consists of three main steps. The first step extracts a set of independent keypoints by an adaptive local feature extractor based on Haar wavelet transform and scale invariant feature transform (SIFT). Then to increase the precision of matched points, we propose to substitute classical matching algorithms by a fast Haar-k-nearest neighbor algorithm and estimate the homography matrix using the Progressive Sample Consensus algorithm. Finally, we warp images according to the projective transformation and blend them with a Laplacian multi-band to create a perfect panorama. Experiments on various images show the effectiveness and the robustness of our approach compared to common image matching techniques such as SIFT, speeded up robust features, optimized random sample consensus with contrario, and image matching by affine simulation.

Dual adaptive alignment and partitioning network for visible and infrared cross-modality person re-identification

Visible and infrared person re-identification (VI-ReID) describes the task of matching the images of a person, captured by visible-light and infrared cameras; this is a particular challenge in night time surveillance applications. Existing cross-modality recognition studies have been conducted mainly with a focus on learning the global and shareable feature representation of pedestrians to handle cross-modality discrepancies. However, the global features of pedestrian images cannot solve the unaligned image pairs efficiently, particularly when encountering the human appearance or posture misalignment caused by inaccurate pedestrian detection boxes. To mitigate the impact of these problems, we propose an end-to-end dual alignment and partitioning network to simultaneously learn global and local modal invariant features of pedestrians. First, we use two adaptive spatial transform modules to align the visible and infrared input images. Subsequently, the aligned image is divided horizontally, and the features of each local block are extracted. Then, we fuse these local features with global features. To alleviate the differences between heterogeneous modals and learn the common feature representation of heterogeneous modals, we map the features of heterogeneous modes into the same feature embedding space. Finally, we use the combination of identity loss and weighted regularized TriHard loss to improve the recognition accuracy. Extensive experimental results on two cross-modality datasets, RegDB and SYSU-MM01, demonstrate the superiority of the proposed method over other existing state-of-the-art methods.

Keypoint detection by wave propagation

We propose to rely on the wave equation for the detection of repeatable keypoints invariant up to image scale and rotation and robust to viewpoint variations, blur, and lighting changes. The algorithm exploits the properties of local spatial–temporal extrema of the evolution of image intensities under the wave propagation to highlight salient symmetries at different scales. Although the image structures found by most state-of-the-art detectors, such as blobs and corners, occur typically on highly textured surfaces, salient symmetries are widespread in diverse kinds of images, including those related to poorly textured objects, which are hardly dealt with by current pipelines based on local invariant features. The impact on the overall algorithm of different numerical wave simulation schemes and their parameters is discussed, and a pyramidal approximation to speed-up the simulation is proposed and validated. Experiments on publicly available datasets show that the proposed algorithm offers state-of-the-art repeatability on a broad set of different images while detecting regions that can be distinctively described and robustly matched.

Robust and ultrafast fiducial marker correspondence in electron tomography by a two-stage algorithm considering local constraints.

MotivationElectron tomography (ET) has become an indispensable tool for structural biology studies. In ET, the tilt series alignment and the projection parameter calibration are the key steps toward high-resolution ultrastructure analysis. Usually, fiducial markers are embedded in the sample to aid the alignment. Despite the advances in developing algorithms to find correspondence of fiducial markers from different tilted micrographs, the error rate of the existing methods is still high such that manual correction has to be conducted. In addition, existing algorithms do not work well when the number of fiducial markers is high.ResultsIn this article, we try to completely solve the fiducial marker correspondence problem. We propose to divide the workflow of fiducial marker correspondence into two stages: (i) initial transformation determination, and (ii) local correspondence refinement. In the first stage, we model the transform estimation as a correspondence pair inquiry and verification problem. The local geometric constraints and invariant features are used to reduce the complexity of the problem. In the second stage, we encode the geometric distribution of the fiducial markers by a weighted Gaussian mixture model and introduce drift parameters to correct the effects of beam-induced motion and sample deformation. Comprehensive experiments on real-world datasets demonstrate the robustness, efficiency and effectiveness of the proposed algorithm. Especially, the proposed two-stage algorithm is able to produce an accurate tracking within an average of ms per image, even for micrographs with hundreds of fiducial markers, which makes the real-time ET data processing possible.Availability and implementationThe code is available at https://github.com/icthrm/auto-tilt-pair. Additionally, the detailed original figures demonstrated in the experiments can be accessed at https://rb.gy/6adtk4.Supplementary information Supplementary data are available at Bioinformatics online.

Ultrasonic Image Diagnosis of Liver and Spleen Injury Based on a Double‐Channel Convolutional Neural Network

Automatic and accurate diagnosis of liver and spleen injury in ultrasonic images is of great significance for the development of automatic clinical diagnosis. In order to realize more accurate ultrasonic image diagnosis of liver and spleen injury, an algorithm of ultrasonic image classification diagnosis of liver and spleen injury based on double‐channel convolutional neural network was proposed. Firstly, the anisotropic diffusion denoising model is used to realize data preprocessing of ultrasonic images of the liver and spleen to improve the image quality of ultrasonic images. Secondly, the external edge of the lesion location was detected to obtain the characteristics of the external edge. Then, the rotation invariant local binary mode feature of the extracted image is taken as the inner texture feature of the image. Finally, the external edge feature and internal texture feature are used as two input channels of the convolutional neural network, respectively, to classify and identify ultrasonic images of liver and spleen injury. The experimental results show that the proposed method diagnoses liver and spleen injury more accurately.

A Robust Invariant Local Feature Matching Method for Changing Scenes

The precise evaluation of camera position and orientation is a momentous procedure of most machine vision tasks, especially visual localization. Aiming at the shortcomings of local features of dealing with changing scenes and the problem of realizing a robust end‐to‐end network that worked from feature detection to matching, an invariant local feature matching method for changing scene image pairs is proposed, which is a network that integrates feature detection, descriptor constitution, and feature matching. In the feature point detection and descriptor construction stage, joint training is carried out based on a neural network. In the feature point extraction and descriptor construction stage, joint training is carried out based on a neural network. To obtain local features with solid robustness to viewpoint and illumination changes, the Vector of Locally Aggregated Descriptors based on Neural Network (NetVLAD) module is introduced to compute the degree of correlation of description vectors from one image to another counterpart. Then, to enhance the relationship between relevant local features of image pairs, the attentional graph neural network (AGNN) is introduced, and the Sinkhorn algorithm is used to match them; finally, the local feature matching results between image pairs are output. The experimental results show that, compared with the existed algorithms, the proposed method enhances the robustness of local features of varying sights, performs better in terms of homography estimation, matching precision, and recall, and when meeting the requirements of the visual localization system to the environment, the end‐to‐end network tasks can be realized.

Local Features Based Deep Learning for Mammographic Image Classification: In Comparison to CNN Models

Abstract Convolutional neural networks (CNN) have achieved tremendous success in tackling various tasks of computer vision problems such as classification of images and recognition of objects. The general trend has been to make networks larger and more complex in order to achieve higher precision. In any case, these advances to improve precision are not really making the systems increasingly effective for size and speed. In this report, an attempt to overcome the aforementioned issues is described. Some well-known scale and rotation invariant local features such as SIFT, SURF, ORB, BRISK, and KAZE were first observed to check which of these having the strong potentiality to replace inclusively the convolutional layers of CNN models. To do this, while retaining the fully connected layers of the corresponding CNN model as the classifier, the convolutional parts of the model were inclusively taken out and assigned the high-level feature maps at the flatten layer with scale and rotation invariant local features. The computational experiments were carried out using the challenging mammographic images provided by MIAS (mammographic image analysis society). It contains 322 grayscale images from 2 different severity, each of which has 1024x1024 pixels. Employing only the fully connected layers of the VGG16 model as the classifier, it is found that SIFT and SURF based features offer much better training and testing accuracies compared to ORB, BRISK, and KAZE in classifying MIAS image datasets. Using SIFT and SURF local features and employing only VGG16 or MobileNetV2 fully connected layers as a classifier, it can be shown that SIFT and SURF based deep learning exhibit state-of-the-art performance on MIAS datasets. SIFT and SURF based deep learning are capable of outperforming the performance of the original VGG16 and MobileNetV2 configuration in terms of accuracy and training time needed on the same datasets.

Real-Time Data Filling and Automatic Retrieval Algorithm of Road Traffic Based on Deep-Learning Method

In order to enhance the real-time and retrieval performance of road traffic data filling, a real-time data filling and automatic retrieval algorithm based on the deep-learning method is proposed. In image detection, the depth representation is extracted according to the detection target area of a general object. The local invariant feature is extracted to describe local attributes in the region, and it is fused with depth representation to complete the real-time data filling of road traffic. According to the results of the database enhancement, the retrieval results of the deep representation level are reordered. In the index stage, unsupervised feature updating is realized by neighborhood information to improve the performance of a feature retrieval. The experimental results show that the proposed method has high recall and precision, a short retrieval time and a low running cost.

A Novel Algorithm for Breast Mass Classification in Digital Mammography Based on Feature Fusion.

Prompt diagnosis of benign and malignant breast masses is essential for early breast cancer screening. Convolutional neural networks (CNNs) can be used to assist in the classification of benign and malignant breast masses. A persistent problem in current mammography mass classification via CNN is the lack of local-invariant features, which cannot effectively respond to geometric image transformations or changes caused by imaging angles. In this study, a novel model that trains both texton representation and deep CNN representation for mass classification tasks is proposed. Rotation-invariant features provided by the maximum response filter bank are incorporated with the CNN-based classification. The fusion after implementing the reduction approach is used to address the deficiencies of CNN in extracting mass features. This model is tested on public datasets, CBIS-DDSM, and a combined dataset, namely, mini-MIAS and INbreast. The fusion after implementing the reduction approach on the CBIS-DDSM dataset outperforms that of the other models in terms of area under the receiver operating curve (0.97), accuracy (94.30%), and specificity (97.19%). Therefore, our proposed method can be integrated with computer-aided diagnosis systems to achieve precise screening of breast masses.