Convolutional Neural Network Descriptors Research Articles

Static hand gesture recognition in sign language based on convolutional neural network with feature extraction method using ORB descriptor and Gabor filter

In this paper, a new structure of deep learning neural network is introduced to identify the static hand gesture in the sign language. The proposed structure includes the convolutional neural network (CNN) and the classical non-intelligent feature extraction method. In the proposed structure, the hand gesture image, after preprocessing and removing its background, passes through three different streams of feature extraction, to well extract of effective features and determine the hand gesture class. These three streams, that independently extract their own specific features, consist of three widely used methods in the hand gesture classification named CNN, Gabor filter and ORB feature descriptor. Then these features are merged and formed the final feature vector. By combining these efficient methods, in addition to achieving a very high accuracy in hand gestures classifying, the proposed structure becomes more resistant to uncertainties such as rotation and ambiguity in the hand gestures. Another prominent feature of the proposed structure is its comprehensiveness on different image databases, compared to the similar methods. The transfer learning technique demonstrates that the proposed structure has the ability to be used as a pre-trained structure for any type of image database. Finally, the proposed structure is applied to the three different databases of Massey, ASL Alphabet and ASL, which have 2520, 87,000 and 23,400 of hand gesture images, respectively. The results show the mean accuracy of the proposed structure for the Massey test set of 758 images, ASL with 7020 test images, and ASL Alphabet with 26,100 test images, at 99.92%, 99.8%, and 99.80% respectively.

A Novel Hybrid Approach Based on Deep CNN to Detect Glaucoma Using Fundus Imaging

Glaucoma is one of the eye diseases stimulated by the fluid pressure that increases in the eyes, damaging the optic nerves and causing partial or complete vision loss. As Glaucoma appears in later stages and it is a slow disease, detailed screening and detection of the retinal images is required to avoid vision forfeiture. This study aims to detect glaucoma at early stages with the help of deep learning-based feature extraction. Retinal fundus images are utilized for the training and testing of our proposed model. In the first step, images are pre-processed, before the region of interest (ROI) is extracted employing segmentation. Then, features of the optic disc (OD) are extracted from the images containing optic cup (OC) utilizing the hybrid features descriptors, i.e., convolutional neural network (CNN), local binary patterns (LBP), histogram of oriented gradients (HOG), and speeded up robust features (SURF). Moreover, low-level features are extracted using HOG, whereas texture features are extracted using the LBP and SURF descriptors. Furthermore, high-level features are computed using CNN. Additionally, we have employed a feature selection and ranking technique, i.e., the MR-MR method, to select the most representative features. In the end, multi-class classifiers, i.e., support vector machine (SVM), random forest (RF), and K-nearest neighbor (KNN), are employed for the classification of fundus images as healthy or diseased. To assess the performance of the proposed system, various experiments have been performed using combinations of the aforementioned algorithms that show the proposed model based on the RF algorithm with HOG, CNN, LBP, and SURF feature descriptors, providing ≤99% accuracy on benchmark datasets and 98.8% on k-fold cross-validation for the early detection of glaucoma.

Can We Detect Face Morphing to Prevent Identity Theft?

A relatively new type of identity theft uses morphed facial images in identification documents in which images of two individuals are digitally blended to create an image that maintains a likeness to each of the original identities. We examined people’s ability to detect facial morphing. We collected 3,500 passport-format facial images. This dataset consists of a diverse set of people across gender, age, and race. Convolutional neural network descriptors are used to extract a low-dimensional, perceptually meaningful, representation of each face. For each of 54 faces, these representations are used to find the most similar face in the dataset. A mid-way morph is generated between each pair of different individuals; another mid-way morph is generated between two different photos of the same individual. The morphs are manually edited to remove obvious artifacts. In Experiment 1a (all experiments, N=100), on each trial, participants viewed two images—an original image alongside a morph (from the same or different individual)—and indicated if they are of the same individual or not. Participants struggled to perform this task accurately and were biased to respond “same” (d’=0.68; B=1.81). In Experiment 1b we focused participants’ attention on the eye/nose/mouth regions and provided feedback on each trial. This did not improve sensitivity but led to a reduced bias (d’=0.58; B=1.09). In Experiment 2a, participants saw a single image—a morph or an original—and indicated if it was a morphed face or not. Participants performed only slightly above chance (d’=0.21; B=0.98). In Experiment 2b, when participants were informed of morphing artifacts to look out for and received feedback, performance improved slightly (d’=0.53; B=0.92). Preliminary results suggest that computational methods for face recognition may outperform humans but remain imperfect. Combined, these results suggest that face morphing might be a worryingly effective technique for committing identity theft.

Region-Level Visual Consistency Verification for Large-Scale Partial-Duplicate Image Search

Most recent large-scale image search approaches build on a bag-of-visual-words model, in which local features are quantized and then efficiently matched between images. However, the limited discriminability of local features and the BOW quantization errors cause a lot of mismatches between images, which limit search accuracy. To improve the accuracy, geometric verification is popularly adopted to identify geometrically consistent local matches for image search, but it is hard to directly use these matches to distinguish partial-duplicate images from non-partial-duplicate images. To address this issue, instead of simply identifying geometrically consistent matches, we propose a region-level visual consistency verification scheme to confirm whether there are visually consistent region (VCR) pairs between images for partial-duplicate search. Specifically, after the local feature matching, the potential VCRs are constructed via mapping the regions segmented from candidate images to a query image by utilizing the properties of the matched local features. Then, the compact gradient descriptor and convolutional neural network descriptor are extracted and matched between the potential VCRs to verify their visual consistency to determine whether they are VCRs. Moreover, two fast pruning algorithms are proposed to further improve efficiency. Extensive experiments demonstrate the proposed approach achieves higher accuracy than the state of the art and provide comparable efficiency for large-scale partial-duplicate search tasks.

Texture synthesis quality assessment using perceptual texture similarity

Texture synthesis plays an important role in computer game and movie industries. Although it has been widely studied, the assessment of the quality of the synthesised textures has received little attention. Inspired by the research progress in perceptual texture similarity estimation, we propose a Texture Synthesis Quality Assessment (TSQA) approach. To our knowledge, this is the first attempt to exploit perceptual texture similarity for the TSQA task. In particular, we introduce two perceptual similarity principles for synthesis quality assessment. Correspondingly, we train two Random Forest (RF) regressors. Given a pair of sample and synthesised textures, the two regressors can be used to predict the global and local quality scores of the synthesised texture respectively. An overall score is generated from the two scores. Our results show that the deep Bag-of-Words (BoW) descriptors, extracted by a pre-trained Convolutional Neural Network (CNN), perform better than, or comparably to, the other nine types of hand-crafted or CNN descriptors and an image quality assessment measure, together with the proposed TSQA approach.

Employing fusion of learned and handcrafted features for unconstrained ear recognition

The authors present an unconstrained ear recognition framework that outperforms state-of-the-art systems in different publicly available image databases. To this end, they developed convolutional neural network (CNN)-based solutions for ear normalisation and description, they used well-known handcrafted descriptors, and they fused learned and handcrafted features to improve recognition. They designed a two-stage landmark detector that successfully worked under untrained scenarios. They used the results generated to perform a geometric image normalisation that boosted the performance of all evaluated descriptors. The proposed CNN descriptor outperformed other CNN-based works in the literature, especially in more challenging scenarios. The fusion of learned and handcrafted matchers appears to be complementary and achieved the best performance in all experiments. The obtained results outperformed all other reported results for the Unconstrained Ear Recognition Challenge, which contains the most difficult database nowadays.

NetVLAD: CNN Architecture for Weakly Supervised Place Recognition.

We tackle the problem of large scale visual place recognition, where the task is to quickly and accurately recognize the location of a given query photograph. We present the following four principal contributions. First, we develop a convolutional neural network (CNN) architecture that is trainable in an end-to-end manner directly for the place recognition task. The main component of this architecture, NetVLAD, is a new generalized VLAD layer, inspired by the "Vector of Locally Aggregated Descriptors" image representation commonly used in image retrieval. The layer is readily pluggable into any CNN architecture and amenable to training via backpropagation. Second, we create a new weakly supervised ranking loss, which enables end-to-end learning of the architecture's parameters from images depicting the same places over time downloaded from Google Street View Time Machine. Third, we develop an efficient training procedure which can be applied on very large-scale weakly labelled tasks. Finally, we show that the proposed architecture and training procedure significantly outperform non-learnt image representations and off-the-shelf CNN descriptors on challenging place recognition and image retrieval benchmarks.

Voronoi-Based Compact Image Descriptors: Efficient Region-of-Interest Retrieval With VLAD and Deep-Learning-Based Descriptors

We investigate the problem of image retrieval based on visual queries when the latter comprise arbitrary regions-of-interest (ROI) rather than entire images. Our proposal is a compact image descriptor that combines the state-of-the-art in content-based descriptor extraction with a multilevel, Voronoi-based spatial partitioning of each dataset image. The proposed multilevel Voronoi-based encoding uses a spatial hierarchical K-means over interest-point locations, and computes a content-based descriptor over each cell. In order to reduce the matching complexity with minimal or no sacrifice in retrieval performance: 1) we utilize the tree structure of the spatial hierarchical K-means to perform top-to-bottom pruning for local similarity maxima; 2) we propose a new image similarity score that combines relevant information from all partition levels into a single measure for similarity; 3) we combine our proposal with a novel and efficient approach for optimal bit allocation within quantized descriptor representations. By deriving both a Voronoi-based VLAD descriptor (called Fast-VVLAD) and a Voronoi-based deep convolutional neural network (CNN) descriptor (called Fast-VDCNN), we demonstrate that our Voronoi-based framework is agnostic to the descriptor basis, and can easily be slotted into existing frameworks. Via a range of ROI queries in two standard datasets, it is shown that the Voronoi-based descriptors achieve comparable or higher mean average precision against conventional grid-based spatial search, while offering more than twofold reduction in complexity. Finally, beyond ROI queries, we show that Voronoi partitioning improves the geometric invariance of compact CNN descriptors, thereby resulting in competitive performance to the current state-of-the-art on whole image retrieval.

INVARIANT DESCRIPTOR LEARNING USING A SIAMESE CONVOLUTIONAL NEURAL NETWORK

In this paper we describe learning of a descriptor based on the Siamese Convolutional Neural Network (CNN) architecture and evaluate our results on a standard patch comparison dataset. The descriptor learning architecture is composed of an input module, a Siamese CNN descriptor module and a cost computation module that is based on the L2 Norm. The cost function we use pulls the descriptors of matching patches close to each other in feature space while pushing the descriptors for non-matching pairs away from each other. Compared to related work, we optimize the training parameters by combining a moving average strategy for gradients and Nesterov's Accelerated Gradient. Experiments show that our learned descriptor reaches a good performance and achieves state-of-art results in terms of the false positive rate at a 95 % recall rate on standard benchmark datasets.

INVARIANT DESCRIPTOR LEARNING USING A SIAMESE CONVOLUTIONAL NEURAL NETWORK

Abstract. In this paper we describe learning of a descriptor based on the Siamese Convolutional Neural Network (CNN) architecture and evaluate our results on a standard patch comparison dataset. The descriptor learning architecture is composed of an input module, a Siamese CNN descriptor module and a cost computation module that is based on the L2 Norm. The cost function we use pulls the descriptors of matching patches close to each other in feature space while pushing the descriptors for non-matching pairs away from each other. Compared to related work, we optimize the training parameters by combining a moving average strategy for gradients and Nesterov's Accelerated Gradient. Experiments show that our learned descriptor reaches a good performance and achieves state-of-art results in terms of the false positive rate at a 95 % recall rate on standard benchmark datasets.