Set Of Local Features Research Articles

A quantitative evaluation of the conceptual consistency of visual words and visual vocabularies

Codebooks are a widely accepted technique to recognise objects by sets of local features. The method has been applied to many classes of objects, even very abstract ones. But although state of the art recognition rates have been reported, the method is still far away from being reliable in any sense that is related to human vision. The literature on this topic emphasises detailed descriptions of statistical estimators over a basic analysis of the data. A deeper understanding of the data is however needed to achieve a further development of the field. In this paper, we therefore present a set of quantitative experiments on codebooks of the popular SIFT descriptors. The results discourage the use of illustrative but overly simplifying descriptions of the visual words approach. It is in particular demonstrated that (1) there are more visually distinct patterns than can be listed in a codebook, (2) one element of a codebook represents a set of many, visually distinct patterns, and (3) there are no single, selective SIFT descriptors to serve as codebook elements. This makes us wonder why the method works after all. We discuss several options.

Robust object detection based on local similar structure statistical matching

We present a robust object detection method to detect generic objects with incompact, complex and changeable shapes without training. First, we build a composite template set, which contains changeful shapes, scales and viewpoints of an interested object class, extract the local structure features from the composite template set and simplify them to construct a non-similar local structure feature set of the object class. Then, we propose a matching method of local similar structure statistical matching (LSSSM) to obtain the similarity image from a test image to the local structure feature set. Finally, we use the method of non-maxima suppression in the similarity image to extract the object position and mark the object in the test image. The experimental results demonstrate that our approach performs effectively on the face and infrared human body detection.

Compounding local invariant features and global deformable geometry for medical image registration.

Using deformable models to register medical images can result in problems of initialization of deformable models and robustness and accuracy of matching of inter-subject anatomical variability. To tackle these problems, a novel model is proposed in this paper by compounding local invariant features and global deformable geometry. This model has four steps. First, a set of highly-repeatable and highly-robust local invariant features, called Key Features Model (KFM), are extracted by an effective matching strategy. Second, local features can be matched more accurately through the KFM for the purpose of initializing a global deformable model. Third, the positional relationship between the KFM and the global deformable model can be used to precisely pinpoint all landmarks after initialization. And fourth, the final pose of the global deformable model is determined by an iterative process with a lower time cost. Through the practical experiments, the paper finds three important conclusions. First, it proves that the KFM can detect the matching feature points well. Second, the precision of landmark locations adjusted by the modeled relationship between KFM and global deformable model is greatly improved. Third, regarding the fitting accuracy and efficiency, by observation from the practical experiments, it is found that the proposed method can improve % of the fitting accuracy and reduce around 50% of the computational time compared with state-of-the-art methods.

Music genre classification based on local feature selection using a self-adaptive harmony search algorithm

This paper proposes an automatic music genre-classification system based on a local feature-selection strategy by using a self-adaptive harmony search (SAHS) algorithm. First, five acoustic characteristics (i.e., intensity, pitch, timbre, tonality, and rhythm) are extracted to generate an original feature set. A feature-selection model using the SAHS algorithm is then employed for each pair of genres, thereby deriving the corresponding local feature set. Finally, each one-against-one support vector machine (SVM) classifier is fed with the corresponding local feature set, and the majority voting method is used to classify each musical recording. Experiments on the GTZAN dataset were conducted, demonstrating that our method is effective. The results show that the local-selection strategies using wrapper and filter approaches ranked first and third in performance among all relevant methods.

Clustering method based on data division and partition

Many classical clustering algorithms do good jobs on their prerequisite but do not scale well when being applied to deal with very large data sets (VLDS). In this work, a novel division and partition clustering method (DP) was proposed to solve the problem. DP cut the source data set into data blocks, and extracted the eigenvector for each data block to form the local feature set. The local feature set was used in the second round of the characteristics polymerization process for the source data to find the global eigenvector. Ultimately according to the global eigenvector, the data set was assigned by criterion of minimum distance. The experimental results show that it is more robust than the conventional clusterings. Characteristics of not sensitive to data dimensions, distribution and number of nature clustering make it have a wide range of applications in clustering VLDS.

Controlled Triangularization Fingerprint Verification Inscribed in a Rhombus Using Fuzzy Feature Matching

This paper proposes a novel method, a Fuzzy Feature Match (FFM) based on a controlled triangle feature set inscribed in a rhombus to match the deformed fingerprints. The fingerprint is represented by the fuzzy feature set. The fuzzy features set similarity is used to analyze the similarity among fingerprints. A fuzzy similarity for two triangles is introduced and then a similarity vector including the triangle-level similarities for all triangles and the rhombus in two fingerprints are constructed. Accordingly, a similarity vector pair is defined to illustrate the similarities between two fingerprints. The FFM method shows the similarity vector pair to a normalized value which quantifies the overall image to image similarity. The algorithm has been evaluated with Noorul Islam (NI) university staff members' fingerprint database. Experimental results confirm that the proposed FFM based on the local triangle feature set inscribed in a rhombus is a reliable and effective algorithm for fingerprint matching with nonlinear distortions.

Exploiting contextual information for prosodic event detection using auto-context

Prosody and prosodic boundaries carry significant information regarding linguistics and paralinguistics and are important aspects of speech. In the field of prosodic event detection, many local acoustic features have been investigated; however, contextual information has not yet been thoroughly exploited. The most difficult aspect of this lies in learning the long-distance contextual dependencies effectively and efficiently. To address this problem, we introduce the use of an algorithm called auto-context. In this algorithm, a classifier is first trained based on a set of local acoustic features, after which the generated probabilities are used along with the local features as contextual information to train new classifiers. By iteratively using updated probabilities as the contextual information, the algorithm can accurately model contextual dependencies and improve classification ability. The advantages of this method include its flexible structure and the ability of capturing contextual relationships. When using the auto-context algorithm based on support vector machine, we can improve the detection accuracy by about 3% and F-score by more than 7% on both two-way and four-way pitch accent detections in combination with the acoustic context. For boundary detection, the accuracy improvement is about 1% and the F-score improvement reaches 12%. The new algorithm outperforms conditional random fields, especially on boundary detection in terms of F-score. It also outperforms an n-gram language model on the task of pitch accent detection.

Fusion of dense spatial features and sparse temporal features for three‐dimensional structure estimation in urban scenes

The authors present a novel approach to improve three-dimensional (3D) structure estimation from an image stream in urban scenes. The authors consider a particular setup, where the camera is installed on a moving vehicle. Applying traditional structure from motion (SfM) technique in this case generates poor estimation of the 3D structure because of several reasons such as texture-less images, small baseline variations and dominant forward camera motion. The authors idea is to introduce the monocular depth cues that exist in a single image, and add time constraints on the estimated 3D structure. The scene is modelled as a set of small planar patches obtained using over-segmentation, and the goal is to estimate the 3D positioning of these planes. The authors propose a fusion scheme that employs Markov random field model to integrate spatial and temporal depth features. Spatial depth is obtained by learning a set of global and local image features. Temporal depth is obtained via sparse optical flow based SfM approach. That allows decreasing the estimation ambiguity by forcing some constraints on camera motion. Finally, the authors apply a fusion scheme to create unique 3D structure estimation.

A two‐dimensional random projected minutiae vicinity decomposition‐based cancellable fingerprint template

ABSTRACTWith the massive deployment of biometric applications, protecting the biometric template has attracted great attention because of the privacy issue. Although many proposals on protecting biometric template have been reported in literature, design a method simultaneously satisfying four criteria, that is performance, non‐invertibility, cancellability, and diversity still remains unsolved. In this paper, we proposed a two‐dimensional random projected minutiae vicinity decomposition (MVD) technique to secure minutiae‐based fingerprint template. Minutiae vicinity is first formed from a set of fingerprint minutiae and further used to generate a set of local features, namely MVD features; then, a random matrix derived from user‐specific token is used to project MVD features for the concealment of the topology of MVD. Comprehensive experiments on fingerprint verification competition datasets are carried out, and the lowest equal error rate obtained in the stolen‐token scenario is 3.07% and 1.02% for fingerprint verification competition 2002 database 1 and database 2, respectively. Besides, detail analyses on the irreversibility, cancellability, template size, and computational cost have been carried out. Copyright © 2013 John Wiley & Sons, Ltd.

Joint learning and weighting of visual vocabulary for bag-of-feature based tissue classification

Automated classification of tissue types of Region of Interest (ROI) in medical images has been an important application in Computer-Aided Diagnosis (CAD). Recently, bag-of-feature methods which treat each ROI as a set of local features have shown their power in this field. Two important issues of bag-of-feature strategy for tissue classification are investigated in this paper: the visual vocabulary learning and weighting, which are always considered independently in traditional methods by neglecting the inner relationship between the visual words and their weights. To overcome this problem, we develop a novel algorithm, Joint-ViVo, which learns the vocabulary and visual word weights jointly. A unified objective function based on large margin is defined for learning of both visual vocabulary and visual word weights, and optimized alternately in the iterative algorithm. We test our algorithm on three tissue classification tasks: classifying breast tissue density in mammograms, classifying lung tissue in High-Resolution Computed Tomography (HRCT) images, and identifying brain tissue type in Magnetic Resonance Imaging (MRI). The results show that Joint-ViVo outperforms the state-of-art methods on tissue classification problems.

EvoDesign: de novo protein design based on structural and evolutionary profiles

Protein design aims to identify new protein sequences of desirable structure and biological function. Most current de novo protein design methods rely on physics-based force fields to search for low free-energy states following Anfinsen’s thermodynamic hypothesis. A major obstacle of such approaches is the inaccuracy of the force field design, which cannot accurately describe the atomic interactions or distinguish correct folds. We developed a new web server, EvoDesign, to design optimal protein sequences of given scaffolds along with multiple sequence and structure-based features to assess the foldability and goodness of the designs. EvoDesign uses an evolution-profile–based Monte Carlo search with the profiles constructed from homologous structure families in the Protein Data Bank. A set of local structure features, including secondary structure, torsion angle and solvation, are predicted by single-sequence neural-network training and used to smooth the sequence motif and accommodate the physicochemical packing. The EvoDesign algorithm has been extensively tested in large-scale protein design experiments, which demonstrate enhanced foldability and structural stability of designed sequences compared with the physics-based designing methods. The EvoDesign server is freely available at http://zhanglab.ccmb.med.umich.edu/EvoDesign.

Multiple scale-specific representations for improved human action recognition

Human action recognition in video is important in many computer vision applications such as automated surveillance. Human actions can be compactly encoded using a sparse set of local spatio-temporal salient features at different scales. The existing bottom-up methods construct a single dictionary of action primitives from the joint features of all scales and hence, a single action representation. This representation cannot fully exploit the complementary characteristics of the motions across different scales. To address this problem, we introduce the concept of learning multiple dictionaries of action primitives at different resolutions and consequently, multiple scale-specific representations for a given video sample. Using a decoupled fusion of multiple representations, we improved the human classification accuracy of realistic benchmark databases by about 5%, compared with the state-of-the art methods.

Object Detection Using SURF and Superpixels

This paper proposes a novel object detection method in which a set of local features inside the superpixels are extracted from the image under analysis acquired by a 3D visual sensor. To increase the segmentation accuracy, the proposed method firstly performs the segmentation of the image, under analysis, using the Simple Linear Iterative Clustering (SLIC) superpixels method. Next the key points inside each superpixel are estimated using the Speed-Up Robust Feature (SURF). These key points are then used to carry out the matching task for every detected keypoints of a scene inside the estimated superpixels. In addition, a probability map is introduced to describe the accuracy of the object detection results. Experimental results show that the proposed approach provides fairly good object detection and confirms the superior performance of proposed scene compared with other recently proposed methods such as the scheme proposed by Mae et al.

Subspace Similarity of Local Feature Set using Canonical Correlation Analysis for View-based Object Recognition

Subspace Similarity of Local Feature Set using Canonical Correlation Analysis for View-based Object Recognition

Spatio-temporal feature-based keyframe detection from video shots using spectral clustering

Keyframe detection is a fundamental component in approaches for large-scale mapping and scene recognition. Assuming that the detection is applied to a set of continuously captured frames, this paper presents a keyframe detector that not only considers the frame content to quantify appearance changes on the sequence, but also the temporal accumulation of evidence. If frames are described as a set of local features, our algorithm proposes a unified framework for comparing local features acquired from consecutive frames by the building of an auxiliary graph-based on the locality of features. Spectral clustering is then employed to obtain tentative graph partitions. Validated partitions will be associated to keyframes. It should be noted that the approach does not need to estimate the motion of the camera, and that the similarity measure defined within this framework can be used for any sort of feature. Experimental results using different types of visual features show the strength of our representation. Moreover, an evaluation methodology has been defined for the quantitative comparison of our keyframe detector against other similar approaches.

Structured learning of local features for human action classification and localization

Human action recognition is a promising yet non-trivial computer vision field with many potential applications. Current advances in bag-of-feature approaches have brought significant insights into recognizing human actions within complex context. It is, however, a common practice in literature to consider action as merely an orderless set of local salient features. This representation has been shown to be oversimplified, which inherently limits traditional approaches from robust deployment in real-life scenarios. In this work, we propose and show that, by taking into account global configuration of local features, we can greatly improve recognition performance. We first introduce a novel feature selection process called Sparse Hierarchical Bayes Filter to select only the most contributive features of each action type based on neighboring structure constraints. We then present the application of structured learning in human action analysis. That is, by representing human action as a complex set of local features, we can incorporate different spatial and temporal feature constraints into the learning tasks of human action classification and localization. In particular, we tackle the problem of action localization in video using structured learning with two alternatives: one is Dynamic Conditional Random Field from probabilistic perspective; the other is Structural Support Vector Machine from max-margin point of view. We evaluate our modular classification-localization framework on various testbeds, in which our proposed framework is proven to be highly effective and robust compared against bag-of-feature methods.

Residual enhanced visual vector as a compact signature for mobile visual search

Many mobile visual search (MVS) systems transmit query data from a mobile device to a remote server and search a database hosted on the server. In this paper, we present a new architecture for searching a large database directly on a mobile device, which can provide numerous benefits for network-independent, low-latency, and privacy-protected image retrieval. A key challenge for on-device retrieval is storing a large database in the limited RAM of a mobile device. To address this challenge, we develop a new compact, discriminative image signature called the Residual Enhanced Visual Vector (REVV) that is optimized for sets of local features which are fast to extract on mobile devices. REVV outperforms existing compact database constructions in the MVS setting and attains similar retrieval accuracy in large-scale retrieval as a Vocabulary Tree that uses 25× more memory. We have utilized REVV to design and construct a mobile augmented reality system for accurate, large-scale landmark recognition. Fast on-device search with REVV enables our system to achieve latencies around 1s per query regardless of external network conditions. The compactness of REVV allows it to also function well as a low-bitrate signature that can be transmitted to or from a remote server for an efficient expansion of the local database search when required.

Patch Distribution Compatible Semisupervised Dimension Reduction for Face and Human Gait Recognition

We propose a new semisupervised learning algorithm, referred to as patch distribution compatible semisupervised dimension reduction, for face and human gait recognition. Each image (a face image or an average human silhouette image) is first represented as a set of local patch features and it is further characterized as the corresponding patch distribution feature, which can be expressed as an image-specific Gaussian mixture model (GMM) adapted from the universal background model. Assuming that the individual components of the image-specific GMMs from all the training images reside on a submanifold, we assign a component-level prediction label matrix to each individual GMM component and introduce a new regularizer based on a set of local submanifold smoothness assumptions in our objective function. We also constrain each component-level prediction label matrix to be consistent with the image-level prediction label matrix , as well as enforce to be close to the given labels for the labeled samples. We further use a linear regression function to provide embeddings for the training data and the unseen test data. Inspired by the recent work flexible manifold embedding, we additionally integrate the regression residue in our objective function to measure the mismatch between and , such that our method can better cope with the data sampled from a nonlinear manifold. Finally, the optimal solutions of the component-level prediction label matrix , the image-level prediction label matrix , the projection matrix , and the bias term b can be simultaneously obtained. Comprehensive experiments on three benchmark face databases CMU PIE, FERET, and AR as well as the USF HumanID gait database clearly demonstrate the effectiveness of our algorithm over other state-of-the-art semisupervised dimension reduction methods.

Automated landmarking and geometric characterization of the carotid siphon

The geometry of the carotid siphon has a large variability between subjects, which has prompted its study as a potential geometric risk factor for the onset of vascular pathologies on and off the internal carotid artery (ICA). In this work, we present a methodology for an objective and extensive geometric characterization of carotid siphon parameterized by a set of anatomical landmarks. We introduce a complete and automated characterization pipeline. Starting from the segmentation of vasculature from angiographic image and its centerline extraction, we first identify ICA by characterizing vessel tree bifurcations and training a support vector machine classifier to detect ICA terminal bifurcation. On ICA centerline curve, we detect anatomical landmarks of carotid siphon by modeling it as a sequence of four bends and selecting their centers and interfaces between them. Bends are detected from the trajectory of the curvature vector expressed in the parallel transport frame of the curve. Finally, using the detected landmarks, we characterize the geometry in two complementary ways. First, with a set of local and global geometric features, known to affect hemodynamics. Second, using large deformation diffeomorphic metric curve mapping (LDDMCM) to quantify pairwise shape similarity. We processed 96 images acquired with 3D rotational angiography. ICA identification had a cross-validation success rate of 99%. Automated landmarking was validated by computing limits of agreement with the reference taken to be the locations of the manually placed landmarks averaged across multiple observers. For all but one landmark, either the bias was not statistically significant or the variability was within 50% of the inter-observer one. The subsequently computed values of geometric features and LDDMCM were commensurate to the ones obtained with manual landmarking. The characterization based on pair-wise LDDMCM proved better in classifying the carotid siphon shape classes than the one based on geometric features. The proposed characterization provides a rich description of geometry and is ready to be applied in the search for geometric risk factors of the carotid siphon.

Estimating size fraction categories of coal particles on conveyor belts using image texture modeling methods

Motivation: Physical properties of coal such as particle size distribution have a large influence on the stability and operational behavior of fluidized bed reactors and metallurgical furnaces. In particular, the presence of large amounts of “fine” particles invariably has a drastic effect on plant performance as a result of impaired gas permeability characteristics of the coal or ore burden. Therefore, monitoring and control of particle size distribution profiles of such aggregate material on reactor feed streams, such as moving conveyor belts, is critical for predictable operation of these processes. Traditionally, the method of sieve analysis using stock or belt cut samples has been widely used in industry. Unfortunately, the reliability and usefulness of belt cut techniques are constrained by frequency of sampling as well as laboratory analysis turnaround times. For real-time monitoring and control purposes, automated sampling and analysis methods are more desirable. Methods: In this study, the problem of estimating the particle size distribution profile of material on a moving conveyor belt is formulated within a texture classification framework, which has its basis in machine vision and incorporates elements from statistical pattern recognition. Using exemplar images of coal particles taken on a process stream, a set of local features that compactly describes the textural properties of each image are expressed in terms of localized nonlinear features called textons. Representation of image information using textons is primarily motivated by insights from neuroscience research on the optimality of linear oriented basis functions as models of perception in early processing of visual information in the cortex regions of the human brain. Using these representations for different textures, nearest neighbor and support vector machine classification models are subsequently used to classify test images. Results: Using a comprehensive evaluation, it is shown that the use of texton representation obtained from decomposing images with linear oriented basis functions can be sufficiently discriminative compared to the use of the widely used second-order statistical features or features from other baseline models. In particular, model performance obtained with appropriately tuned filters suggest the importance of including shape and spatial structure information in an image representation for texture classification of coal particles. Furthermore, using nonlinear support vector machines rather than nearest neighbor classifiers significantly improved classification performance. A texture classification approach to particle size profile estimation has potential applications in the online monitoring of the proportion of “fines” in coal material on moving conveyor belts.