Bag Of Features Research Articles

RGBD Co-saliency Detection via Bagging-Based Clustering

With the additional depth information, RGBD co-saliency detection, which is an emerging and interesting issue in saliency detection, aims to discover the common salient objects in a set of RGBD images. This letter proposes a novel RGBD co-saliency model using bagging-based clustering. First, candidate object regions are generated based on RGBD single saliency maps and region pre-segmentation. Then, in order to make regional clustering more robust to different image sets, the feature bagging method is introduced to randomly generate multiple clustering results and the cluster-level weak co-saliency maps. Finally, a clustering quality (CQ) criterion is devised to adaptively integrate the weak co-saliency maps into the final co-saliency map for each image. Experimental results on a public RGBD co-saliency dataset show that the proposed co-saliency model significantly outperforms the state-of-the-art co-saliency models.

Towards Reversal-Invariant Image Representation

State-of-the-art image classification approaches are mainly based on robust image representation, such as the bag-of-features (BoF) model or the convolutional neural network (CNN) architecture. In real applications, the orientation (left/right) of an image or an object might vary from sample to sample, whereas some handcrafted descriptors (e.g., SIFT) and network operations (e.g., convolution) are not reversal-invariant, leading to the unsatisfied stability of image features extracted from these models. To deal with, a popular solution is to augment the dataset by adding a left-right reversed copy for each image. This strategy improves the recognition accuracy to some extent, but also brings the price of almost doubled time and memory consumptions on both the training and testing stages. In this paper, we present an alternative solution based on designing reversal-invariant representation of local patterns, so that we can obtain the identical representation for an image and its left-right reversed copy. For the BoF model, we design a reversal-invariant version of SIFT descriptor named Max-SIFT, a generalized RIDE algorithm which can be applied to a large family of local descriptors. For the CNN architecture, we present a simple idea of generating reversal-invariant deep features (RI-Deep), and, inspired by which, design reversal-invariant convolution (RI-Conv) layers to increase the CNN capacity without increasing the model complexity. Experiments reveal consistent accuracy gain on various image classification tasks, including scene understanding, fine-grained object recognition, and large-scale visual recognition.

Characterization and assessment of fabric smoothness appearance based on sparse coding

Traditional computer vision methods cannot match human performance well on fabric smoothness classification, as this is a subjective assessment based on sparse, comprehensive and low-cost visual perception. This paper reports a new assessment method of fabric smoothness appearance, including feature designing and wrinkle classification. A multidimensional feature was designed by generalizing vector quantization of dense scale-invariant feature transform (SIFT) descriptors for sparse coding and max pooling. Sparse coding provides clear understanding about the receptive fields of visual neurons and can build a codebook from the low features. A one-against-rest linear support vector machine (SVM) was utilized to classify the nine grades of smoothness and quantized to level 0.1 by space distance. Results showed that the proposed approach achieved remarkable classification accuracy in comparison with Bag-of-Feature (BOF) and Spatial Pyramid Matching (SPM) algorithms.

Neural Bag-of-Features learning

In this paper, a neural learning architecture for the well-known Bag-of-Features (BoF) model, called Neural Bag-of-Features, is proposed. The Neural BoF model is formulated in two neural layers: a Radial Basis Function (RBF) layer and an accumulation layer. The ability of the Neural BoF model to improve the classification performance is demonstrated using four datasets, including a large-scale dataset, and five different feature types. The gains are two-fold: the classification accuracy increases and, at the same time, smaller networks can be used, reducing the required training and testing time. Furthermore, the Neural BoF natively supports training and classifying from feature streams. This allows the proposed method to efficiently scale to large datasets. The streaming process can also be used to introduce noise and reduce the over-fitting of the network. Finally, the Neural BoF provides a framework that can model and extend the dictionary learning methodology.

Information Clustering Using Manifold-Based Optimization of the Bag-of-Features Representation.

In this paper, a manifold-based dictionary learning method for the bag-of-features (BoF) representation optimized toward information clustering is proposed. First, the spectral representation, which unwraps the manifolds of the data and provides better clustering solutions, is formed. Then, a new dictionary is learned in order to make the histogram space, i.e., the space where the BoF historgrams exist, as similar as possible to the spectral space. The ability of the proposed method to improve the clustering solutions is demonstrated using a wide range of datasets: two image datasets, the 15-scene dataset and the Corel image dataset, one video dataset, the KTH dataset, and one text dataset, the RT-2k dataset. The proposed method improves both the internal and the external clustering criteria for two different clustering algorithms: 1) the -means and 2) the spectral clustering. Also, the optimized histogram space can be used to directly assign a new object to its cluster, instead of using the spectral space (which requires reapplying the spectral clustering algorithm or using incremental spectral clustering techniques). Finally, the learned representation is also evaluated using an information retrieval setup and it is demonstrated that improves the retrieval precision over the baseline BoF representation.

Structuralizing biomedical abstracts with discriminative linguistic features

ObjectiveNearly 75% of the abstracts in MEDLINE papers present in an unstructured format. This study aims to automate the reformatting of unstructured abstracts into the Introduction, Methods, Results, and Discussion (IMRAD) format. The quality of this reformatting relies on the features used in sentence classification. Therefore, we explored the most effective linguistic features in MEDLINE papers. MethodsWe constructed a feature set consisting of bag of words, linguistic features, grammatical features, and structural features. In order to evaluate the effectiveness, which is the capability of the sentence classification with the features, three datasets from PubMed Central Open Access Subset were selected and constructed: (1) structured abstract (SA) for training, (2) unstructured RCT abstract (UA-1) and (3) unstructured general abstract (UA-2). F-score and accuracy were used to measure the effectiveness on IMRAD section level and the overall classification. ResultsAdding linguistic features improves the classification of the abstract sentence from 1.2% to 35.8% in terms of accuracy in three abstract datasets. The highest accuracies achieved were 91.7% in SA, 86.3% in UA-1, and 77.9% in UA-2. Linguistic features (dimensions=15) had fewer dimensions than bag-of-words (dimensions= 1541). All representative linguistic features (n-gram and verb phrase, and noun phrase) for each section are identified in our system (available at http://abstract.bike.re.kr). ConclusionLinguistic features can be used to effectively classify sentence with low computation burden in MEDLINE abstract.

Real-Time Vehicle Make and Model Recognition Based on a Bag of SURF Features

In this paper, we propose and evaluate unexplored approaches for real-time automated vehicle make and model recognition (VMMR) based on a bag of speeded-up robust features (BoSURF) and demonstrate the suitability of these approaches for vehicle identification systems. The proposed approaches use SURF features of vehicles' front- or rear-facing images and retain the dominant characteristic features ( codewords ) in a dictionary . Two schemes of dictionary building are evaluated: “ single dictionary ” and “ modular dictionary. ” Based on the optimized dictionaries, the SURF features of vehicles' front- or rear-face images are embedded into BoSURF histograms, which are used to train multiclass support vector machines (SVMs) for classification. Two real-time VMMR classification schemes are proposed and evaluated: a single multiclass SVM and an ensemble of multiclass SVM based on attribute bagging. The processing speed and accuracy of the VMMR system are affected greatly by the size of the dictionary. The tradeoff between speed and accuracy is studied to determine optimal dictionary sizes for the VMMR problem. The effectiveness of our approaches is demonstrated through cross-validation tests on a recent publicly accessible VMMR data set. The experimental results prove the superiority of our work over the state of the art, in terms of both processing speed and accuracy, making it highly applicable to real-time VMMR systems.

Object classification using a local texture descriptor and a support vector machine

Objects classification or object detection is one of the most challenging tasks in computer vision. Digital images taken of real-life scenes capture objects at different positions, rotations and scales. Furthermore, variations in lighting, shape, color and texture within the same class make object classification a huge obstacle for computer vision algorithms. The most robust methodologies related to variations in lighting, rotation, color and scale, are based on “key points” localization, followed by applying a local descriptor to each surrounding region. Researchers have used various methods for detecting key points and have applied various local descriptors. Of these, the Scale-Invariant Feature Transform (SIFT), Speeded Up Robust Features (SURF) and Center-Symmetric Local Binary Pattern (CS-LBP) methods have obtained good performance and are associated with clustering algorithms or histogram representation based on independent features (Bag of Features (BoF)). In the BoF approach, the visual codebook extracted around the “key points” regions can effectively describe objects by their appearance based on local texture analysis. Recently, we proposed two new texture descriptors for object detection based on the Local Mapped Pattern (LMP) approach. The Mean-Local Mapped Pattern (M-LMP) and the Center Symmetric Local Mapped Pattern (CS-LMP) exhibit better performance than SIFT and CS-LBP, but prior results have shown that the size of descriptors could be reduced without loss of sensitivity. In this paper, we investigated reducing the size of the M-LMP descriptor and then evaluating its performance for object classification by a Support Vector Machine (SVM) classifier. In our experiments, we implemented an object recognition system based on the M-LMP reduced descriptor, and compared our results against the SIFT, Local Intensity Order Pattern (LIOP) and CS-LMP descriptors. The object classification results were analyzed using a BoF model and a SVM classifier, with the result that performance using the reduced descriptor is better than the other three well-known methods tested and also requires less processing time.

Multitask Learning of Compact Semantic Codebooks for Context-Aware Scene Modeling.

In the past few decades, we have witnessed the success of bag-of-features (BoF) models in scene classification, object detection, and image segmentation. Whereas it is also well acknowledged that the limitation of BoF-based methods lies in the low-level feature encoding and coarse feature pooling. This paper proposes a novel scene classification method, which leverages several semantic codebooks learned in a multitask fashion for robust feature encoding, and designs a context-aware image representation for efficient feature pooling. Apart from conventional universal codebook learning approaches, the proposed method encodes each class of local features with a unique semantic codebook, which captures the distinct distribution of different semantic classes more effectively. Instead of learning each semantic codebook separately, we learn a compact global codebook, of which each semantic codebook is a sparse subset, with a two-stage iterative multitask learning algorithm. While minimizing the clustering divergence, the semantic codeword assignment is solved by submodular optimization simultaneously. Built upon the global and semantic codebooks, a context-aware image representation is further developed to encode both global and semantic features in image representation via contextual quantization, semantic response computation, and semantic pooling. Extensive experiments have been conducted to validate the effectiveness of the proposed method on various public benchmarks with several popular local features.

A deep bag-of-features model for the classification of melanomas in dermoscopy images.

Deep learning and unsupervised feature learning have received great attention in past years for their ability to transform input data into high level representations using machine learning techniques. Such interest has been growing steadily in the field of medical image diagnosis, particularly in melanoma classification. In this paper, a novel application of deep learning (stacked sparse auto-encoders) is presented for skin lesion classification task. The stacked sparse auto-encoder discovers latent information features in input images (pixel intensities). These high-level features are subsequently fed into a classifier for classifying dermoscopy images. In addition, we proposed a new deep neural network architecture based on bag-of-features (BoF) model, which learns high-level image representation and maps images into BoF space. Then, we examine how using this deep representation of BoF, compared with pixel intensities of images, can improve the classification accuracy. The proposed method is evaluated on a test set of 244 skin images. To test the performance of the proposed method, the area under the receiver operating characteristics curve (AUC) is utilized. The proposed method is found to achieve 95% accuracy.

Appearance based background subtraction for PTZ cameras

Traditional background subtraction algorithms assume the camera is static and are based on simple per-pixel models of scene appearance. This leads to false detections when the camera moves. While this can sometimes be addressed by online image registration, this approach is prone to dramatic failures and long-term drift. We present a novel background subtraction algorithm designed for pan-tilt-zoom cameras that overcomes this challenge without the need for explicit image registration. The proposed algorithm automatically trains a discriminative background model, which is global in the sense that it is the same regardless of image location. Our approach first extracts multiple features from across the image and uses principal component analysis for dimensionality reduction. The extracted features are then grouped to form a Bag of Features. A global background model is then learned from the bagged features using Support Vector Machine. The proposed approach is fast and accurate. Having a single global model makes it computationally inexpensive in comparison to traditional pixel-wise models. It outperforms several state-of-the-art algorithms on the CDnet 2014 pan-tilt-zoom and baseline categories and Hopkins155 dataset. In particular, it achieves an F-Measure of 75.41% on the CDnet dataset PTZ category, significantly better than the previously reported best score of 62.07%. These results show that by removing the coupling between detection model and spatial location, we significantly increase the robustness to camera motion.

Simple Techniques Make Sense: Feature Pooling and Normalization for Image Classification

Image classification is a fundamental task in computer vision, implying a wide range of challenging problems, such as object recognition, scene understanding, and image tagging. One of the most popular approaches to image classification, the bag-of-features (BoF) model, represents an image with a long feature vector and adopts machine learning algorithms for training and testing. Owing to its simplicity and scalability, the BoF model is widely used in both academic research studies and industrial applications. This paper discusses the feature summarization stage, including pooling and normalization, in the BoF model. We show that these two modules, although devalued sometimes, have important impacts on image classification performance. We present two algorithms, i.e., generalized regular spatial pooling for constructing a better group of spatial bins and hierarchical feature normalization for assigning proper weights for regional feature normalization. Both algorithms are independent of the descriptor extraction and feature encoding stages, and therefore, they could be freely transplanted onto many other classification frameworks based on local feature statistics. We further provide insightful discussions for the nature of designing efficient image classification models. Experiments verify that the proposed algorithm achieves state-of-the-art results on a wide range of image classification data sets.

An image classification method that considers privacy-preservation

Using images that are dispersed in a network can improve image classification performance; however, it is easy to leak the image holders’ privacy. In this paper, we present a novel method that considers both image classification performance and the image holders’ privacy-preservation. Our method contains four stages; the feature extraction stage follows the Bag-of-Features (BoF) framework, but it introduces significantly novel stages in the training, classification and ensemble stages. In the training stage, we split the image classification into many one-vs.-one classification problems; a set of binary classifiers as weak learners are learned on a local machine, to classify each image in an autonomous system. In the classification stage, a local machine sends features of unlabeled images to each image autonomous system, and then, each image autonomous system and local machine finish a prediction task under their own trained model. Afterward, the image autonomous systems send their prediction labels back to the local system. In the ensemble stage, we gather all of the prediction labels from the image autonomous systems and design an ensemble system to ensemble them into final classification results. This ensemble system must consider both the efficiency of the different trained models in each image autonomous system and the efficiency of the trained model on each class in the one-vs.-one classification problems. During the whole process, only the features of unlabeled images and prediction labels are transmitted in the network, which largely reduces the risk of privacy leakage. We conducted experiments on the network environment, to prove that our method has preferable performance. Our first experiment demonstrates that the performance of our method achieves state-of-the-art performance on four public dataset benchmarks, which shows that our method has competitive performance. The second experiment shows that our method achieves state-of-the-art performance with even fewer training samples, which means that there is less communication overhead and better privacy-preservation.

“Climate Change” Frames Detection and Categorization Based on Generalized Concepts

The subliminal impact of framing of social, political and environmental issues such as climate change has been studied for a long time in political science and communications research. Media framing offers “interpretative package” for average citizens on how to make sense of climate change and its consequences to their livelihoods, how to deal with its negative impacts, and which mitigation or adaptation policies to support. A line of related work has used bag of words and word-level features to detect frames automatically in text. Such works face limitations since standard keyword based features may not generalize well to accommodate surface variations in text when different keywords are used for similar concepts. In this paper, we develop a new type of textual features that generalize (subject,verb,object) triplets extracted from text, by clustering them into high-level concepts. We utilize these concepts as features to detect frames in text. Our corpus comprises more than 45,000 climate change related sentences. Expert coders annotated those sentences as Frame/Non-Frame and framed sentences were mapped into one of four general frame categories: solution, problem threat, cause, and motivation. Compared to uni-gram and bi-gram based models, classification and clustering using our generalized concepts yielded better discriminating features and a higher accuracy classifier with a 12% boost (i.e. from 74% to 83% in f-measure) and 0.91 clustering purity for Frame/Non-Frame detection.

Statistical feature bag based background subtraction for local change detection

This article proposes a novel background subtraction (BGS) technique to detect local changes corresponding to the movement of the objects in video scenes. Here we propose an efficient combination of six local features; three existing and three newly proposed. For background modeling and subtraction here a statistical parametric biunique model is proposed. In the proposed BGS scheme, during the background training phase, the multi-valued features corresponding to background pixels are collected. A few simple statistical parameters are used to characterize each feature. For background subtraction, the multi-valued features computed at each pixel location are compared with those of the computed parameters corresponding to that feature. For each pixel location, different labels (either object or background) are obtained due to different features. For assigning a final label to the pixel in the target frame a majority voting based label fusion technique is used. The proposed technique is successfully tested over several video sequences and found to be providing better results compared to various existing state-of-the-art techniques with three performance evaluation measures.

Perspectives on combining Nonlinear Laser Scanning Microscopy and Bag-of-Features data classification strategies for automated disease diagnostics

Nonlinear Laser Scanning Microscopy (NLSM) techniques have been demonstrated in the past two decades as powerful imaging tools for disease diagnostics (DD). Currently, in most DD related experiments the interpretation of NLSM data sets is performed by trained specialists. Such approaches are both time consuming and prone to errors due to inter- and intra-observer discrepancies. The Bag-of-Features (BoF) paradigm has demonstrated its potential usefulness with respect to automated data classification in the frame of multiple experiments, but its intersections with the field of NLSM are at this moment scarce, to say the least. In this paper we review recent progress on DD using NLSM, and discuss necessary steps and potential future perspectives for merging NLSM and BoF to achieve complex frameworks for automated DD with high sensitivity and specificity.

One-Shot Learning of Human Activity With an MAP Adapted GMM and Simplex-HMM.

This paper presents a novel activity class representation using a single sequence for training. The contribution of this representation lays on the ability to train an one-shot learning recognition system, useful in new scenarios where capturing and labeling sequences is expensive or impractical. The method uses a universal background model of local descriptors obtained from source databases available on-line and adapts it to a new sequence in the target scenario through a maximum a posteriori adaptation. Each activity sample is encoded in a sequence of normalized bag of features and modeled by a new hidden Markov model formulation, where the expectation-maximization algorithm for training is modified to deal with observations consisting in vectors in a unit simplex. Extensive experiments in recognition have been performed using one-shot learning over the public datasets Weizmann, KTH, and IXMAS. These experiments demonstrate the discriminative properties of the representation and the validity of application in recognition systems, achieving state-of-the-art results.

Image retrieval by addition of spatial information based on histograms of triangular regions

The compositional and content attributes of images carry information that enhances the performance of image retrieval. Standard images are constructed by following the rule of thirds that divides an image into nine equal parts by placing objects or regions of interest at the intersecting lines of the grid. An image represents regions and objects that are in a spatial semantic relationship with respect to each other. While the Bag of Features (BoF) representation is commonly used for image retrieval, it lacks spatial information. In this paper, we present two novel image representation methods based on the histograms of triangles, which add spatial information to the inverted index of BoF representation. Histograms of triangles are computed at two levels, by dividing an image into two and four triangles that are evaluated separately. Extensive experiments and comparisons conducted on two datasets demonstrate that the proposed image representations enhance the performance of image retrieval.

Enhanced bag of words using multilevel k-means for human activity recognition

This paper aims to enhance the bag of features in order to improve the accuracy of human activity recognition. In this paper, human activity recognition process consists of four stages: local space time features detection, feature description, bag of features representation, and SVMs classification. The k-means step in the bag of features is enhanced by applying three levels of clustering: clustering per video, clustering per action class, and clustering for the final code book. The experimental results show that the proposed method of enhancement reduces the time and memory requirements, and enables the use of all training data in the k-means clustering algorithm. The evaluation of accuracy of action classification on two popular datasets (KTH and Weizmann) has been performed. In addition, the proposed method improves the human activity recognition accuracy by 5.57% on the KTH dataset using the same detector, descriptor, and classifier.

Improved Bag of Feature for Automatic Polyp Detection in Wireless Capsule Endoscopy Images

Wireless capsule endoscopy (WCE) needs computerized method to reduce the review time for its large image data. In this paper, we propose an improved bag of feature (BoF) method to assist classification of polyps in WCE images. Instead of utilizing a single scale-invariant feature transform (SIFT) feature in the traditional BoF method, we extract different textural features from the neighborhoods of the key points and integrate them together as synthetic descriptors to carry out classification tasks. Specifically, we study influence of the number of visual words, the patch size and different classification methods in terms of classification performance. Comprehensive experimental results reveal that the best classification performance is obtained with the integrated feature strategy using the SIFT and the complete local binary pattern (CLBP) feature, the visual words with a length of 120, the patch size of 8*8, and the support vector machine (SVM). The achieved classification accuracy reaches 93.2%, confirming that the proposed scheme is promising for classification of polyps in WCE images.