Class-specific Codebook Research Articles

Agglomerative Clustering and Residual-VLAD Encoding for Human Action Recognition

Human action recognition has gathered significant attention in recent years due to its high demand in various application domains. In this work, we propose a novel codebook generation and hybrid encoding scheme for classification of action videos. The proposed scheme develops a discriminative codebook and a hybrid feature vector by encoding the features extracted from CNNs (convolutional neural networks). We explore different CNN architectures for extracting spatio-temporal features. We employ an agglomerative clustering approach for codebook generation, which intends to combine the advantages of global and class-specific codebooks. We propose a Residual Vector of Locally Aggregated Descriptors (R-VLAD) and fuse it with locality-based coding to form a hybrid feature vector. It provides a compact representation along with high order statistics. We evaluated our work on two publicly available standard benchmark datasets HMDB-51 and UCF-101. The proposed method achieves 72.6% and 96.2% on HMDB51 and UCF101, respectively. We conclude that the proposed scheme is able to boost recognition accuracy for human action recognition.

Improved Class-Specific Codebook with Two-Step Classification for Scene-Level Classification of High Resolution Remote Sensing Images

With the rapid advances in sensors of remote sensing satellites, a large number of highresolution images (HRIs) can be accessed every day. Land use classification using high-resolution images has become increasingly important as it can help to overcome the problems of haphazard, deteriorating environmental quality, loss of prime agricultural lands, and destruction of important wetlands, and so on. Recently, local feature with bag-of-words (BOW) representation has been successfully applied to land-use scene classification with HRIs. However, the BOW representation ignores information from scene labels, which is critical for scene-level land-use classification. Several algorithms have incorporated information from scene labels into BOW by calculating a class-specific codebook from the universal codebook and coding a testing image with a number of histograms. Those methods for mapping the BOW feature to some inaccurate class-specific codebooks may increase the classification error. To effectively solve this problem, we propose an improved class-specific codebook using kernel collaborative representation based classification (KCRC) combined with SPM approach and SVM classifier to classify the testing image in two steps. This model is robust for categories with similar backgrounds. On the standard Land use and Land Cover image dataset, the improved class-specific codebook achieves an average classification accuracy of 93% and demonstrates superiority over other state-of-the-art scene-level classification methods.

Combining background information and a top-down model for computing salient objects

Predicting the salient object region in real scenes has progressed significantly in recent years. In this work, we propose a novel method for computing salient object regions by combining background information and a top-down visual saliency model, which is well-suited for locating category-specific salient objects in cluttered real scenes. First, we used a robust background measure to acquire clean saliency maps by optimizing background information. Second, we learned a top-down saliency object model by combining a class-specific codebook and conditional random fields (CRFs) during the training phase. Furthermore, our model used the locality-constrained linear codes as latent CRF variables. Finally, we computed salient object regions by combining the robust background measure and top-down model. Experimental results on the Graz-02 and PASCAL VOC2007 datasets show that our method creates much better saliency maps than current state-of-the-art methods.

Computing object-based saliency via locality-constrained linear coding and conditional random fields

Predicting object location using a top-down saliency model has grown increasingly popular in recent years. In this work, we combine locality-constrained linear coding (LLC) with a conditional random field (CRF), and construct a top-down saliency model to generate a specific object-based saliency map. During the training phase, we use the LLC codes as the latent variables of the CRF model, and meanwhile learn a class-specific codebook by CRF modulation. In the testing phase, we use this top-down model to distinguish specific objects from a cluttered background. Finally, we evaluate the experimental results on the MSRA-B, Garz-02, Weizmann Horse, and Plane datasets by applying the developed object-based saliency model. The performance shows that our approach can not only improve the precision but also dramatically reduce the computational complexity.

Fine-Grained Image Classification via Low-Rank Sparse Coding With General and Class-Specific Codebooks.

This paper tries to separate fine-grained images by jointly learning the encoding parameters and codebooks through low-rank sparse coding (LRSC) with general and class-specific codebook generation. Instead of treating each local feature independently, we encode the local features within a spatial region jointly by LRSC. This ensures that the spatially nearby local features with similar visual characters are encoded by correlated parameters. In this way, we can make the encoded parameters more consistent for fine-grained image representation. Besides, we also learn a general codebook and a number of class-specific codebooks in combination with the encoding scheme. Since images of fine-grained classes are visually similar, the difference is relatively small between the general codebook and each class-specific codebook. We impose sparsity constraints to model this relationship. Moreover, the incoherences with different codebooks and class-specific codebooks are jointly considered. We evaluate the proposed method on several public image data sets. The experimental results show that by learning general and class-specific codebooks with the joint encoding of local features, we are able to model the differences among different fine-grained classes than many other fine-grained image classification methods.

Towards Effective Image Classification Using Class-Specific Codebooks and Distinctive Local Features

Local image features, which are robust to scale, view, and orientation changes in images, play a key factor in developing effective visual classification systems. However, there are two main limitations to exploit these features in image classification problems: 1) a large number of key-points are located during the feature detection process, and 2) most of the key-points arise in background regions, which do not contribute to the classification process. In order to decrease the inverse effects of these limitations , we propose a new codebook generation approach through employing a new clustering method that generates class-specific codebooks along with a novel feature selection method in the bag-of-words model. We evaluate the performance of different classification techniques including Naive Bayesian, k-NN, and SVM on distinctive features. Experiments conducted on PASCAL Visual Object Classification collections have shown that the class-specific codebooks along with distinctive image features can significantly improve the classification performances.

Object Detection Via Structural Feature Selection and Shape Model

In this paper, we propose an approach for object detection via structural feature selection and part-based shape model. It automatically learns a shape model from cluttered training images without need to explicitly use bounding boxes on objects. Our approach first builds a class-specific codebook of local contour features, and then generates structural feature descriptors by combining context shape information. These descriptors are robust to both within-class variations and scale changes. Through exploring pairwise image matching using fast earth mover's distance, feature weights can be iteratively updated. Those discriminative foreground features are assigned high weights and then selected to build a part-based shape model. Finally, object detection is performed by matching each testing image with this model. Experiments show that the proposed method is very effective. It has achieved comparable performance to the state-of-the-art shape-based detection methods, but requires much less training information.

Spatiotemporal Localization and Categorization of Human Actions in Unsegmented Image Sequences

In this paper we address the problem of localization and recognition of human activities in unsegmented image sequences. The main contribution of the proposed method is the use of an implicit representation of the spatiotemporal shape of the activity which relies on the spatiotemporal localization of characteristic ensembles of feature descriptors. Evidence for the spatiotemporal localization of the activity is accumulated in a probabilistic spatiotemporal voting scheme. The local nature of the proposed voting framework allows us to deal with multiple activities taking place in the same scene, as well as with activities in the presence of clutter and occlusion. We use boosting in order to select characteristic ensembles per class. This leads to a set of class specific codebooks where each codeword is an ensemble of features. During training, we store the spatial positions of the codeword ensembles with respect to a set of reference points, as well as their temporal positions with respect to the start and end of the action instance. During testing, each activated codeword ensemble casts votes concerning the spatiotemporal position and extend of the action, using the information that was stored during training. Mean Shift mode estimation in the voting space provides the most probable hypotheses concerning the localization of the subjects at each frame, as well as the extend of the activities depicted in the image sequences. We present classification and localization results for a number of publicly available datasets, and for a number of sequences where there is a significant amount of clutter and occlusion.

Multiscale Categorical Object Recognition Using Contour Fragments

Psychophysical studies [9], [17] show that we can recognize objects using fragments of outline contour alone. This paper proposes a new automatic visual recognition system based only on local contour features, capable of localizing objects in space and scale. The system first builds a class-specific codebook of local fragments of contour using a novel formulation of chamfer matching. These local fragments allow recognition that is robust to within-class variation, pose changes, and articulation. Boosting combines these fragments into a cascaded sliding-window classifier, and mean shift is used to select strong responses as a final set of detections. We show how learning can be performed iteratively on both training and test sets to boot-strap an improved classifier. We compare with other methods based on contour and local descriptors in our detailed evaluation over 17 challenging categories, and obtain highly competitive results. The results confirm that contour is indeed a powerful cue for multi-scale and multi-class visual object recognition.