Pre-learned Classifiers Research Articles

Multilayer deep features with multiple kernel learning for action recognition

In accurate action recognition, discriminative human-region representation as auxiliary information is critical for fusing multiple visual clues in a video and further improving the recognition performance. To this end, in this paper we propose integrating a novel representation named multilayer deep features (MDF) of both the human region and whole image area into an extended region-aware multiple kernel learning (ER-MKL) framework. To be specific, we first exploit the human cues with the help of the off-the-shelf semantic segmentation models. Then more powerful representations MDF are constructed by concatenating activations at the last convolutional layer and fully connected layer. Finally, the proposed framework termed ER-MKL is presented to learn a robust classifier for fusing human-region MDF and whole-region MDF. In addition to combining multiple kernels derived from features of heterogeneous image regions, ER-MKL also considers the sets of pre-learned classifiers and incorporates prior knowledge of different regions. Extensive evaluations on the JHMDB and UCF Sports datasets validate the effectiveness and the superiority of our proposed approach.

Hierarchical deep semantic representation for visual categorization

Visual features are unsatisfactory to effectively describe the visual semantics. However, single layer based semantic modeling may be not able to cope with complicated semantic contents. In this paper, we propose Hierarchical Deep Semantic Representation (H-DSR), a hierarchical framework which combines semantic context modeling with visual features. First, the input image is sampled with spatially fixed grids. Deep features are then extracted for each sample in particular location. Second, using pre-learned classifiers, a detection response map is constructed for each patch. Semantic representation is then extracted from the map, which have a sense of latent semantic context. We combine the semantic and visual representations for joint representation. Third, a hierarchical deep semantic representation is built with recurrent reconstructions using three layers. The concatenated visual and semantic representations are used as the inputs of subsequent layers for semantic representation extraction. Finally, we verify the effectiveness of H-DSR for visual categorization on two publicly available datasets: Oxford Flowers 17 and UIUC-Sports. Improved performances are obtained over many baseline methods.

LMDT: A weakly-supervised large-margin-domain-transfer for handwritten digit recognition

Performance of handwritten character recognition systems degrades significantly when they are trained and tested on different databases. In this paper, we propose a novel large margin domain transfer algorithm, which is able to jointly reduce the data distribution mismatch of training (source) and test (target) datasets, as well as learning a target classifier by relying on a set of pre-learned classifiers with the labeled source data in addition to a few available target labels. The proposed method optimizes the combination coefficients of pre-learned classifiers to obtain the minimum mismatch between results on the source and target datasets. Our method is applicable both in semi-supervised and unsupervised domain adaptation scenarios, while most of the previous competing domain adaptation methods work only in semi-supervised scenario. Experiments on adaptation to different handwritten digit datasets demonstrate that this method achieves superior classification accuracy on target sets, comparing to the state of the art methods. Quantitative evaluation shows that an unsupervised adaptation reduces the error rates by 40.2% comparing with the SVM classifier trained by the labeled samples from the source domain.

Domain Adaptation of Deformable Part-Based Models.

The accuracy of object classifiers can significantly drop when the training data (source domain) and the application scenario (target domain) have inherent differences. Therefore, adapting the classifiers to the scenario in which they must operate is of paramount importance. We present novel domain adaptation (DA) methods for object detection. As proof of concept, we focus on adapting the state-of-the-art deformable part-based model (DPM) for pedestrian detection. We introduce an adaptive structural SVM (A-SSVM) that adapts a pre-learned classifier between different domains. By taking into account the inherent structure in feature space (e.g., the parts in a DPM), we propose a structure-aware A-SSVM (SA-SSVM). Neither A-SSVM nor SA-SSVM needs to revisit the source-domain training data to perform the adaptation. Rather, a low number of target-domain training examples (e.g., pedestrians) are used. To address the scenario where there are no target-domain annotated samples, we propose a self-adaptive DPM based on a self-paced learning (SPL) strategy and a Gaussian Process Regression (GPR). Two types of adaptation tasks are assessed: from both synthetic pedestrians and general persons (PASCAL VOC) to pedestrians imaged from an on-board camera. Results show that our proposals avoid accuracy drops as high as 15 points when comparing adapted and non-adapted detectors.

Learning the object location, scale and view for image categorization with adapted classifier

In this paper, we first propose a scheme to learn with automatically aligned objects for image categorization. To fulfil this purpose, the learning is formulated into a multiple instance learning problem with bags. Each bag is a virtually generated image set from an original image. The virtually generated image set covers multiple possible object locations, scales and views. Secondly, we propose a novel adaptive learning method to take advantage of the sharing information among different categories in image categorization. The method models the sharing information between one category and the other categories by learning a classifier of the current category from perturbed pre-learned classifiers from other categories. And the adaptive learning is realized in the multiple kernel learning framework. Finally, to align the images from one category and share the information among different categories simultaneously, the proposed multiple instance learning scheme and the proposed adaptive learning method are integrated into one learning framework. A new formulation called Adaptive Multiple Instance Learning (A-MIL) is derived. The optimization method is provided. To evaluate the proposed method, comprehensive experiments are conducted on two well-known public image categorization datasets (Caltech101 and 15Scenes). The experimental results show that the proposed method outperforms the state-of-the-arts in classification accuracy.

A subset method for improving Linear Discriminant Analysis

Linear Discriminant Analysis (LDA) is one of the most popular methods for dimension reduction. However, it suffers from class separation problem for C-class when the reduced dimensionality is less than C−1. To cope with this problem, we propose a subset improving method in this paper. In the method, the subspaces are found for each subset rather than that for the entire data set. To partition the entire data set into subsets, a cost matrix is first estimated from the training set with the pre-learned classifier, then the graph cut method is adopted to minimize the cost between each subset. We use LDA to find subspaces for each subset. Experimental results based on different applications demonstrate both the generality and effectiveness of the proposed method.

Action Recognition Using Multilevel Features and Latent Structural SVM

We first propose a new low-level visual feature, called spatio-temporal context distribution feature of interest points, to describe human actions. Each action video is expressed as a set of relative XYT coordinates between pairwise interest points in a local region. We learn a global Gaussian mixture model (GMM) (referred to as a universal background model) using the relative coordinate features from all the training videos, and then we represent each video as the normalized parameters of a video-specific GMM adapted from the global GMM. In order to capture the spatio-temporal relationships at different levels, multiple GMMs are utilized to describe the context distributions of interest points over multiscale local regions. Motivated by the observation that some actions share similar motion patterns, we additionally propose a novel mid-level class correlation feature to capture the semantic correlations between different action classes. Each input action video is represented by a set of decision values obtained from the pre-learned classifiers of all the action classes, with each decision value measuring the likelihood that the input video belongs to the corresponding action class. Moreover, human actions are often associated with some specific natural environments and also exhibit high correlation with particular scene classes. It is therefore beneficial to utilize the contextual scene information for action recognition. In this paper, we build the high-level co-occurrence relationship between action classes and scene classes to discover the mutual contextual constraints between action and scene. By treating the scene class label as a latent variable, we propose to use the latent structural SVM (LSSVM) model to jointly capture the compatibility between multilevel action features (e.g., low-level visual context distribution feature and the corresponding mid-level class correlation feature) and action classes, the compatibility between multilevel scene features (i.e., SIFT feature and the corresponding class correlation feature) and scene classes, and the contextual relationship between action classes and scene classes. Extensive experiments on UCF Sports, YouTube and UCF50 datasets demonstrate the effectiveness of the proposed multilevel features and action-scene interaction based LSSVM model for human action recognition. Moreover, our method generally achieves higher recognition accuracy than other state-of-the-art methods on these datasets.