Local Discriminative Information Research Articles

A Weighted Sparse Neighbourhood-Preserving Projections for Face Recognition

ABSTRACTDimensionality reduction algorithms are widely applied to high-dimensional data pre-processing, especially for face images. In this paper, we propose an unsupervised sparse subspace learning approach called weighted sparse neighbourhood-preserving projections (WSNPP) for face recognition. Unlike many existing approaches such as sparsity-preserving projections (SPP), where the constructive weights are computed by the classical sparse representation (SR), WSNPP utilizes a weighted SR model to represent samples. The obtained projections can contain more local discriminant information than classical sparse subspace learning methods. Moreover, WSNPP puts a constraint on the number of nonzero reconstruction coefficients and hence is more robust to global noises and time saving. Experiments on AR, Yale-B and ORL image datasets demonstrate its effectiveness.

Supervised distance metric learning through maximization of the Jeffrey divergence

Over the past decades, distance metric learning has attracted a lot of interest in machine learning and related fields. In this work, we propose an optimization framework for distance metric learning via linear transformations by maximizing the Jeffrey divergence between two multivariate Gaussian distributions derived from local pairwise constraints. In our method, the distance metric is trained on positive and negative difference spaces, which are built from the neighborhood of each training instance, so that the local discriminative information is preserved. We show how to solve this problem with a closed-form solution rather than using tedious optimization procedures. The solution is easy to implement, and tractable for large-scale problems. Experimental results are presented for both a linear and a kernelized version of the proposed method for k-nearest neighbors classification. We obtain classification accuracies superior to the state-of-the-art distance metric learning methods in several cases while being competitive in others.

Unsupervised discriminative hashing

Hashing is one of the popular solutions for approximate nearest neighbor search because of its low storage cost and fast retrieval speed, and many machine learning algorithms are adapted to learn effective hash function. As hash codes of the same cluster are similar to each other while the hash codes in different clusters are dissimilar, we propose an unsupervised discriminative hashing learning method (UDH) to improve discrimination among hash codes in different clusters. UDH shares a similar objective function with spectral hashing algorithm, and uses a modified graph Laplacian matrix to exploit local discriminant information. In addition, UDH is designed to enable efficient out-of-sample extension. Experiments on real world image datasets demonstrate the effectiveness of our novel approach for image retrieval.

Enhanced manifold regularization for semi-supervised classification.

Manifold regularization (MR) has become one of the most widely used approaches in the semi-supervised learning field. It has shown superiority by exploiting the local manifold structure of both labeled and unlabeled data. The manifold structure is modeled by constructing a Laplacian graph and then incorporated in learning through a smoothness regularization term. Hence the labels of labeled and unlabeled data vary smoothly along the geodesics on the manifold. However, MR has ignored the discriminative ability of the labeled and unlabeled data. To address the problem, we propose an enhanced MR framework for semi-supervised classification in which the local discriminative information of the labeled and unlabeled data is explicitly exploited. To make full use of labeled data, we firstly employ a semi-supervised clustering method to discover the underlying data space structure of the whole dataset. Then we construct a local discrimination graph to model the discriminative information of labeled and unlabeled data according to the discovered intrinsic structure. Therefore, the data points that may be from different clusters, though similar on the manifold, are enforced far away from each other. Finally, the discrimination graph is incorporated into the MR framework. In particular, we utilize semi-supervised fuzzy c-means and Laplacian regularized Kernel minimum squared error for semi-supervised clustering and classification, respectively. Experimental results on several benchmark datasets and face recognition demonstrate the effectiveness of our proposed method.

υ-Support vector machine based on discriminant sparse neighborhood preserving embedding

In this paper, we mainly focus on two issues (1) SVM is very sensitive to noise. (2) The solution of SVM does not take into consideration of the intrinsic structure and the discriminant information of the data. To address these two problems, we first propose an integration model to integrate both the local manifold structure and the local discriminant information into l1 graph embedding. Then we add the integration model into the objection function of υ-support vector machine. Therefore, a discriminant sparse neighborhood preserving embedding υ-support vector machine (υ-DSNPESVM) method is proposed. The theoretical analysis demonstrates that υ-DSNPESVM is a reasonable maximum margin classifier and can obtain a very lower generalization error upper bound by minimizing the integration model and the upper bound of margin error. Moreover, in the nonlinear case, we construct the kernel sparse representation-based l1 graph for υ-DSNPESVM, which is more conducive to improve the classification accuracy than l1 graph constructed in the original space. Experimental results on real datasets show the effectiveness of the proposed υ-DSNPESVM method.

Semi-supervised class-specific feature selection for VHR remote sensing images

ABSTRACTFeatures relevant to a thematic class, that is, class-specific features are beneficial to thematic information extraction. However, existing class-specific feature selection methods require abundant labelled samples, while sample labelling is always labour intensive and time consuming. Therefore, it is necessary to select class-specific features with insufficient labelled objects. In this paper, we raise this problem as semi-supervised class-specific feature selection and propose a new two-stage method. First, a weight matrix fully integrates local geometrical structure and discriminative information. Second, the weight matrix is incorporated into a -norm minimization optimization problem of data reconstruction to objectively measure the effectiveness of features for a thematic class. Different from the explicit binarization in the label vector, the new method only implicitly employs binarization in the weight matrix. With area under receiver-operating characteristic curve, class-specific features result in an increase from 3% and 4% on average for Bayes and linear support vector machine, respectively.

SAR Target Configuration Recognition Using Tensor Global and Local Discriminant Embedding

This letter proposes a method that can preserve the global and local discriminative information based on the tensor representation to achieve feature extraction for synthetic aperture radar (SAR) target configuration recognition. We model SAR images of targets with different configurations as different manifolds, and each manifold is represented as a collection of maximal linear patches (MLPs), each depicted by a subspace. The manifold-to-manifold distance and subspace-to-subspace distance are used to maintain the global discriminative structure of data. Meanwhile, point-to-point distance (PPD) in an MLP is exploited to keep the local discriminative information of data. These two terms are then integrated to maintain the structure of data. Experimental results on the moving and stationary target automatic recognition (MSTAR) database demonstrate the effectiveness of the proposed method.

Hybrid CNN and Dictionary-Based Models for Scene Recognition and Domain Adaptation

Convolutional neural network (CNN) has achieved state-of-the-art performance in many different visual tasks. Learned from a large-scale training dataset, CNN features are much more discriminative and accurate than the hand-crafted features. Moreover, CNN features are also transferable among different domains. On the other hand, traditional dictionarybased features (such as BoW and SPM) contain much more local discriminative and structural information, which is implicitly embedded in the images. To further improve the performance, in this paper, we propose to combine CNN with dictionarybased models for scene recognition and visual domain adaptation. Specifically, based on the well-tuned CNN models (e.g., AlexNet and VGG Net), two dictionary-based representations are further constructed, namely mid-level local representation (MLR) and convolutional Fisher vector representation (CFV). In MLR, an efficient two-stage clustering method, i.e., weighted spatial and feature space spectral clustering on the parts of a single image followed by clustering all representative parts of all images, is used to generate a class-mixture or a classspecific part dictionary. After that, the part dictionary is used to operate with the multi-scale image inputs for generating midlevel representation. In CFV, a multi-scale and scale-proportional GMM training strategy is utilized to generate Fisher vectors based on the last convolutional layer of CNN. By integrating the complementary information of MLR, CFV and the CNN features of the fully connected layer, the state-of-the-art performance can be achieved on scene recognition and domain adaptation problems. An interested finding is that our proposed hybrid representation (from VGG net trained on ImageNet) is also complementary with GoogLeNet and/or VGG-11 (trained on Place205) greatly.

Semi-Supervised Image Classification Based on Local and Global Regression

The insufficiency of labeled samples is a major problem in automatic image annotation. However, unlabeled samples are readily available and abundant. Hence, semi-supervised learning methods, which utilize partly labeled samples and a large amount of unlabeled samples, have attracted increased attention in the field of image classification. During the past decade, graph-based semi-supervised learning became one of the most important research areas in semi-supervised learning. In this letter, we propose a novel and effective graph based semi-supervised learning method for image classification. The new method is based on local and global regression regularization. The local regression regularization adopts a set of local classification functions to preserve both local discriminative and geometrical information; while the global regression regularization preserves the global discriminative information and calculates the projection matrix for out-of-sample extrapolation. Extensive simulations based on synthetic and real-world datasets verify the effectiveness of the proposed method.

An Efficient Semi-Supervised Classifier Based on Block-Polynomial Mapping

In this paper, we propose a block-polynomial mapping for image feature learning, which can be efficiently represented by the matrix Khatri-Rao product. The block-polynomial mapping not only captures the local discriminative information within the image structure, but is also much more efficient than the traditional kernel mapping. Moreover, we embed the proposed mapping into the manifold regularization framework for semi-supervised image classification. Experimental results demonstrate that, while maintaining a comparable classification accuracy, the proposed algorithm performs much more efficient than the state-of-the-art methods.

Neighborhood Discriminant Hashing for Large-Scale Image Retrieval.

With the proliferation of large-scale community-contributed images, hashing-based approximate nearest neighbor search in huge databases has aroused considerable interest from the fields of computer vision and multimedia in recent years because of its computational and memory efficiency. In this paper, we propose a novel hashing method named neighborhood discriminant hashing (NDH) (for short) to implement approximate similarity search. Different from the previous work, we propose to learn a discriminant hashing function by exploiting local discriminative information, i.e., the labels of a sample can be inherited from the neighbor samples it selects. The hashing function is expected to be orthogonal to avoid redundancy in the learned hashing bits as much as possible, while an information theoretic regularization is jointly exploited using maximum entropy principle. As a consequence, the learned hashing function is compact and nonredundant among bits, while each bit is highly informative. Extensive experiments are carried out on four publicly available data sets and the comparison results demonstrate the outperforming performance of the proposed NDH method over state-of-the-art hashing techniques.

Semi-supervised feature selection based on local discriminative information

Feature selection has been an effective way to reduce the dimensionality of the high dimensional data. In this paper, we propose a novel feature selection method which achieves batch feature selection using both supervised and unsupervised data samples. The objective function includes three parts: first, under the assumption that each data sample has been assigned a class label, the ratio of between class scatter matrix and total scatter matrix should be minimized, where the scatter matrices are formed by the selected features of these data samples; second, we use linear regression to model the correlations between the data samples with supervision information and their class labels; last, we use l2,1-norm to guarantee the sparsity of the feature selection matrix and exploit the sharing information between supervised and unsupervised data samples jointly. Different from existing methods, our approach exploits local discriminative information to construct the model, therefore we obtain better results from extensive experiments compared with the existing methods.

Locality Alignment Discriminant Analysis for Visualizing Regional English

In this paper, a novel dimensionality reduction algorithm named locality alignment discriminant analysis (LADA) for visualizing regional English is proposed. In the LADA algorithm, the proposed intrinsic graph or penalty graph measures the similarities between each pairwise textual slices, which can better characterize the intra-class compactness and inter-class separability; the projection matrix obtained by the proposed method is orthogonal, which can eliminate the redundancy between different projection directions, and is more effective for preserving the intrinsic geometry and improving the discriminating ability. To evaluate the performance of the algorithm, a regional written English corpus is designed and collected. Consequently, articles are split into slices and then transformed into 140-dimensional data points by 140 text style markers. Finally, variations existing in the regional written English are attempted to be recognized with our proposed LADA. The similarity among different types of English can be observed by the data plots. The results of visualization and numerical comparison indicate that LADA outperforms other existing algorithms in handling regional English data, as the proposed LADA can better preserve the local discriminative information embedded in the data, which is suitable for pattern classification.

Local discriminant non-negative matrix factorization feature extraction for hyperspectral image classification

Non-negative matrix factorization (NMF) ignores both the local geometric structure of and the discriminative information contained in a data set. A manifold geometry-based NMF dimension reduction method called local discriminant NMF (LDNMF) is proposed in this paper. LDNMF preserves not only the non-negativity but also the local geometric structure and discriminative information of the data. The local geometric and discriminant structure of the data manifold can be characterized by a within-class graph and a between-class graph. An efficient multiplicative updating procedure is produced, and its global convergence is guaranteed theoretically. Experimental results on two hyperspectral image data sets show that the proposed LDNMF is a powerful and promising tool for extracting hyperspectral image features.

Global–local fisher discriminant approach for face recognition

In this paper, we proposed a linear discriminant approach, namely global---local Fisher discriminant analysis (GLFDA) that explicitly considers both the local and global discriminant structures embedded in data. To be specific, GLFDA constructs two graphs to, respectively, model the global and local discriminant structures and then incorporates discriminant structures and local intrinsic structure, which characterizes the within-class compactness, into the objective function for dimensionality reduction. Thus, GLFDA well encodes the discriminant information, especially the local discriminant information of data. Experimental results on AR, YALE, and UMIST databases show the effectiveness of the proposed algorithm.

Supervised Spatio-Temporal Neighborhood Topology Learning for Action Recognition

Supervised manifold learning has been successfully applied to action recognition, in which class label information could improve the recognition performance. However, the learned manifold may not be able to well preserve both the local structure and global constraint of temporal labels in action sequences. To overcome this problem, this paper proposes a new supervised manifold learning algorithm called supervised spatio-temporal neighborhood topology learning (SSTNTL) for action recognition. By analyzing the topological characteristics in the context of action recognition, we propose to construct the neighborhood topology using both supervised spatial and temporal pose correspondence information. Employing the property in locality preserving projection (LPP), SSTNTL solves the generalized eigenvalue problem to obtain the best projections that not only separates data points from different classes, but also preserves local structures and temporal pose correspondence of sequences from the same class. Experimental results demonstrate that SSTNTL outperforms the manifold embedding methods with other topologies or local discriminant information. Moreover, compared with state-of-the-art action recognition algorithms, SSTNTL gives convincing performance for both human and gesture action recognition.

Locally connected graph embedding for semisupervised image classification

For classifying images with various appearances, graph embedding based subspace learning has difficulty in taking a comprehensive consideration of both local geometrical structure and between-class discriminative information. In addition, when no sufficient training samples exist, using only the simple weight graph corresponding to labeled samples, the embedding subspace may not be accurately modeled. We present a semisupervised graph embedding algorithm by combining graph embedding and sparse representation. This algorithm can effectively learn a compact and semantic subspace by using a locally connected graph, which can model the geometrical structure and essential correlation of subclusters within a class and can fully utilize both labeled and unlabeled samples. Moreover, using L2,1-norm, the proposed algorithm can preserve the sparse representation property of images from the original space in the lower dimensional projected space. Our experiments demonstrate that the proposed algorithm has better performance than the alternatives reported in recent literature.

Sparse local discriminant projections for discriminant knowledge extraction and classification

One of the major disadvantages of the linear dimensionality reduction algorithms, such as principle component analysis (PCA) and linear discriminant analysis (LDA), is that the projections are lack of physical interpretation. Moreover, which features or variables play an important role in feature extraction and classification in classical linear dimensionality reduction methods is still not investigated well. This paper proposes a novel supervised learning method called sparse local discriminant projections (SLDPs) for linear dimensionality reduction. Differed from the recent manifold-learning-based methods such as local preserving projections (LPPs), SLDP introduces a sparse constraint into the objective function and integrates the local geometry, discriminant information and within-class geometry to obtain the sparse projections. The sparse projections can be efficiently computed by the Elastic Net. The most important and interesting thing is that the sparse projections learned by SLDP have a direct physical interpretation and provide us the discriminant knowledge and insightful understanding for the extracted features. The experimental results show that SLDP can give reasonable semantic results and achieves competitive performance compared with some techniques such as PCA, LPP, neighbourhood preserving embedding (NPE) and the recently proposed unified sparse subspace learning (USSL).

A unified dimensionality reduction framework for semi-paired and semi-supervised multi-view data

Canonical correlation analysis (CCA) is a popular and powerful dimensionality reduction method to analyze paired multi-view data. However, when facing semi-paired and semi-supervised multi-view data which widely exist in real-world problems, CCA usually performs poorly due to its requirement of data pairing between different views and un-supervision in nature. Recently, several extensions of CCA have been proposed, however, they just handle the semi-paired scenario by utilizing structure information in each view or just deal with semi-supervised scenario by incorporating the discriminant information. In this paper, we present a general dimensionality reduction framework for semi-paired and semi-supervised multi-view data which naturally generalizes existing related works by using different kinds of prior information. Based on the framework, we develop a novel dimensionality reduction method, termed as semi-paired and semi-supervised generalized correlation analysis (S^2GCA). S^2GCA exploits a small amount of paired data to perform CCA and at the same time, utilizes both the global structural information captured from the unlabeled data and the local discriminative information captured from the limited labeled data to compensate the limited pairedness. Consequently, S^2GCA can find the directions which make not only maximal correlation between the paired data but also maximal separability of the labeled data. Experimental results on artificial and four real-world datasets show its effectiveness compared to the existing related dimensionality reduction methods.

Tensor Locally Linear Discriminative Analysis

This letter presents a Tensor Locally Linear Discriminative Analysis (TLLDA) method for image presentation. TLLDA is originated from the Local Fisher Discriminant Analysis (LFDA), but TLLDA offers some advantages over LFDA. 1) TLLDA can preserve the local discriminative information of image data as LFDA. 2) TLLDA represents images as matrices or 2-order tensors rather than vectors, so TLLDA keeps the spatial locality of pixels in the images. 3) TLLDA avoids the singularity that may be suffered by LFDA. 4) TLLDA is faster than LFDA. Simulations on two real databases verified the validity of TLLDA. Results show that TLLDA is highly competitive with some widely used techniques.