Between-class Scatter Matrices Research Articles

Saliency-Based Multilabel Linear Discriminant Analysis.

Linear discriminant analysis (LDA) is a classical statistical machine-learning method, which aims to find a linear data transformation increasing class discrimination in an optimal discriminant subspace. Traditional LDA sets assumptions related to the Gaussian class distributions and single-label data annotations. In this article, we propose a new variant of LDA to be used in multilabel classification tasks for dimensionality reduction on original data to enhance the subsequent performance of any multilabel classifier. A probabilistic class saliency estimation approach is introduced for computing saliency-based weights for all instances. We use the weights to redefine the between-class and within-class scatter matrices needed for calculating the projection matrix. We formulate six different variants of the proposed saliency-based multilabel LDA (SMLDA) based on different prior information on the importance of each instance for their class(es) extracted from labels and features. Our experiments show that the proposed SMLDA leads to performance improvements in various multilabel classification problems compared to several competing dimensionality reduction methods.

Providing loyalty discounts to customers in retail stores

This paper describes the study of different types of algorithms being commonly used for face recognition. We will use the algorithm that yields better accuracy to detect potential customers purchasing in a retail store and accordingly providing discounts based on their loyalty.. In recent times discounts to customers are provided in terms of sale or as a general discounts but there is no scope of discounts based on loyalty or trust that customers show on their purchase irrespective of any age group. Thus the set of all faces in the feature space is treated as a high-dimensional vector space. The faces of different subjects as being in different classes for all subjects in the training set, we establish a framework for performing a cluster separation analysis in the feature space. Also, having labeled all instances in the training set and having defined all the classes, we compute the within and between-class scatter matrices. The most commonly and used mechanism for face detection is Eigen faces algorithm but however we will go through a couple of other algorithms in order to determine better accuracy in face recognition which will our prime goal. Then we will use Fisher's LDA mechanism in order to classify the buyer's as high, mid and low potential groups and accordingly take decision based on that. Hence it can overcome the problem of varying illumination. It is used to extract the magnitude of edge information and it works well even during the variations in poses and the illumination conditions. HOG works well under such challenging situations as it represents directionality of edge information thereby making it significant for the study of pattern and structure of the interested object.

A Novel Approach to Fuzzy-Based Facial Feature Extraction and Face Recognition

Generalized two-dimensional Fisher’s linear discriminant (G-2DFLD) is an effective feature extraction technique that maximizes class separability along row and column directions simultaneously. In this paper, we propose a fuzzy logic-based feature extraction technique, called fuzzy generalized two-dimensional Fisher’s linear discriminant analysis (FG-2DLDA) method which is extended version of the G-2DFLD method. This paper also explores the use of the proposed method for face recognition using radial basis function (RBF) neural network as a classifier. Fuzzy membership matrix values are calculated by fuzzy k -nearest neighbour (F k- NN) algorithm for the training samples. These fuzzy membership values are combined with the training samples to generate global mean and class-wise mean training images. Thereafter, the global and class-wise mean images are used to generate fuzzy within- and between-class scatter matrices along the both directions. Finally, by solving the Eigen value problems of these scatter matrices, we find the optimal fuzzy projection vectors, which actually used to generate more discriminant features. Our proposed method has been evaluated on the four public face databases using RBF neural network and establish that the proposed FG-2DLDA method provides favourable recognition rates than some contemporary face recognition methods.

Optimized regularized linear discriminant analysis for feature extraction in face recognition

In a reduced dimensional space, linear discriminant analysis looks for a projective transformation that can maximizes separability among classes. Since linear discriminant analysis demands the within-class scatter matrix appear to non-singular, which cannot directly used in condition of small sample size (SSS) issues in which the dimension of image is much higher, while the number of samples isn’t unlimited. Both the between-class and within-class scatter matrices are always exceedingly ill-posed in SSS problems. And many algorithms are suffered from small sample size issues still. To solve SSS problems, many methods including regularized linear discriminant analysis were proposed. In this article, a way was presented by optimized regularized linear discriminant analysis for feature extraction in FR which can not only fix the singularity problem existing in scatter matrix but also the problem of parameter estimation. The experiment is conducted on several databases and promising results are obtained compared to some state-of-the-art methods to demonstrate the effectiveness of the proposed approach.

Semi-Supervised Tensorial Locally Linear Embedding for Feature Extraction Using PolSAR Data

Recently, the semi-supervised feature extraction, which uses both a few labeled samples and a large quantity of unlabeled samples to improve discrimination of extracted features, has attracted attention in PolSAR data processing. However, most of the feature extraction methods are based on matrix operations and need to re-arrange the multiple features into vectors. Such a vectorization way may break the spatial structure and correlation among the neighboring pixels, which leads to the degradation of classification performance. To address this issue and incorporate the superiority of semi-supervised learning, a novel semi-supervised tensorial locally linear embedding (STLLE) method is proposed for PolSAR data. Each pixel and its spatial neighbors in a local window are represented as a third-order tensor, whose first two orders represent the neighborhood pixels, and the third order denotes the multiple features. The feature extraction is converted to find an optimal projection that can transform the high-dimensional features to low-dimensional ones. This can be solved by designing an objective function, in which the tensorial discriminant item includes the between-class and within-class scatter matrices on each mode of the labeled tensors. A regularizer is based on tensorial locally linear embedding graph using labeled and unlabeled tensor samples. Moreover, the pairwise class constraints are also introduced. Finally, we can use all the extracted features with a classifier such as K-Nearest Neighbor. Experimental results on three real PolSAR datasets demonstrate that STLLE significantly improves classification accuracies compared with state-of-the-art methods, which clearly verifies that the extracted features preserve the data structures well and are more discriminative.

RECOGNITION OF LOCAL CAPITAL USING SYMMETRIC TWO-DIMENSIONAL LINEAR DISCRIMINANT ANALYSIS

Statistical pattern recognition is a system that aims to classify a number of objects to a number of categories or classes. Given a data matrix A, A = {Π1, Π2,…, Πk} where Πi consist of ni point data of ith class then patterns in each classes can classify and separate distance of within and between-class in datasets. In this paper, Symmetric Two-Dimensional Linear Discriminant Analysis proposed to maximize the between-class scatter matrices (Sb) and minimize the within-class scatter matrices (Sw), and can recognition the symmetric capital letter by hand writing such as A, B, C, D, E, H, I, K, M, O, S, U, V, W and Y by using ADL2-D algorithm.

Self-weighted discriminative feature selection via adaptive redundancy minimization

Abstract In this paper, a novel self-weighted orthogonal linear discriminant analysis (SOLDA) method is firstly proposed, such that optimal weight can be automatically achieved to balance both between-class and within-class scatter matrices. Since correlated features tend to have similar rankings, multiple adopted criteria might lead to the state that top ranked features are selected with large correlations, such that redundant information is brought about. To minimize associated redundancy, an original regularization term is introduced to the proposed SOLDA problem to penalize the high-correlated features. Different from other methods and techniques, we optimize redundancy matrix as a variable instead of setting it as a priori, such that correlations among all the features can be adaptively evaluated. Additionally, a brand new recursive method is derived to achieve the selection matrix heuristically, such that closed form solution can be obtained with holding the orthogonality. Consequently, self-weighted discriminative feature selection via adaptive redundancy minimization (SDFS-ARM) method can be summarized, such that non-redundant discriminative features could be selected correspondingly. Eventually, the effectiveness of the proposed SDFS-ARM method is further validated by the empirical results.

A new hybrid feature extraction method in a dyadic scheme for classification of hyperspectral data

ABSTRACTThe feature extraction is an important preprocessing step of the classification procedure particularly in high-dimensional data with limited number of training samples. Conventional supervised feature extraction methods, for example, linear discriminant analysis (LDA), generalized discriminant analysis, and non-parametric weighted feature extraction ones, need to calculate scatter matrices. In these methods, within-class and between-class scatter matrices are used to formulate the criterion of class separability. Because of the limited number of training samples, the accurate estimation of these matrices is not possible. So the classification accuracy of these methods falls in a small sample size situation. To cope with this problem, a new supervised feature extraction method namely, feature extraction using attraction points (FEUAP) has been recently proposed in which no statistical moments are used. Thus, it works well using limited training samples. To take advantage of this method and LDA one, this article combines them by a dyadic scheme. In the proposed scheme, the similar classes are grouped hierarchically by the k-means algorithm so that a tree with some nodes is constructed. Then the class of each pixel is determined from this scheme. To determine the class of each pixel, depending on the node of the tree, we use FEUAP or LDA for a limited or large number of training samples, respectively. The experimental results demonstrate the better performance of the proposed hybrid method in comparison with other supervised feature extraction methods in a small sample size situation.

Low-resolution face recognition with single sample per person

As a growing number of low-resolution (LR) face images are captured by surveillance cameras, LR face recognition has been a hot issue for recent years. Previous efforts on LR face recognition typically assume each subject has multiple high-resolution (HR) training samples. However, this assumption may not hold in some special cases such as law-enforcement where only a single HR sample per person exists in the training set. For LR face recognition in SSPP scenario, it often suffers from overfitting and singular matrix problems. In this paper, we are the first to investigate LR face recognition with single sample per person, and propose a cluster-based regularized simultaneous discriminant analysis (C-RSDA) method based on SDA. C-RSDA regularizes the between-class and within-class scatter matrices respectively with inter-cluster and intra-cluster scatter matrices, where the cluster-based scatter matrices are computed from unsupervised clustering. With the cluster-based scatter matrices, not only the singularity problem is resolved, but overfitting problem is overcomed as more variations are exploited from the limited training samples. Thus, the proposed C-RSDA enhances the discriminative power of the feature subspace. We extensively evaluate C-RSDA on recognizing LR face images captured in both controlled and uncontrolled environments. The encouraging experimental results demonstrate the effectiveness of the proposed approach.

Separability-Oriented Subclass Discriminant Analysis.

Linear discriminant analysis (LDA) is a classical method for discriminative dimensionality reduction. The original LDA may degrade in its performance for non-Gaussian data, and may be unable to extract sufficient features to satisfactorily explain the data when the number of classes is small. Two prominent extensions to address these problems are subclass discriminant analysis (SDA) and mixture subclass discriminant analysis (MSDA). They divide every class into subclasses and re-define the within-class and between-class scatter matrices on the basis of subclass. In this paper we study the issue of how to obtain subclasses more effectively in order to achieve higher class separation. We observe that there is significant overlap between models of the subclasses, which we hypothesise is undesirable. In order to reduce their overlap we propose an extension of LDA, separability oriented subclass discriminant analysis (SSDA), which employs hierarchical clustering to divide a class into subclasses using a separability oriented criterion, before applying LDA optimisation using re-defined scatter matrices. Extensive experiments have shown that SSDA has better performance than LDA, SDA and MSDA in most cases. Additional experiments have further shown that SSDA can project data into LDA space that has higher class separation than LDA, SDA and MSDA in most cases.

A Novel Generalized Fuzzy Canonical Correlation Analysis Framework for Feature Fusion and Recognition

In this paper, a novel CCA-based dimensionality reduction method called generalized fuzzy canonical correlation analysis (GFCCA) is proposed. GFCCA combines the generalized canonical correlation analysis and fuzzy set theory. GFCCA redefines the fuzzy between-class and within-class scatter matrices that relate directly to the samples distribution information. For nonlinear separated problems, we extend the kernel extension of GFCCA with positive definite kernels and indefinite kernels. Experiments on real-world data sets are performed to test and evaluate the effectiveness of the proposed algorithms.

Probability Maximum Margin Criterion for Crop Disease Recognition

<abstract> <sc><b>Abstract.</b> </sc>Crop disease seriously affects the yield and quality of crops and cause farmers to experience economic losses. Automatic detection of crop disease is an essential research topic to detect the disease symptoms as soon as they appear on the leaves. For crop disease management, it is important to automatically detect the crop diseases at a stage so as to treat them properly. In subspace dimension reduction-based data classification methods, the main topic is to design the between-class and within-class scatter matrices to obtain the mapping matrix from the training samples. But, there is much difference between the training samples. To eliminate the difference in designing the scatter matrices, in this article, based on the prior probability, a probability maximum margin criterion (MMC) is proposed for leaf spot image processing. Compared to the traditional MMC algorithm, in the proposed method, the two scatter matrices are expressed by the weighted mean vector of all training samples, which can overcome the influence of the outliers and the noise points, meanwhile improve the crop disease recognition rate. In this study, more than 100 classifying features are extracted from the disease leaf images of two kinds of cucumber diseases; probability MMC is performed for reduced dimensions in feature data processing, and then K-nearest-neighbor classifier is used to identify the cucumber diseases. The experimental results on a cucumber disease leaf image database show that the proposed method is effective for crop disease recognition.

A new face recognition method via semi-discrete decomposition for one sample problem

Fisher linear discriminant analysis (FLDA) is a popular approach which has been widely used for feature extraction in face recognition tasks. FLDA is seeking optimal projection vectors by maximizing the ratio of between-class scatter matrices vs. within-class scatter matrices. However, the within-class scatter matrices cannot be calculated directly when one training sample each object is available. In this case, image decomposition plays an important role in the reconstructed within-class scatter matrices. So, the performance of image decomposition is crucial. In this paper, a based on semi-discrete decomposition (SDD) method is proposed to solve single training sample image per person problem. First, the original single image and its transpose are calculated via using SDD method in the training set. Then, the within-class scatter matrices are constituted by using the original image and its two approximation images of each person. And meanwhile, the optimal projection vectors are obtained via the FLDA algorithm in the new training set. Finally, we use the nearest neighbor classifier to complete the final classification. The performance of proposed method is evaluated on ORL, Yale and FERET databases. The experimental results show that the proposed method is outperforms singular value decomposition (SVD)-based and QR decomposition with column pivoting (QRCP)-based methods in terms of recognition rates.

Maximum–minimum–median average MSD-based approach for face recognition

A new and efficient improved maximum scatter difference (MSD) model is introduced in this paper. The main weakness of the MSD model is that the class mean vector is constructed via class sample average when the within-class and between-class scatter matrices are formed. For a few of given samples with non-ideal conditions (e.g., variations of expression, pose and noisy environment), the assessment result is very weak by using the class sample average. That is because there will be some outliers in these samples. Therefore, the recognition performance of maximum scatter difference criterion will decline significantly. To solve the problem, in the traditional MSD model, we use within-class maximum–minimum–median average vector to construct within-class scatter matrix (Sw) and between-class scatter matrix (Sb) instead of within-class mean vector. The experimental results show that an improvement of the MSD model is possible with the proposed technique in ORL and Yale face database recognition problems.

A Class-Information-Based Sparse Component Analysis Method to Identify Differentially Expressed Genes on RNA-Seq Data.

With the development of deep sequencing technologies, many RNA-Seq data have been generated. Researchers have proposed many methods based on the sparse theory to identify the differentially expressed genes from these data. In order to improve the performance of sparse principal component analysis, in this paper, we propose a novel class-information-based sparse component analysis (CISCA) method which introduces the class information via a total scatter matrix. First, CISCA normalizes the RNA-Seq data by using a Poisson model to obtain their differential sections. Second, the total scatter matrix is gotten by combining the between-class and within-class scatter matrices. Third, we decompose the total scatter matrix by using singular value decomposition and construct a new data matrix by using singular values and left singular vectors. Then, aiming at obtaining sparse components, CISCA decomposes the constructed data matrix by solving an optimization problem with sparse constraints on loading vectors. Finally, the differentially expressed genes are identified by using the sparse loading vectors. The results on simulation and real RNA-Seq data demonstrate that our method is effective and suitable for analyzing these data.

Robust discriminative extreme learning machine for relevance feedback in image retrieval

Relevance feedback (RF) has long been an important approach for multi-media retrieval because of the semantic gap in image content, where SVM based methods are widely applied to RF of content-based image retrieval. However, RF based on SVM still has some limitations: (1) the high dimension of image features always make the RF time-consuming; (2) the model of SVM is not discriminative, because labels of image features are not sufficiently exploited. To solve above problems, we proposed robust discriminative extreme learning machine (RDELM) in this paper. RDELM involved both robust within-class and between-class scatter matrices to enhance the discrimination capacity of ELM for RF. Furthermore, an angle criterion dimensionality reduction method is utilized to extract the discriminative information for RDELM. Experimental results on four benchmark datasets (Corel-1K, Corel-5K, Corel-10K and MSRC) illustrate that our proposed RF method in this paper achieves better performance than several state-of-the-art methods.

A Hyper spectral Images Classification Method Based on Maximum Scatter Discriminant Analysis

To overcome “small sample size problem” problem faced by some hyper spectral classification methods, the Maximum Scatter Discriminant criterion is used to analyzed hyperspectral data. Maximum Scatter Discriminant analysis searches for the project axes by maximizing the difference of between-class scatter and within-class scatter matrices, which avoid to calculate the inverse of matrices. Experiment results on Indian Pines HSI data set show that the proposed method outperforms the other methods in terms of recognition accuracy. The proposed method is an effective and feasible method for hyper pectral data classification.

Unconstrained face verification with a dual-layer block-based metric learning

In this paper, a dual-layer block-based metric learning technique is proposed to better discriminate the face image pairs and accelerate the overall verification process under the unconstrained environment. The input images are processed as blocks to provide a richer base of face features. Our proposed method is formed by two layers, in which the first layer assists in extracting the compact block-based descriptors without the existence of full class label information and to refine the within-class and between-class scatter matrices while the second layer integrates the face descriptors of all blocks. The proposed scheme has computational advantage over the single metric learning method while it exploits the correlations among the multiple metrics from different descriptors. The performance of our proposed method is evaluated on the Labeled Faces in the Wild database and achieves an improved performance when compared with the state-of-the-art methods in terms of verification rate and computation time.

Phone classification via manifold learning based dimensionality reduction algorithms

Mechanical limitations imposed on the articulators during speech production lead to a limitation of the intrinsic dimensionality of speech signals. This limitation leads to a specific neighborhood structure of speech sounds when they are represented in a high-dimensional feature space. We investigate whether phone classification can be improved by exploiting this neighborhood structure, by means of extended variants of the conventional Linear Discriminant Analysis (LDA) based on manifold learning.In this extended LDA approach, the within-class and between-class scatter matrices are defined in terms of adjacency graphs. We compare extensions of LDA that use either a full adjacency graph or an adjacency graph defined in the neighborhood of the training observations. In addition, we apply different kernels for weighing the distances in the graphs via different kernels, of which the Adaptive Kernel is proposed in this paper.Experiments with TIMIT show that while LDA algorithms that use the full adjacency graph do not outperform traditional LDA, the algorithms that exploit only local information provide significantly better results than traditional LDA. These improvements are not uniform across different broad phonetic classes, which suggests that the added value of the neighborhood structure is phone class dependent. The structure is represented by locally different densities in the neighborhood of feature vectors that are representative of a specific phone in a specific context.

Scatter balance: an angle-based supervised dimensionality reduction.

Subspace selection is widely applied in data classification, clustering, and visualization. The samples projected into subspace can be processed efficiently. In this paper, we research the linear discriminant analysis (LDA) and maximum margin criterion (MMC) algorithms intensively and analyze the effects of scatters to subspace selection. Meanwhile, we point out the boundaries of scatters in LDA and MMC algorithms to illustrate the differences and similarities of subspace selection in different circumstances. Besides, the effects of outlier classes on subspace selection are also analyzed. According to the above analysis, we propose a new subspace selection method called angle linear discriminant embedding (ALDE) on the basis of angle measurement. ALDE utilizes the cosine of the angle to get new within-class and between-class scatter matrices and avoids the small sample size problem simultaneously. To deal with high-dimensional data, we extend ALDE to a two-stage ALDE (TS-ALDE). The synthetic data experiments indicate that ALDE can balance the within-class and between-class scatters and be robust to outlier classes. The experimental results based on UCI machine-learning repository and image databases show that TS-ALDE has a lower time complexity than ALDE while processing high-dimensional data.