Scatter Matrices Research Articles

A fast algorithm for feature extraction of hyperspectral images using the first order statistics

A new supervised feature extraction method appropriate for small sample size situations is proposed in this work. The proposed method is based on the first-order statistics, in which there is no need to estimate the scatter matrices. Thus, the presented method not only can avoid the singularity problem in small sample size situations but also can achieve high performance in such situations. In addition, due to the fact that the proposed algorithm only exploits the first order statistical moments, it is very fast making it suitable for real-time hyperspectral scene analysis. The proposed method makes a matrix whose columns are obtained by averaging training samples of different classes. Then, a new transform is used to map the features from the original space into a new low-dimensional space such that the new features are as different from each other as possible. Subsequently, to capture the inherent nonlinearity of the original data, the algorithm is improved using the kernel trick. In experiments, four widely-used hyperspectral datasets, namely, Indian Pines, University of Pavia, Salinas, and Botswana are classified. The experimental results show that the proposed algorithm achieves state-of-the-art results in small sample size situations.

Performance of Linear Discriminant Analysis Using Different Robust Methods

This study aims to combine the new deterministic minimum covariance determinant (DetMCD) algorithm with linear discriminant analysis (LDA) and compare it with the fast minimum covariance determinant (FastMCD), fast consistent high breakdown (FCH), and robust FCH (RFCH) algorithms. LDA classifies new observations into one of the unknown groups and it is widely used in multivariate statistical analysis. The LDA mean and covariance matrix parameters are highly influenced by outliers. The DetMCD algorithm is highly robust and resistant to outliers and it is constructed to overcome the outlier problem. Moreover, the DetMCD algorithm is used to estimate location and scatter matrices. The DetMCD, FastMCD, FCH, and RFCH algorithms are applied to estimate misclassification probability using robust LDA. All the algorithms are expected to improve the LDA model for classification purposes in banks, such as bankruptcy and failures, and to distinguish between Islamic and conventional banks. The performances of the estimators are investigated through simulation and actual data.

Feature Extraction Using Fractional-Order Embedding Direct Linear Discriminant Analysis

In this paper, a novel LDA-based dimensionality reduction method called fractional-order embedding direct LDA (FEDLDA) is proposed. More specifically, we redefine the fractional-order between-class and within-class scatter matrices which can significantly reduce the deviation of sample covariance matrices caused by the noise disturbance and limited number of training samples; then the novel feature extraction criterion based on the direct LDA (DLDA) and the idea of fractional-order embedding is applied. Experiments on AT&T, Yale and AR face image databases are performed to test and evaluate the effectiveness of the proposed algorithms. Extensive experimental results show that FEDLDA outperforms DLDA and other closely related methods in terms of classification accuracy and efficiency.

Nonparametrically trained PLDA for short duration i-vector speaker verification

The duration of speech segments can significantly impact the performance of text-independent speaker verification systems. In real world applications which require high accuracy on short utterances, the performance of i-vector speaker verification framework degrades significantly considering that i-vectors extracted from short utterances are less reliable (i.e., uncertainty is higher) than those extracted from long utterances. Therefore, to handle duration variability properly, a more realistic approach seems to be required. This study is an extension to our recently proposed nearest neighbor probabilistic linear discriminant analysis (NN-PLDA) which estimates the parameters of PLDA in i-vector speaker verification framework using a nonparametric form rather than maximum likelihood estimation (MLE) obtained by an EM algorithm, and has been shown to provide superior performance. In NN-PLDA, the between-speaker covariance matrix that represents global information about the speaker variability is replaced with a local estimation computed on a nearest neighbor basis for each target speaker. Compared to their parametric counterparts, the nonparametric between- and within-speaker scatter matrices can better exploit the discriminant information in training data and are more adapted to sample distributions. In this paper, we provide further analysis on the proposed nonparametrically trained PLDA as well as introduce a duration variability modeling technique in the estimation of the within-speaker scatter matrix as to compensate for the effect of limited speech data. We evaluate our approach using core–10sec and 10sec–10sec telephone trial conditions of NIST 2010 SRE as well as on the truncated test utterances in extended core condition with duration less than 10 s. We also present the results obtained by the successful incorporation of NN-PLDA on the recent NIST 2016 speaker recognition evaluation. In all experiments, considerable performance improvement is obtained with the proposed technique compared to a generatively trained PLDA model.

Performance of an automatic inspection system for classification of Fusarium Moniliforme damaged corn seeds by image analysis

This paper presents algorithms for pre-processing, feature selection and classifier design which are used for Parameter Refining in the task of development of an automatic system for recognition and grading of corn seeds with external signs of Fusarium Moniliforme disease. The abilities of several feature selection methods – FDR, Scatter matrices and Stepwise Discriminant Analysis and two classification methods - Support Vector Machine (SVM) and K-Nearest Neighbours (K-NN) are investigated. Design and implementation of the system also has been showed. The system could continually present one by one positioned corn kernels to CCD camera, perform a classification procedure of captured images and discharge seeds to assigned containers. The software was developed in LabVIEW environment including image analysis and classification procedures performed using MATLAB Script. Results for total error rate of 8.4% - 7.2% from preliminary classification related to 8480 seeds from16 Bulgarian varieties and total error rate of 6.95% - 20.4% for experimental results obtained with the system during the control measurements of the seed sample are obtained.

RECOGNITION OF LOCAL CAPITAL USING SYMMETRIC TWO-DIMENSIONAL LINEAR DISCRIMINANT ANALYSIS

Statistical pattern recognition is a system that aims to classify a number of objects to a number of categories or classes. Given a data matrix A, A = {Π1, Π2,…, Πk} where Πi consist of ni point data of ith class then patterns in each classes can classify and separate distance of within and between-class in datasets. In this paper, Symmetric Two-Dimensional Linear Discriminant Analysis proposed to maximize the between-class scatter matrices (Sb) and minimize the within-class scatter matrices (Sw), and can recognition the symmetric capital letter by hand writing such as A, B, C, D, E, H, I, K, M, O, S, U, V, W and Y by using ADL2-D algorithm.

Detecting Deviating Data Cells

ABSTRACTA multivariate dataset consists of n cases in d dimensions, and is often stored in an n by d data matrix. It is well-known that real data may contain outliers. Depending on the situation, outliers may be (a) undesirable errors, which can adversely affect the data analysis, or (b) valuable nuggets of unexpected information. In statistics and data analysis, the word outlier usually refers to a row of the data matrix, and the methods to detect such outliers only work when at least half the rows are clean. But often many rows have a few contaminated cell values, which may not be visible by looking at each variable (column) separately. We propose the first method to detect deviating data cells in a multivariate sample which takes the correlations between the variables into account. It has no restriction on the number of clean rows, and can deal with high dimensions. Other advantages are that it provides predicted values of the outlying cells, while imputing missing values at the same time. We illustrate the method on several real datasets, where it uncovers more structure than found by purely columnwise methods or purely rowwise methods. The proposed method can help to diagnose why a certain row is outlying, for example, in process control. It also serves as an initial step for estimating multivariate location and scatter matrices.

Self-weighted discriminative feature selection via adaptive redundancy minimization

Abstract In this paper, a novel self-weighted orthogonal linear discriminant analysis (SOLDA) method is firstly proposed, such that optimal weight can be automatically achieved to balance both between-class and within-class scatter matrices. Since correlated features tend to have similar rankings, multiple adopted criteria might lead to the state that top ranked features are selected with large correlations, such that redundant information is brought about. To minimize associated redundancy, an original regularization term is introduced to the proposed SOLDA problem to penalize the high-correlated features. Different from other methods and techniques, we optimize redundancy matrix as a variable instead of setting it as a priori, such that correlations among all the features can be adaptively evaluated. Additionally, a brand new recursive method is derived to achieve the selection matrix heuristically, such that closed form solution can be obtained with holding the orthogonality. Consequently, self-weighted discriminative feature selection via adaptive redundancy minimization (SDFS-ARM) method can be summarized, such that non-redundant discriminative features could be selected correspondingly. Eventually, the effectiveness of the proposed SDFS-ARM method is further validated by the empirical results.

A new hybrid feature extraction method in a dyadic scheme for classification of hyperspectral data

ABSTRACTThe feature extraction is an important preprocessing step of the classification procedure particularly in high-dimensional data with limited number of training samples. Conventional supervised feature extraction methods, for example, linear discriminant analysis (LDA), generalized discriminant analysis, and non-parametric weighted feature extraction ones, need to calculate scatter matrices. In these methods, within-class and between-class scatter matrices are used to formulate the criterion of class separability. Because of the limited number of training samples, the accurate estimation of these matrices is not possible. So the classification accuracy of these methods falls in a small sample size situation. To cope with this problem, a new supervised feature extraction method namely, feature extraction using attraction points (FEUAP) has been recently proposed in which no statistical moments are used. Thus, it works well using limited training samples. To take advantage of this method and LDA one, this article combines them by a dyadic scheme. In the proposed scheme, the similar classes are grouped hierarchically by the k-means algorithm so that a tree with some nodes is constructed. Then the class of each pixel is determined from this scheme. To determine the class of each pixel, depending on the node of the tree, we use FEUAP or LDA for a limited or large number of training samples, respectively. The experimental results demonstrate the better performance of the proposed hybrid method in comparison with other supervised feature extraction methods in a small sample size situation.

Block linear discriminant analysis for visual tensor objects with frequency or time information

Recently, due to the advancement of acquisition techniques, visual tensor data have been accumulated in a great variety of engineering fields, e.g., biometrics, neuroscience, surveillance and remote sensing. How to analyze and learn with such tensor objects thus becomes an important and growing interest in machine learning community. In this paper, we propose a block linear discriminant analysis (BLDA) algorithm to extract features for visual tensor objects such as multichannel/hyperspectral face images or human gait videos. Taking the inherent characteristic of such tensor data into account, we unfold tensor objects according to their spatial information and frequency/time information, and represent them in a block matrix form. As a result, the block form between-class and within-class scatter matrices are constructed, and a related block eigen-decomposition is solved to extract features for classification. Comprehensive experiments have been carried out to test the effectiveness of the proposed method, and the results show that BLDA outperforms existing algorithms like DATER, 2DLDA, GTDA, UMLDA, STDA and MPCA for visual tensor object analysis.

Nonnegative Discriminant Matrix Factorization

Nonnegative matrix factorization (NMF), which aims at obtaining the nonnegative low-dimensional representation of data, has received wide attention. To obtain more effective nonnegative discriminant bases from the original NMF, in this paper, a novel method called nonnegative discriminant matrix factorization (NDMF) is proposed for image classification. NDMF integrates the nonnegative constraint, orthogonality, and discriminant information in the objective function. NDMF considers the incoherent information of both factors in standard NMF and is proposed to enhance the discriminant ability of the learned base matrix. NDMF projects the low-dimensional representation of the subspace of the base matrix to regularize the NMF for discriminant subspace learning. Based on the Euclidean distance metric and the generalized Kullback-Leibler (KL) divergence, two kinds of iterative algorithms are presented to solve the optimization problem. The between- and within-class scatter matrices are divided into positive and negative parts for the update rules and the proofs of the convergence are also presented. Extensive experimental results demonstrate the effectiveness of the proposed method in comparison with the state-of-the-art discriminant NMF algorithms.

Low-resolution face recognition with single sample per person

As a growing number of low-resolution (LR) face images are captured by surveillance cameras, LR face recognition has been a hot issue for recent years. Previous efforts on LR face recognition typically assume each subject has multiple high-resolution (HR) training samples. However, this assumption may not hold in some special cases such as law-enforcement where only a single HR sample per person exists in the training set. For LR face recognition in SSPP scenario, it often suffers from overfitting and singular matrix problems. In this paper, we are the first to investigate LR face recognition with single sample per person, and propose a cluster-based regularized simultaneous discriminant analysis (C-RSDA) method based on SDA. C-RSDA regularizes the between-class and within-class scatter matrices respectively with inter-cluster and intra-cluster scatter matrices, where the cluster-based scatter matrices are computed from unsupervised clustering. With the cluster-based scatter matrices, not only the singularity problem is resolved, but overfitting problem is overcomed as more variations are exploited from the limited training samples. Thus, the proposed C-RSDA enhances the discriminative power of the feature subspace. We extensively evaluate C-RSDA on recognizing LR face images captured in both controlled and uncontrolled environments. The encouraging experimental results demonstrate the effectiveness of the proposed approach.

Separability-Oriented Subclass Discriminant Analysis.

Linear discriminant analysis (LDA) is a classical method for discriminative dimensionality reduction. The original LDA may degrade in its performance for non-Gaussian data, and may be unable to extract sufficient features to satisfactorily explain the data when the number of classes is small. Two prominent extensions to address these problems are subclass discriminant analysis (SDA) and mixture subclass discriminant analysis (MSDA). They divide every class into subclasses and re-define the within-class and between-class scatter matrices on the basis of subclass. In this paper we study the issue of how to obtain subclasses more effectively in order to achieve higher class separation. We observe that there is significant overlap between models of the subclasses, which we hypothesise is undesirable. In order to reduce their overlap we propose an extension of LDA, separability oriented subclass discriminant analysis (SSDA), which employs hierarchical clustering to divide a class into subclasses using a separability oriented criterion, before applying LDA optimisation using re-defined scatter matrices. Extensive experiments have shown that SSDA has better performance than LDA, SDA and MSDA in most cases. Additional experiments have further shown that SSDA can project data into LDA space that has higher class separation than LDA, SDA and MSDA in most cases.

A Novel Generalized Fuzzy Canonical Correlation Analysis Framework for Feature Fusion and Recognition

In this paper, a novel CCA-based dimensionality reduction method called generalized fuzzy canonical correlation analysis (GFCCA) is proposed. GFCCA combines the generalized canonical correlation analysis and fuzzy set theory. GFCCA redefines the fuzzy between-class and within-class scatter matrices that relate directly to the samples distribution information. For nonlinear separated problems, we extend the kernel extension of GFCCA with positive definite kernels and indefinite kernels. Experiments on real-world data sets are performed to test and evaluate the effectiveness of the proposed algorithms.

Robust Maximum Association Estimators

ABSTRACTThe maximum association between two multivariate variables and is defined as the maximal value that a bivariate association measure between one-dimensional projections and can attain. Taking the Pearson correlation as projection index results in the first canonical correlation coefficient. We propose to use more robust association measures, such as Spearman’s or Kendall’s rank correlation, or association measures derived from bivariate scatter matrices. We study the robustness of the proposed maximum association measures and the corresponding estimators of the coefficients yielding the maximum association. In the important special case of being univariate, maximum rank correlation estimators yield regression estimators that are invariant against monotonic transformations of the response. We obtain asymptotic variances for this special case. It turns out that maximum rank correlation estimators combine good efficiency and robustness properties. Simulations and a real data example illustrate the robustness and the power for handling nonlinear relationships of these estimators. Supplementary materials for this article are available online.

Design of robust fractional order PID controllers for coupled tank systems using multi-objective particle swarm optimisation

Liquid level control in multiple connected tanks is a nonlinear control problem and is centre to various diverse industrial establishments, especially chemical process industries. The quality of products and process safety greatly relies upon the quality of control. In this paper, robust liquid level control for coupled tank systems has been proposed using optimal fractional order PID controllers. The controller tuning has been posed as multi-objective mixed sensitivity minimisation problem and is solved using multi-objective variant of particle swarm optimisation (MOPSO). At the end of the optimisation process, Pareto optimal set (POS) of solutions has been obtained. For choosing an optimal solution from the POS, there are several graphical methods like Pareto front, scatter matrices, parallel coordinate representations, etc. But these methods offer a very compact representation of the data and the problem arises for m-dimensional problems. So to ease the decision-making process, the paper also explores the use of level diagrams for aiding the decision maker in choosing the ideal solution from the Pareto optimal set. The designed fractional order PID controllers present robust behaviour to external disturbances and efficient tracking of the reference signal and the results obtained has been compared with different controller tuning algorithms.

Supervised Feature Extraction of Hyperspectral Images Using Partitioned Maximum Margin Criterion

Dimensionality reduction is an important task where the aim is to reduce the number of features and make the system less time consuming for classification. Here, the drawbacks of Fisher’s linear-discriminant-analysis-based feature extraction (FE) methods are addressed and a proposal is made to overcome it as well as to reduce the Hughes phenomenon and computational complexity of the system. The proposed FE technique initially partitions the complete set of features into several highly correlated subgroups. Then a linear transformation is performed using a maximal margin criterion over each subgroup. The proposed method is supervised in nature, because prior information about the class label of data is required to calculate the maximum margin criterion based on interclass and intraclass scatter matrices. Experiments are conducted with the PaviaU and Indian pine data sets, and the results are compared with five state-of-the-art techniques, both qualitatively and quantitatively, to demonstrate the effectiveness of the proposed method.

Examination the Usefulness of Selected Classification Methods in the Pre-Qualification of Construction Contractors

For most of projects, including major infrastructure enterprises a key success factor is an appropriately selected general contractor. In addition to technical and organizational criteria the assessment of its financial capacity is strongly important to provide liquidity to the cash flows of the project. Unfortunately, the practical experience of the construction market during the economic downturn shows that many projects have been delayed or suspended as a result of the bankruptcy of general contractors. The purpose of this paper is to present the results of studies on the use of Partial Least Square, Fisher Linear Discriminant, Support Vector Machine methods, and the authors' proprietary approach to the Extraction by Applying Weighted Criterion of Difference Scatter Matrices (EWCDSM) for the pre-qualification of potential contractors. The study included an ex post evaluation of selected large construction companies, operated in Poland by reference the fact of their bankruptcy or maintain the market to historical financial data for the prior year period. Then, for the same set of data the authors performed the identification of companies at risk of bankruptcy. The results obtained are confronted with the actual condition of the surveyed companies. Proposed operating models have been validated on publicly available financial data of construction companies listed on the Polish Stock Exchange.

The evaluation of socio-economic development of development agency regions in Turkey using classical and robust principal component analyses

ABSTRACTIn this study, classical and robust principal component analyses are used to evaluate socioeconomic development of regions of development agencies that give service on the purpose of decreasing development difference among regions in Turkey. Due to the high differences between development levels of regions outlier problem occurs, hence robust statistical methods are used. Also, classical and robust statistical methods are used to investigate if there are any outliers in data set. In classic principal component analyse, the number of observations must be larger than the number of variables. Otherwise determinant of covariance matrix is zero. In Robust method for Principal Component Analysis (ROBPCA), a robust approach to principal component analyse in high-dimensional data, even if the number of variables is larger than the number of observations, principal components are obtained. In this paper, firstly 26 development agencies are evaluated with 19 variables by using principal component analysis based on classical and robust scatter matrices and then these 26 development agencies are evaluated with 46 variables by using the ROBPCA method.

Probability Maximum Margin Criterion for Crop Disease Recognition

<abstract> <sc><b>Abstract.</b> </sc>Crop disease seriously affects the yield and quality of crops and cause farmers to experience economic losses. Automatic detection of crop disease is an essential research topic to detect the disease symptoms as soon as they appear on the leaves. For crop disease management, it is important to automatically detect the crop diseases at a stage so as to treat them properly. In subspace dimension reduction-based data classification methods, the main topic is to design the between-class and within-class scatter matrices to obtain the mapping matrix from the training samples. But, there is much difference between the training samples. To eliminate the difference in designing the scatter matrices, in this article, based on the prior probability, a probability maximum margin criterion (MMC) is proposed for leaf spot image processing. Compared to the traditional MMC algorithm, in the proposed method, the two scatter matrices are expressed by the weighted mean vector of all training samples, which can overcome the influence of the outliers and the noise points, meanwhile improve the crop disease recognition rate. In this study, more than 100 classifying features are extracted from the disease leaf images of two kinds of cucumber diseases; probability MMC is performed for reduced dimensions in feature data processing, and then K-nearest-neighbor classifier is used to identify the cucumber diseases. The experimental results on a cucumber disease leaf image database show that the proposed method is effective for crop disease recognition.