Subclass Discriminant Analysis Research Articles

Global subclass discriminant analysis

Linear discriminant analysis (LDA) is a powerful supervised dimensionality reduction method for analysing high-dimensional data. However, LDA cannot use locality information in data, which makes LDA degrade dramatically in performance on multimodal data. A number of LDA variants have been proposed to exploit locality information in data, including subclass-based LDAs. We discover a problem with these variants, which is that subclasses are selected on a within-class basis without considering other classes. This causes the loss of important information at class boundaries. In this paper, we present a novel variant of subclass-based LDA, Global Subclass Discriminant Analysis (GSDA). Unlike other subclass-based LDAs, GSDA selects subclasses from global clusters that may cross class boundaries, thus utilising within-class information and between-class information. More specifically, GSDA applies an effective clustering algorithm to the whole data to construct global clusters. It then utilises the local structure refining strategy on these global clusters to construct subclasses. Finally, GSDA learns a representative data subspace by maximising inter-subclass distance and minimising intra-subclass distance simultaneously. GSDA is extensively evaluated on a wide range of public datasets through comparison with the state-of-the-art LDA algorithms. Experimental results demonstrate its superiority in terms of accuracy and run times.

Eliminating the high false‐positive rate in defect prediction through BayesNet with adjustable weight

AbstractIn defect prediction, a high false‐positive rate (FPR) caused by class imbalance not only increases the workload of testing and development but also consumes unnecessary costs. Many defect models against class imbalance have been proposed to improve the accuracy of defect prediction, but their ability to reduce FPR is unclear. To solve these problems, we first proposed a BayesNet with adjustable weights, called WBN, to reduce the FPR in software defect prediction, which is an algorithm independent of data preprocessing techniques. The mechanism of our WBN is to change the sampling probability of the misclassified instances when training the defect model, making the BayesNet model focus more on false alarm instances. And then, we investigate the FPR of five mainstream defect models for solving class imbalance and select them as comparison models to test the validity of our methods. The experimental result on eight open‐source projects shows that a) our WBN, in in‐version defect prediction (IVDP) and cross‐version defect prediction (CVDP), effectively reduces FPR with means of 0.384 and 0.322, respectively; b) compared with improved subclass discriminant analysis (ISDA) that is the lowest FPR in all control models, our WBN not only reduced the FPR but maintained recall whose mean value was 0.797, whereas ISDA did not, with an average recall of only 0.397; c) our WBN, in CVDP, not only reduces FPR, but also has significant superiority over five control defect models and baseline. Besides, we also found that the class imbalance difference between the test set and the training set has an impact on CVDP performance, recommending that practitioners choose the best dataset for CVDP from the defect data of the historical version through special technology.

Simultaneous Prediction of Wrist and Hand Motions via Wearable Ultrasound Sensing for Natural Control of Hand Prostheses

Simultaneous prediction of wrist and hand motions is essential for the natural interaction with hand prostheses. In this paper, we propose a novel multi-out Gaussian process (MOGP) model and a multi-task deep learning (MTDL) algorithm to achieve simultaneous prediction of wrist rotation (pronation/supination) and finger gestures for transradial amputees via a wearable ultrasound array. We target six finger gestures with concurrent wrist rotation in four transradial amputees. Results show that MOGP outperforms previously reported subclass discriminant analysis for both predictions of discrete finger gestures and continuous wrist rotation. Moreover, we find that MTDL has the potential to improve the accuracy of finger gesture prediction compared to MOGP and classification-specific deep learning, albeit at the expense of reducing the accuracy of wrist rotation prediction. Extended comparative analysis shows the superiority of ultrasound over surface electromyography. This paper prioritizes exploring the performance of wearable ultrasound on the simultaneous prediction of wrist and hand motions for transradial amputees, demonstrating the potential of ultrasound in future prosthetic control. Our ultrasound-based adaptive prosthetic control dataset (Ultra-Pro) will be released to promote the development of the prosthetic community.

Speed-up and multi-view extensions to subclass discriminant analysis

In this paper, we propose a speed-up approach for subclass discriminant analysis and formulate a novel efficient multi-view solution to it. The speed-up approach is developed based on graph embedding and spectral regression approaches that involve eigendecomposition of the corresponding Laplacian matrix and regression to its eigenvectors. We show that by exploiting the structure of the between-class Laplacian matrix, the eigendecomposition step can be substituted with a much faster process. Furthermore, we formulate a novel criterion for multi-view subclass discriminant analysis and show that an efficient solution to it can be obtained in a similar manner to the single-view case. We evaluate the proposed methods on nine single-view and nine multi-view datasets and compare them with related existing approaches. Experimental results show that the proposed solutions achieve competitive performance, often outperforming the existing methods. At the same time, they significantly decrease the training time.

Two-dimensional Subclass Discriminant Analysis for face recognition

Dimensionality reduction plays a major role in face recognition. Discriminant analysis (DA) and principal component analysis (PCA) are two of the most important approaches in this field. In particular, subclass discriminant analysis (SDA) is a well-known scheme for feature extraction and dimensionality reduction. It is widely used in many high-dimensional data-driven applications, namely face recognition and image retrieval. It is also found to be applicable under various scenarios. However, it has high cost in time and space given the need for an eigendecomposition involving the scatter matrices, known as the singularity problem. This limitation is caused by the high-dimensional space of data, particularly when dimensions exceed the number of observations. Recent advances widely reported that 2D methods with matrix-based representation perform better than the traditional 1D vector-based ones. In this paper, we propose a novel 2D-SDA algorithm to avoid the “curse of dimensionality” and address the singularity issue. The performance of the proposed algorithm is evaluated for face recognition in terms of recognition performance and computational cost. Experiments are conducted on four benchmark face databases and compared to several competitive 1D and 2D methods based on PCA and DA. Results show that 2DSVD achieves the best recognition performance at low dimensions. In particular, 2D-SDA works significantly better on large-sized data sets where intra-class variation is the most important.

Simultaneous Prediction of Wrist/Hand Motion via Wearable Ultrasound Sensing.

The ability to predict wrist and hand motions simultaneously is essential for natural controls of hand protheses. In this paper, we propose a novel method that includes subclass discriminant analysis (SDA) and principal component analysis for the simultaneous prediction of wrist rotation (pronation/supination) and finger gestures using wearable ultrasound. We tested the method on eight finger gestures with concurrent wrist rotations. Results showed that SDA was able to achieve accurate classification of both finger gestures and wrist rotations under dynamic wrist rotations. When grouping the wrist rotations into three subclasses, about 99.2 ± 1.2% of finger gestures and 92.8 ± 1.4% of wrist rotations can be accurately classified. Moreover, we found that the first principal component (PC1) of the selected ultrasound features was linear to the wrist rotation angle regardless of finger gestures. We further used PC1 in an online tracking task for continuous wrist control and demonstrated that a wrist tracking precision ( R2 ) of 0.954 ± 0.012 and a finger gesture classification accuracy of 96.5 ± 1.7% can be simultaneously achieved, with only two minutes of user training. Our proposed simultaneous wrist/hand control scheme is training-efficient and robust, paving the way for musculature-driven artificial hand control and rehabilitation treatment.

View-Independent Discriminant Analysis with Gradient Self-Similarities for Action Recognition under Multiple Views

In Multi-View Human Action Recognition, the actions are not of single view and hence to achieve an effective recognition performance under multi-view actions, there is a need of multi-view subclass discrimination analysis. Based on this inspiration, this paper proposed a novel action recognition framework based on the Subclass Discriminant Analysis (SDA), an extended version of Linear Discriminant Analysis (LDA). Further, a new key frames selection method is proposed based on Self-Similarity Matrix (SSM), called as Gradient SSM (GSSM). Once the key frames are selected, a composite feature set is extracted through three different set filters such as Gaussian Filter, Gabor filter and Wavelet Filter. Next, the SDA obtain an optimal feature subspace for every action under multiple Views. Finally the SVM algorithm classifies the action. The proposed framework is systematically evaluated on IXMAS dataset and NIXMAS dataset. Experimental results enumerate that our method outperforms the conventional techniques in terms of recognition accuracy.

Hyperspectral remote sensing image feature extraction based on spectral clustering and subclass discriminant analysis

ABSTRACTHyperspectral remote sensing images (HRSIs) have the problems of high dimensionality and phenomenon of the same subject with different spectra. A class subdivision and feature extraction method based on spectral clustering (SC) and subclass discriminant analysis (SDA), namely SC-SDA, is presented. Firstly, when the overall separability is improved should a class be subdivided. Secondly, a generalized simple matching coefficient (GSMC) is proposed to evaluate the similarity of the clustering results in neighbouring dimensionality SC subspaces, and the SC subspace dimensionality corresponding to the maximum GSMC is selected. Then, SC is performed in the selected SC subspace according to the number of subclasses selected by intra-class separability. Finally, SDA is executed based on the class subdivision result. The experimental results of four real HRSIs datasets show that the classification results of the SC-SDA method are superior to those of linear discriminant analysis, separability-oriented subclass discriminant analysis and SDA methods.

Fault diagnosis based on variable-weighted separability-oriented subclass discriminant analysis

Fisher discriminant analysis (FDA) is a classical method for discriminative dimensionality reduction, which has been extensively studied in the pattern classification field and widely used for fault diagnosis in industrial process monitoring. The classification performance of original FDA may degrade in overlapping, non-Gaussian or nonlinear data, and may be unable to extract sufficient features to interpret the data when the number of classes is small. To overcome these limitations, this paper proposes a combined variable weighting (VW) and separability-oriented subclass discriminant analysis (SSDA) scheme (denoted as VW-SSDA) for fault diagnosis. VW-SSDA utilizes K-means clustering to divide a class into subclasses using a separability oriented criterion, and all fault data are weighted by the corresponding weight vectors before applying FDA optimisation using re-defined scatter matrices. The proposed approach is applied to the Tennessee Eastman process. The results demonstrate that VW-SSDA shows better fault diagnosis performance than the several improved variants of the conventional FDA.

An Improved SDA Based Defect Prediction Framework for Both Within-Project and Cross-Project Class-Imbalance Problems

Background. Solving the class-imbalance problem of within-project software defect prediction (SDP) is an important research topic. Although some class-imbalance learning methods have been presented, there exists room for improvement. For cross-project SDP, we found that the class-imbalanced source usually leads to misclassification of defective instances. However, only one work has paid attention to this cross-project class-imbalance problem. Objective. We aim to provide effective solutions for both within-project and cross-project class-imbalance problems. Method. Subclass discriminant analysis (SDA), an effective feature learning method, is introduced to solve the problems. It can learn features with more powerful classification ability from original metrics. For within-project prediction, we improve SDA for achieving balanced subclasses and propose the improved SDA (ISDA) approach. For cross-project prediction, we employ the semi-supervised transfer component analysis (SSTCA) method to make the distributions of source and target data consistent, and propose the SSTCA+ISDA prediction approach. Results . Extensive experiments on four widely used datasets indicate that: 1) ISDA-based solution performs better than other state-of-the-art methods for within-project class-imbalance problem; 2) SSTCA+ISDA proposed for cross-project class-imbalance problem significantly outperforms related methods. Conclusion . Within-project and cross-project class-imbalance problems greatly affect prediction performance, and we provide a unified and effective prediction framework for both problems.

Separability-Oriented Subclass Discriminant Analysis.

Linear discriminant analysis (LDA) is a classical method for discriminative dimensionality reduction. The original LDA may degrade in its performance for non-Gaussian data, and may be unable to extract sufficient features to satisfactorily explain the data when the number of classes is small. Two prominent extensions to address these problems are subclass discriminant analysis (SDA) and mixture subclass discriminant analysis (MSDA). They divide every class into subclasses and re-define the within-class and between-class scatter matrices on the basis of subclass. In this paper we study the issue of how to obtain subclasses more effectively in order to achieve higher class separation. We observe that there is significant overlap between models of the subclasses, which we hypothesise is undesirable. In order to reduce their overlap we propose an extension of LDA, separability oriented subclass discriminant analysis (SSDA), which employs hierarchical clustering to divide a class into subclasses using a separability oriented criterion, before applying LDA optimisation using re-defined scatter matrices. Extensive experiments have shown that SSDA has better performance than LDA, SDA and MSDA in most cases. Additional experiments have further shown that SSDA can project data into LDA space that has higher class separation than LDA, SDA and MSDA in most cases.

Subclass Graph Embedding and a Marginal Fisher Analysis paradigm

Subspace learning techniques have been extensively used for dimensionality reduction (DR) in many pattern classification problem domains. Recently, methods like Subclass Discriminant Analysis (SDA) and Clustering-based Discriminant Analysis (CDA), which use subclass information for the discrimination between the data classes, have attracted much attention. In parallel, important work has been accomplished on Graph Embedding (GE), which is a general framework unifying several subspace learning techniques. In this paper, GE has been extended in order to integrate subclass discriminant information resulting to the novel Subclass Graph Embedding (SGE) framework, which is the main contribution of our work. It is proven that SGE encapsulates a diversity of both supervised and unsupervised unimodal methods like Locality Preserving Projections (LPP), Principal Component Analysis (PCA) and Linear Discriminant Analysis (LDA). The theoretical link of SDA and CDA methods with SGE is also established. Along these lines, it is shown that SGE comprises a generalization of the typical GE framework including subclass DR methods. Moreover, it allows for an easy utilization of kernels for confronting non-linearly separable data. Employing SGE, in this paper a novel DR algorithm, which uses subclass discriminant information, called Subclass Marginal Fisher Analysis (SMFA) has been proposed. Through a series of experiments on various real-world datasets, it is shown that SMFA outperforms in most of the cases the state-of-the-art demonstrating the efficacy and power of SGE as a platform to develop new methods.

Improved SDA based on mixed weighted Mahalanobis distance

We propose an improved subclass discriminant analysis based on the mixed weighted Mahalanobis distance, with the aim of resolving the classification problem that samples are multi-subclass distributed and the computed vectors are sub-optimal. There are three contributions in this paper. First, we used some theoretical support to improve the traditional discriminant criterion function in subclass discriminant analysis. Second, we propose a new distance measure called the mixed weighted Mahalanobis distance (MWMD), which considers the influence of the sample size and sample scatter. Finally, inspired by approximate weighted linear discriminant analysis, we applied MWMD to the improved criterion function. Our experimental results confirmed that the proposed method has a better classification performance than other discriminant analysis methods, using an artificial dataset and some benchmark databases.

Structural max-margin discriminant analysis for feature extraction

Subclass discriminant analysis (SDA) is a recently developed dimensionality reduction technique which takes into consideration the intrinsic structure information lurking in data by approximating unknown distribution of each class with multiple Gaussian distributions, namely, subclasses. However, in SDA, the separability between heterogeneous subclasses, i.e. those from different classes, is measured by the between-subclass scatter calculated as average distance between the means of these subclasses. In this paper, in the view of maximum margin principle, we propose a novel feature extraction method coined structural max-margin discriminant analysis (SMDA), in order to enhance the performance of SDA. Specifically, SMDA targets at finding an orthogonal linear embedded subspace in which the margin, defined as the minimum pairwise between-subclasses distance, is maximized and simultaneously the within-subclasses scatter is minimized. The concrete formulation of the resulting model boils down to a nonconvex optimization problem that can be solved by combining the constrained concave–convex procedure with the column generation technique. We evaluate the proposed SMDA on several benchmark datasets and the experimental results confirm the effectiveness of the proposed method.

Robust nose tip localization based on two-stage subclass discriminant analysis

Nose is one of the salient features in a human face, and its localization is important for face recognition, face pose recognition, 3D face reconstruction, etc. In this paper, a novel nose tip localization method is proposed, which is based on two-stage subclass discriminant analysis (SDA). At the first stage, some randomly selected image patches are used as negative samples for the training of SDA classifier, and nose is detected from the whole face image. The second stage refines nose tip position by using some nose context patches as negative samples for the training of SDA classifier. The proposed method detects nose from the whole face image and no a priori knowledge about the layout of face components is used. Experimental results on AR images show that the proposed method can achieve high nose tip localization rates, and is robust to changes of illumination and facial expression.

Subclass Discriminant Analysis of morphological and textural features for HEp-2 staining pattern classification

Classifying HEp-2 fluorescence patterns in Indirect Immunofluorescence (IIF) HEp-2 cell imaging is important for the differential diagnosis of autoimmune diseases. The current technique, based on human visual inspection, is time-consuming, subjective and dependent on the operator's experience. Automating this process may be a solution to these limitations, making IIF faster and more reliable. This work proposes a classification approach based on Subclass Discriminant Analysis (SDA), a dimensionality reduction technique that provides an effective representation of the cells in the feature space, suitably coping with the high within-class variance typical of HEp-2 cell patterns. In order to generate an adequate characterization of the fluorescence patterns, we investigate the individual and combined contributions of several image attributes, showing that the integration of morphological, global and local textural features is the most suited for this purpose. The proposed approach provides an accuracy of the staining pattern classification of about 90%.

Enhanced iterative projection for subclass discriminant analysis under EM-alike framework

Mixture discriminant analysis (MDA) and subclass discriminant analysis (SDA) belong to the supervised classification approaches. They have advantage over the standard linear discriminant analysis (LDA) in large sample size problems, since both of them divide the samples in each class into subclasses which keep locality but LDA does not. However, since the current MDA and SDA algorithms perform subclass division in just one step in the original data space before solving the generalized eigenvalue problem, two problems are exposed: (1) they ignore the relation among classes since subclass division is performed in each isolated class; (2) they cannot guarantee good performance of classifiers in the transformed space, because locality in the original data space may not be kept in the transformed space. To address these problems, this paper presents a new approach for subclass division based on k-means clustering in the projected space, class by class using the iterative steps under EM-alike framework. Experiments are performed on the artificial data set, the UCI machine learning data sets, the CENPARMI handwritten numeral database, the NUST603 handwritten Chinese character database, and the terrain cover database. Extensive experimental results demonstrate the performance advantages of the proposed method.

Automatic prostate MR image segmentation with sparse label propagation and domain-specific manifold regularization.

Automatic prostate segmentation in MR images plays an important role in prostate cancer diagnosis. However, there are two main challenges: (1) Large inter-subject prostate shape variations; (2) Inhomogeneous prostate appearance. To address these challenges, we propose a new hierarchical prostate MR segmentation method, with the main contributions lying in the following aspects: First, the most salient features are learnt from atlases based on a subclass discriminant analysis (SDA) method, which aims to find a discriminant feature subspace by simultaneously maximizing the inter-class distance and minimizing the intra-class variations. The projected features, instead of only voxel-wise intensity, will be served as anatomical signature of each voxel. Second, based on the projected features, a new multi-atlases sparse label fusion framework is proposed to estimate the prostate likelihood of each voxel in the target image from the coarse level. Third, a domain-specific semi-supervised manifold regularization method is proposed to incorporate the most reliable patient-specific information identified by the prostate likelihood map to refine the segmentation result from the fine level. Our method is evaluated on a T2 weighted prostate MR image dataset consisting of 66 patients and compared with two state-of-the-art segmentation methods. Experimental results show that our method consistently achieves the highest segmentation accuracies than other methods under comparison.

Palmprint and Face Multi-Modal Biometric Recognition Based on SDA-GSVD and Its Kernelization

When extracting discriminative features from multimodal data, current methods rarely concern themselves with the data distribution. In this paper, we present an assumption that is consistent with the viewpoint of discrimination, that is, a person's overall biometric data should be regarded as one class in the input space, and his different biometric data can form different Gaussians distributions, i.e., different subclasses. Hence, we propose a novel multimodal feature extraction and recognition approach based on subclass discriminant analysis (SDA). Specifically, one person's different bio-data are treated as different subclasses of one class, and a transformed space is calculated, where the difference among subclasses belonging to different persons is maximized, and the difference within each subclass is minimized. Then, the obtained multimodal features are used for classification. Two solutions are presented to overcome the singularity problem encountered in calculation, which are using PCA preprocessing, and employing the generalized singular value decomposition (GSVD) technique, respectively. Further, we provide nonlinear extensions of SDA based multimodal feature extraction, that is, the feature fusion based on KPCA-SDA and KSDA-GSVD. In KPCA-SDA, we first apply Kernel PCA on each single modal before performing SDA. While in KSDA-GSVD, we directly perform Kernel SDA to fuse multimodal data by applying GSVD to avoid the singular problem. For simplicity two typical types of biometric data are considered in this paper, i.e., palmprint data and face data. Compared with several representative multimodal biometrics recognition methods, experimental results show that our approaches outperform related multimodal recognition methods and KSDA-GSVD achieves the best recognition performance.

Face feature extraction and recognition based on discriminant subclass-center manifold preserving projection

Manifold learning is an effective dimensional reduction technique for face feature extraction, which, generally speaking, tends to preserve the local neighborhood structures of given samples. However, neighbors of a sample often comprise more inter-class data than intra-class data, which is an undesirable effect for classification. In this paper, we address this problem by proposing a subclass-center based manifold preserving projection (SMPP) approach, which aims at preserving the local neighborhood structure of subclass-centers instead of given samples. We theoretically show from a probability perspective that, neighbors of a subclass-center would comprise of more intra-class data than inter-class data, and thus is more desirable for classification. In order to take full advantage of the class separability, we further propose the discriminant SMPP (DSMPP) approach, which incorporates the subclass discriminant analysis (SDA) technique to SMPP. In contrast to related discriminant manifold learning methods, DSMPP is formulated as a dual-objective optimization problem and we present analytical solution to it. Experimental results on the public AR, FERET and CAS-PEAL face databases demonstrate that the proposed approaches are more effective than related manifold learning and discriminant manifold learning methods in classification performance.