Binary Linear Discriminant Analysis Research Articles

Systemic Lupus Erythematosus: How Machine Learning Can Help Distinguish between Infections and Flares.

Systemic Lupus Erythematosus (SLE) is a multifaceted autoimmune ailment that impacts multiple bodily systems and manifests with varied clinical manifestations. Early detection is considered the most effective way to save patients' lives, but detecting severe SLE activity in its early stages is proving to be a formidable challenge. Consequently, this work advocates the use of Machine Learning (ML) algorithms for the diagnosis of SLE flares in the context of infections. In the pursuit of this research, the Random Forest (RF) method has been employed due to its performance attributes. With RF, our objective is to uncover patterns within the patient data. Multiple ML techniques have been scrutinized within this investigation. The proposed system exhibited around a 7.49% enhancement in accuracy when compared to k-Nearest Neighbors (KNN) algorithm. In contrast, the Support Vector Machine (SVM), Binary Linear Discriminant Analysis (BLDA), Decision Trees (DT) and Linear Regression (LR) methods demonstrated inferior performance, with respective values around 81%, 78%, 84% and 69%. It is noteworthy that the proposed method displayed a superior area under the curve (AUC) and balanced accuracy (both around 94%) in comparison to other ML approaches. These outcomes underscore the feasibility of crafting an automated diagnostic support method for SLE patients grounded in ML systems.

General sparse multi-class linear discriminant analysis

Discrimination with high dimensional data is often more effectively done with sparse methods that use a fraction of predictors rather than using all the available ones. In recent years, some effective sparse discrimination methods based on Fisher’s linear discriminant analysis (LDA) have been proposed for binary class problems. Extensions to multi-class problems are suggested in those works; however, they have some drawbacks such as the heavy computational cost for a large number of classes. We propose an approach to generalize a binary LDA solution into a multi-class solution while avoiding the limitations of the existing methods. Simulation studies with various settings, as well as real data examples including next generation sequencing data, confirm the effectiveness of the proposed approach.

Universum linear discriminant analysis

Universum learning has been used for classification and clustering, and obtains favourable improvements with the help of Universum – the samples that do not belong to either class of interest. In this reported work, universum learning is extended to dimensionality reduction by incorporating it with linear discriminant analysis (LDA). However, for the C-class problem, LDA can get at most C−1 projection directions due to the rank limitation. The C−1 directions are not enough for sufficient discrimination, which has motivated the adaption of the one-against-one trick to decompose the original C-class LDA into 0.5C(C−1) binary LDA ones for getting more directions. Uiniversum learning is then introduced to each binary LDA and the method is termed as universum linear discriminant analysis (ULDA). ULDA aims to find discriminant directions by maximising the distance between two target classes and simultaneously minimising the distance between the Universum and the mean of the target classes. The experiments on UCI datasets demonstrate the advantages and effectiveness of the ULDA.

The Validity of Three-Class Hotelling Trace (3-HT) in Describing Three-Class Task Performance: Comparison of Three-Class Volume Under ROC Surface (VUS) and 3-HT

In order to describe multiclass classification performance, several figures of merit (FOM) have been proposed. Among the earliest and most widely known of these is the three-class Hotelling trace (3-HT). The goal of this paper is to present theoretical and empirical data demonstrating the failure of 3-HT as a measure of three-class task performance. To help do this, we contrast it to a newly proposed three-class FOM, the volume under the three-class receiver operating characteristic (ROC) surface (VUS). The VUS is obtained from a decision theory based three-class ROC analysis method which has been proved to extend the decision theoretic, linear discriminant analysis (LDA), and psychophysical foundations of binary ROC analysis to a three-class paradigm. We demonstrate empirically that the VUS and 3-HT do not have a monotonic relationship in general when describing three-class task performance. Numerical experiments demonstrated that the VUS provided reasonable results, while the 3-HT failed to distinguish between the case where all objects could be perfectly classified from the case where only one pair of the classes could be perfectly classified. We have provided theoretical explanations of this failure of 3-HT. The significance of this work goes beyond merely demonstrating the problems of the 3-HT, it demonstrates that a FOM that is mathematically correct and has a strong theoretical basis can provide results that violate a common sense understanding of three-class task performance. This fact raises the question of "how to evaluate a classification performance evaluation method?" We believe the answer to this question lies in the theoretical foundations of binary ROC analysis. We have thus contrasted the two FOMs in terms of three fundamental theories underlying binary ROC analysis: decision theory, binary linear discriminant analysis, and the equivalence of two psychophysical classification procedures. These theoretical investigations demonstrated the importance of extending and unifying all the fundamental theories of binary classification in the development of a three-class FOM; violating one of theses fundamental binary classification theories may, as it did for the L-HT, provide predictions of three-class task performance that do not agree with a common sense understanding of three-class task performance.

A Novel Design of 4-Class BCI Using Two Binary Classifiers and Parallel Mental Tasks

A novel 4-class single-trial brain computer interface (BCI) based on two (rather than four or more) binary linear discriminant analysis (LDA) classifiers is proposed, which is called a “parallel BCI.” Unlike other BCIs where mental tasks are executed and classified in a serial way one after another, the parallel BCI uses properly designed parallel mental tasks that are executed on both sides of the subject body simultaneously, which is the main novelty of the BCI paradigm used in our experiments. Each of the two binary classifiers only classifies the mental tasks executed on one side of the subject body, and the results of the two binary classifiers are combined to give the result of the 4-class BCI. Data was recorded in experiments with both real movement and motor imagery in 3 able-bodied subjects. Artifacts were not detected or removed. Offline analysis has shown that, in some subjects, the parallel BCI can generate a higher accuracy than a conventional 4-class BCI, although both of them have used the same feature selection and classification algorithms.

Fast linear discriminant analysis using binary bases

Linear Discriminant Analysis (LDA) is a widely used technique for pattern classification. It seeks the linear projection of the data to a low dimensional subspace where the data features can be modelled with maximal discriminative power. The main computation in LDA is the dot product between LDA base vector and the data point which involves costly element-wise floating point multiplications. In this paper, we present a fast linear discriminant analysis method called binary LDA (B-LDA), which possesses the desirable property that the subspace projection operation can be computed very efficiently. We investigate the LDA guided non-orthogonal binary subspace method to find the binary LDA bases, each of which is a linear combination of a small number of Haar-like box functions. We also show that B-LDA base vectors are nearly orthogonal to each other. As a result, in the non-orthogonal vector decomposition process, the computationally intensive pseudo-inverse projection operator can be approximated by the direct dot product without causing significant distance distortion. This direct dot product projection can be computed as a linear combination of the dot products with a small number of Haar-like box functions which can be efficiently evaluated using the integral image. The proposed approach is applied to face recognition on ORL and FERET dataset. Experiments show that the discriminative power of binary LDA is preserved and the projection computation is significantly reduced.