Discriminative Dimensionality Reduction Research Articles

Utilizing unsupervised machine learning to uncover novel phenogroups within the broad QRS complex

Abstract Background Conduction disease can manifest as a broad QRS complex on the ECG. Broad QRS is conventionally categorised as left bundle branch block (LBBB) and right bundle branch block (RBBB) or non-specific intraventricular conduction delay (IVCD), based on QRS morphology. These subgroups were coined over a century ago and have been relevant for heart failure with reduced ejection fraction and cardiac resynchronization therapy (CRT). However, there may be more precise phenogroups underlying broad QRS complexes. Purpose Using unsupervised machine learning to define novel broad QRS phenogroups and characterise them using clinical variables and disease outcomes. Methods We extracted 51 relevant features using a Variational Autoencoder from 47,456 median-beat ECGs with broad QRS intervals (QRS ≥ 130ms) from a secondary care cohort. These were input into Discriminative Dimensionality Reduction via Learning a Tree (DDRTree), a clustering method that models the continuum of heterogeneity by projecting the underlying distribution as a tree structure in a low-dimensional space, while assigning distinct branch/cluster labels. Results Six distinct phenogroups were identified within broad QRS morphologies (Figure-1). Two branches were RBBB dominant (Branch 1: more males, high-comorbidity; Branch 6: low-comorbidity/healthy-RBBB). Two branches were LBBB dominant (Branch 4: mixed IVCD with CHB burden; Branch 5: more females, CRT responders). Two branches were IVCD/mixed-phenotype (Branch 2: IVCD-dominant, high risk of future cardiovascular events; Branch 3: average/core branch). Within LBBB, Branch 4 and 5 showed similar trends with improving echocardiography profiles and CRT response when adjusted for covariates, each compared against the core branch. Branch 5 was distinct due its high risk of future HF (HR 1.19 [1.07-1.33] p < 0.01), cardiovascular death (HR 1.35 [1.20-1.50] p < 0.0001), all-cause mortality (HR 1.08 [1.01-1.16] p < 0.05) and CHB (HR 1.35 [1.16-1.56] p < 0.0001) (Figure-2). Conversely within RBBB, Branch 1 and 6 showed stark differences across various traits. Branch 1 captured a high-risk profile associated to higher risk of cardiovascular death (HR 1.37 [1.24-1.52] p < 0.0001), all-cause mortality (HR 1.20 [1.12-1.29] p < 0.0001) and CHB (HR 1.15 [1.01-1.32] p < 0.05) (Figure-2), while Branch 6 retained a low-risk phenotype. The tree dimensions also revealed relevant trends in cardiac anatomy and valvular dysfunction; the top-left region in Figure-1 associated with higher left ventricular ejection fraction and lower left ventricular end-diastolic dimension, lower left ventricular end-systolic dimension and a greater CRT response. Conclusion Through unsupervised methods we have identified conduction disease phenogroups with additive value for disease prognosis and CRT response prediction beyond traditional LBBB and RBBB labels. These novel phenogroups may help guide precision care for patients with a broad QRS complex.Figure-1Figure-2

Heterogeneity of glycaemic phenotypes in type 1 diabetes

Aims/hypothesisOur study aims to uncover glycaemic phenotype heterogeneity in type 1 diabetes.MethodsIn the Study of the French-speaking Society of Type 1 Diabetes (SFDT1), we characterised glycaemic heterogeneity thanks to a set of complementary metrics: HbA1c, time in range (TIR), time below range (TBR), CV, Gold score and glycaemia risk index (GRI). Applying the Discriminative Dimensionality Reduction with Trees (DDRTree) algorithm, we created a phenotypic tree, i.e. a 2D visual mapping. We also carried out a clustering analysis for comparison.ResultsWe included 618 participants with type 1 diabetes (52.9% men, mean age 40.6 years [SD 14.1]). Our phenotypic tree identified seven glycaemic phenotypes. The 2D phenotypic tree comprised a main branch in the proximal region and glycaemic phenotypes in the distal areas. Dimension 1, the horizontal dimension, was positively associated with GRI (coefficient [95% CI]) (0.54 [0.52, 0.57]), HbA1c (0.39 [0.35, 0.42]), CV (0.24 [0.19, 0.28]) and TBR (0.11 [0.06, 0.15]), and negatively with TIR (−0.52 [−0.54, −0.49]). The vertical dimension was positively associated with TBR (0.41 [0.38, 0.44]), CV (0.40 [0.37, 0.43]), TIR (0.16 [0.12, 0.20]), Gold score (0.10 [0.06, 0.15]) and GRI (0.06 [0.02, 0.11]), and negatively with HbA1c (−0.21 [−0.25, −0.17]). Notably, socioeconomic factors, cardiovascular risk indicators, retinopathy and treatment strategy were significant determinants of glycaemic phenotype diversity. The phenotypic tree enabled more granularity than traditional clustering in revealing clinically relevant subgroups of people with type 1 diabetes.Conclusions/interpretationOur study advances the current understanding of the complex glycaemic profile in people with type 1 diabetes and suggests that strategies based on isolated glycaemic metrics might not capture the complexity of the glycaemic phenotypes in real life. Relying on these phenotypes could improve patient stratification in type 1 diabetes care and personalise disease management.Graphical

Nonparametric locality reforming projection for linear discriminative dimensionality reduction

Discriminative dimensionality reduction (DDR) makes significant sense in high-dimensional data analysis. Over the decades, research has demonstrated that modeling the intrinsic local structure can well represent the data. Using pairwise neighboring data points to represent the between-class and within-class differences, local structure-based DDR methods are effective. However, how to set the parameters, e.g., the parameter of k-nearest neighbors, is still an open question, which makes the algorithms perform unstably. To address this issue, we propose a nonparametric local structure-based DDR method, named nonparametric locality reforming projection (NLRP). Also, we propose an effective analytical method for effectiveness analysis in both cases of sufficient and insufficient training samples for DDR methods. The experiment results on the AR, Carnegie Mellon University pose, illumination, and expression, and ORL databases are consistent with the theoretical analysis and verified the effectiveness of the proposed NLRP method. Especially, NLRP performs well in the case of very insufficient samples.

Building Outline Delineation From VHR Remote Sensing Images Using the Convolutional Recurrent Neural Network Embedded With Line Segment Information

Recently, several recurrent neural network (RNN)-based models have been proposed to delineate the outlines of buildings from very high resolution (VHR) remote sensing images. These models first use convolutional neural networks (CNNs) to recognize the boundary fragments by learning probability maps of both edges and corners and then feed them into RNN to find and link a set of sequent corners into external boundaries of buildings. However, caused by the category imbalance of edges and corners, the local ambiguity of edge detection is very serious, which significantly affects the accuracy of predicted outline corners. To tackle this challenge, this article introduces a convolutional RNN embedded with line segment information (LSI-RNN), a novel network that aims to directly detect line segment instead of edges. To achieve this, LSI-RNN utilizes an additional cotraining branch to generate an attraction field map (AFM) by neural discriminative dimensionality reduction (NDDR) layer. Consequently, the conventional classification problem of edges is converted to a regression problem of line segments, thus solving the aforementioned issues. Experimental results over three remote sensing datasets with different spatial resolutions show that the proposed method consistently outperforms other state-of-the-art methods.

Large Margin Graph Embedding-Based Discriminant Dimensionality Reduction

Discriminant graph embedding-based dimensionality reduction methods have attracted more and more attention over the past few decades. These methods construct an intrinsic graph and penalty graph to preserve the intrinsic geometry structures of intraclass samples and separate the interclass samples. However, the marginal samples cannot be accurately characterized only by penalty graphs since they treat every sample equally. In practice, these marginal samples often influence the classification performance, which needs to be specially tackled. In this study, the near neighbors’ hypothesis margin of marginal samples has been further maximized to separate the interclass samples and improve the discriminant ability by integrating intrinsic graph and penalty graph. A novel discriminant dimensionality reduction named LMGE-DDR has been proposed. Several experiments on public datasets have been conducted to verify the effectiveness of the proposed LMGE-DDR such as ORL, Yale, UMIST, FERET, CMIU-PIE09, and AR. LMGE-DDR performs better than other compared methods, and the corresponding standard deviation of LMGE-DDR is smaller than others. This demonstrates that the evaluation method verifies the effectiveness of the introduced method.

Theoretical framework in graph embedding-based discriminant dimensionality reduction

• A theoretical framework considering hypothesis-margin is derived to guide the selection of weight functions. • a more effective supervised discriminant graph embedding-based dimensionality reduction method is introduced. Graph embedding-based discriminative dimensionality reduction has remained to be a popular research topic over the past few decades. The weight functions in adjacent graphs are the key to the performance of methods. In practice, the weight functions are always achieved experimentally. Thus far, the selection of effective weight functions has no any theoretical guidance. A theoretical framework considering hypothesis-margin is derived in this study to guide the selection of weight functions, whose truth is verified in a popular algorithm and a more effective supervised discriminant graph embedding-based dimensionality reduction method is introduced. Many experimental results demonstrate the truth of the proposed theoretical framework and the effectiveness of the introduced method. Importantly, the proposed framework can provide theoretical support for the selection of weight functions in the graph embedding-based dimensionality reduction.

Kernel alignment unsupervised discriminative dimensionality reduction

The performance of most existing adaptive graph learning methods, which adjust data similarity matrix according to the data representation, depends on the hypotheses that the data representation is a good indicator of the underlying data structure. However, this hypothesis is not always applicable when dealing with high dimensional data. In this paper, we propose a novel kernel alignment unsupervised discriminative dimensionality reduction (KaUDDR) algorithm. By integrating adaptive graph learning and feature learning into a joint learning framework, graph construction and dimensionality reduction are conducted simultaneously to guarantee the optimality of graph for feature learning in the proposed algorithm. Data kernel and similarity indicator kernel are defined by learned graph and the projected data in a low-dimensional subspace, a compact and discriminative data representation in the projected subspace is obtained by means of kernel alignment to explore the consistency between the projected data kernel and similarity indicator kernel. Experimental results on dimensionality reduction as well as clustering show that our method consistently outperforms the related unsupervised dimensionality reduction algorithm.

Detection of Black Spot of Rose Based on Hyperspectral Imaging and Convolutional Neural Network

Black spot is one of the seriously damaging plant diseases in China, especially in rose production. Hyperspectral technology reflects both external features and internal structure information of measured samples, which can be used to identify the disease. In this research, both the spectral and image features of two infected roses with black spot were used to train a convolutional neural network (CNN) model. Multiple scattering correction (MSC) and standard normal variable (SNV) methods were applied to preprocess the spectral data. Cropping, median filtering and binarization were pretreatments used on the hyperspectral images. Three CNN models based on Alexnet, VGG16 and neural discriminative dimensionality reduction (NDDR) were evaluated by analyzing the classification accuracy and loss function. The results show that the CNN model based on the fusion of features has higher accuracy. The highest accuracies of detection of blackspot in different roses are 12–26 (100%) and 13–54 (99.95%), applying the NDDR-CNN model. Therefore, this research indicates that the spectral analysis based on CNN can detect black spot of roses, which provides a reference for the detection of other plant diseases, and has favorable research significance as well as prospect for development.

Representative null space LDA for discriminative dimensionality reduction

Null space Linear Discriminant Analysis (NLDA) was proposed twenty years ago to overcome the singularity problem of LDA in practical applications. With two decades of technique development, many Discriminative Dimensionality Reduction (DDR) methods that outperform NLDA have been proposed. This paper provides new insight into NLDA and illustrates that NLDA is much more powerful after solving its inherent problem. The main problem of NLDA is the intrinsic overfitting problem. An ideal NLDA model is proposed to analyze its overfitting problem. Based on the ideal NLDA model, a more reasonable Representative NLDA (RNLDA) method is proposed to prevent overfitting. Two simple but efficient RNLDA algorithms are proposed to implement the RNLDA method with a theoretical proof. This study theoretically analyzed and indicated that applying the classical but simple hold-out pretraining method can automatically set the only parameter to achieve high performance. Extensive experiments with eight databases demonstrate the superior performance of the RNLDA method over state-of-the-art DDR methods.

Supervised discriminative dimensionality reduction by learning multiple transformation operators

Analyzing and learning from high dimensional data have always been challenging in machine learning tasks, causing serious computational complexities and poor learning performances. Supervised dimensionality reduction is a popular technique to address such challenges in supervised learning tasks, where data are accompanied with labels. Traditionally, such techniques mostly learn one single transformation to project data into a low-dimensional discriminative subspace. However, learning only one transformation for the whole data could be dominated by one or several classes, and the rest of classes receive less discrimination in the reduced space. That is to say, learning one transformation is insufficient to properly discriminate classes of data in the reduced space because they may have complex and completely dissimilar distributions. This insufficiency becomes even more serious if the number of classes increases, leading to poor discrimination and lessening the learning performance in the reduced space. To overcome this limitation, we propose a novel supervised dimensionality reduction method, which learns per-class transformations by optimizing a newly designed and efficient objective function. The proposed method captures more discriminative information from each single class of data compared to the case of one single transformation. Moreover, the proposed objective function enjoys several desirable properties: (1) maximizing margins between the transformed classes of data, (2) having a closed-form solution, (3) being easily kernelized in the case of nonlinear data,(4) preventing overfitting, and (5) ensuring the transformations are sparse in rows so that discriminative features are learned in the reduced space. Experimental results verify that the proposed method is superior to the related state-of-the-art methods and promising in generating discriminative embeddings.

Device-free passive wireless localization system with weighted transferable discriminative dimensionality reduction method

Device-free passive wireless indoor localization has attracted great interest due to the widespread deployment of Wi-Fi devices and the rapid growth in demand for location-based services. In this paper, we propose a novel device-free passive wireless localization system with a weighted transferable discriminative dimensionality reduction method (termed TLLOC). It utilizes the channel state information (CSI) extracted from a single link to estimate the location of the target, neither requiring the target to wear any electronic equipment nor deploying a large number of APs and Monitor Devices. To cope with the problem of reduced localization accuracy caused by the unpredictable nature of CSI over time that ignored by most previous CSI-based localization works, a novel weighted transferable discriminative dimensionality reduction (termed WTR) method combining transfer learning and dimensionality reduction is proposed. The WTR method constructs a low-dimensional latent space, which can simultaneously improve the discrimination of training samples and narrow the distribution divergence between the training samples and the test samples, further enhancing the performance of our system. Experimental results are presented to confirm that TLLOC can effectively improve localization accuracy while saving a great amount of the calibration cost, compared with the other existing methods in a representative indoor environment.

Speech emotion recognition using hybrid spectral-prosodic features of speech signal/glottal waveform, metaheuristic-based dimensionality reduction, and Gaussian elliptical basis function network classifier

Abstract In this paper, a hybrid system consisting of three stages of feature extraction, dimensionality reduction, and feature classification is proposed for speech emotion recognition (SER). At feature extraction stage, an informationally-rich spectral-prosodic hybrid feature vector comprised of perceptual-spectral features; that is, mel-frequency cepstral coefficient (MFCC), perceptual linear prediction coefficient (PLPC), and perceptual minimum variance distortionless response (PMVDR) coefficient along with the prosodic feature of pitch (i.e. F0) are extracted for each frame. This feature vector is extracted from both speech signal and its glottal-waveform. The first and the second-order derivatives are then added to the above-mentioned vector to form a high-dimensional hybrid feature vector characterized by a large number of dimensions. At the next stage, i.e. dimensionality reduction, the dimensionality of this feature vector is reduced using a new proposed quantum-behaved particle swarm optimization (QPSO)-based approach. In this paper, a new QPSO algorithm (so-called, pQPSO) is presented that makes use of a truncated Laplace distribution (TLD) to generate new particles and thus to produce solutions (i.e. particles) that are all within a valid range of a problem (contrary to the standard QPSO). The contraction-expansion (CE) factor of the proposed pQPSO is also selected adaptively. Using the proposed QPSO algorithm, an optimal discriminative dimensionality reduction matrix (i.e. projection matrix) is estimated with emotion classification accuracy as a class-discriminative criterion. At the subsequent stage, vectors with reduced feature dimensionality are fed into a Gaussian elliptical basis function (GEBF)-type neural network classifier to detect their speech emotion. To accelerate the training phase of the GEBF classifier, a fast-scaled conjugate gradient (SCG) algorithm is correspondingly employed that does not need to adjust the learning rate. Finally, the proposed method is evaluated on three standard emotional speech databases of Berlin Database of Emotional Speech (EMODB), Surrey Audio-Visual Expressed Emotion (SAVEE), and Interactive Emotional Dyadic Motion Capture (IEMOCAP). The experimental results showed that the proposed method was more accurate than state-of-the-art ones in terms of detecting speech emotions.

Discriminative dimensionality reduction for sensor drift compensation in electronic nose: A robust, low-rank, and sparse representation method

Abstract Sensor drift, which is a critical issue in the field of sensor measurements, has plagued the sensor community in the past several decades. How to tackle the sensor drift problem using expert and intelligent systems has gained increasing attention. Most sensor drift compensation methods ignore the sparse and low-rank characteristics of sensor signals. In this paper, we propose a discriminative dimensionality reduction method for sensor drift compensation in the electronic nose. The proposed method consists of four major components. (1) The distribution discrepancy between source data and target data is alleviated via projecting all the data into a common subspace. (2) A sparse and low-rank reconstruction coefficient matrix is employed to preserve the global and local structures of sensor signals. (3) An error matrix is introduced to deal with outliers. (4) The source data label information is taken into consideration to avoid overlapping of samples with different labels in the common subspace. The formulated minimization problem with constraints can be solved in an iterative manner. The effectiveness of the proposed method has been verified by conducting experiments on two sensor drift datasets. The proposed method may provide new insights into the gas sensor drift compensation systems or other expert and intelligent systems.

Speech emotion recognition using discriminative dimension reduction by employing a modified quantum-behaved particle swarm optimization algorithm

In recent years, Speech Emotion Recognition (SER) has received considerable attention in affective computing field. In this paper, an improved system for SER is proposed. In the feature extraction step, a hybrid high-dimensional rich feature vector is extracted from both speech signal and glottal-waveform signal using techniques such as MFCC, PLPC, and MVDR. The prosodic features derived from fundamental frequency (f0) contour are also added to this feature vector. The proposed system is based on a holistic approach that employs a modified quantum-behaved particle swarm optimization (QPSO) algorithm (called pQPSO) to estimate both the optimal projection matrix for feature-vector dimension reduction and Gaussian Mixture Model (GMM) classifier parameters. Since the problem parameters are in a limited range and the standard QPSO algorithm performs a search in an infinite range, in this paper, the QPSO is modified in such a way that it uses a truncated probability distribution and makes the search more efficient. The system works in real-time and is evaluated on three standard emotional speech databases Berlin database of emotional speech (EMO-DB), Surrey Audio-Visual Expressed Emotion (SAVEE) and Interactive Emotional Dyadic Motion Capture (IEMOCAP). The proposed method improves the accuracy of the SER system compared to classical methods such as FA, PCA, PPCA, LDA, standard QPSO, wQPSO, and deep neural network, and also outperforms many state-of-the-art recent approaches that use the same datasets.

Integration of Cancer Genomics Data for Tree-based Dimensionality Reduction and Cancer Outcome Prediction.

Accurate outcome prediction is crucial for precision medicine and personalized treatment of cancer. Researchers have found that multi-dimensional cancer omics studies outperform each data type (mRNA, microRNA, methylation or somatic copy number alteration) study in human disease research. Existing methods leveraging multiple level of molecular data often suffer from various limitations, e. g., heterogeneity, poor robustness or loss of generality. To overcome these limitations, we presented the tree-based dimensionality reduction approach for the identification of smooth tree graph and developed accurate predictive model for clinical outcome prediction. We demonstrated that 1) Discriminative Dimensionality Reduction via learning a Tree (DDRTree) achieved reduced dimension space tree with statistical significance; 2) Tree based support vector machine (SVM) classifier improved prediction performance of cancer recurrence as compared to t-test based SVM classifier; 3) Tree based SVM classifier was robust with regard to the different number of multi-markers; 4) Combining multiple omics data improved prediction performance of cancer recurrence as compared to a single-omics data; and 5) Tree based SVM classifier achieved similar or better prediction performance when compared to the features from state-of-the-art feature selection methods. Our results demonstrated great potential of the tree-based dimensionality reduction approach based clinical outcome prediction.

Fault diagnosis based on variable-weighted separability-oriented subclass discriminant analysis

Abstract Fisher discriminant analysis (FDA) is a classical method for discriminative dimensionality reduction, which has been extensively studied in the pattern classification field and widely used for fault diagnosis in industrial process monitoring. The classification performance of original FDA may degrade in overlapping, non-Gaussian or nonlinear data, and may be unable to extract sufficient features to interpret the data when the number of classes is small. To overcome these limitations, this paper proposes a combined variable weighting (VW) and separability-oriented subclass discriminant analysis (SSDA) scheme (denoted as VW-SSDA) for fault diagnosis. VW-SSDA utilizes K-means clustering to divide a class into subclasses using a separability oriented criterion, and all fault data are weighted by the corresponding weight vectors before applying FDA optimisation using re-defined scatter matrices. The proposed approach is applied to the Tennessee Eastman process. The results demonstrate that VW-SSDA shows better fault diagnosis performance than the several improved variants of the conventional FDA.

Discriminative Dimension Reduction via Maximin Separation Probability Analysis.

In this paper, we propose a novel discriminative dimension reduction (DR) method, maximin separation probability analysis (MSPA), which maximizes the minimum separation probability of all classes in the reduced low-dimensional subspace. Separation probability is a novel class separability measure, which gives a lower bound of the generalization accuracy for a learned linear classifier in a binary classification problem. The proposed MSPA duly considers the separation of all class pairs in multiclass linear discriminant analysis (LDA) and thus improves the subsequent classification performance. DR via MSPA leads to a nonconvex optimization problem. We develop an algorithm to solve the problem and the global optimal solution can be found by converting the original problem into a series of second-order cone programming problems. A low-computational cost extension and a non-LDA with kernel mapping of MSPA are also provided in this paper. The experimental results on 14 real-world datasets show our methods are superior to other state-of-the-art algorithms in discriminative DR tasks.

Sparsity and Geometry Preserving Graph Embedding for Dimensionality Reduction

Graph embedding is a very useful dimensionality reduction technique in pattern recognition. In this paper, we develop a novel discriminative dimensionality reduction technique entitled sparsity and geometry preserving graph embedding (SGPGE). SGPGE can not only capture the sparse reconstructive relationships among training samples but also discover the intrinsic geometry and latent discrimination from high-dimensional data. In SGPGE, the novel ways of constructing global and local adjacent graphs are developed. The built graphs with discriminant and geometrical information are more informative in graph embedding. Integrating the sparse reconstruction with the designed local and global adjacent graph constructions, SGPGE newly characterizes with-class sparsity and geometry preserving scatter, between-class sparsity and geometry preserving scatter and local sparsity and geometry preserving scatter to formulate the objective function of dimensionality reduction, and learns the discriminative transformation matrix using maximum margin criterion. Moreover, we also propose three variants of SGPGE. Experimental results on six public face datasets have demonstrated that the proposed methods are effective dimensionality reduction techniques with very good performance for classification in the embedding subspace.

Cross-subject mental workload classification using kernel spectral regression and transfer learning techniques

Due to the poor generalizability of the subject-specific mental workload (MWL) classifier, we propose a cross-subject MWL recognition framework in this paper. Firstly, we use fuzzy mutual information-based wavelet-packet transform (FMI-WPT) technique to extract the salient physiological features of the MWL. Then, we combine kernel spectral regression (KSR) and transferable discriminative dimensionality reduction (TDDR) methods to reduce the dimensionality of the feature vector and to transfer the classifier model across subjects. Finally, the measured data analysis results are presented to show the enhanced performance of the proposed framework for multi-class MWL recognition.

DiReT: An effective discriminative dimensionality reduction approach for multi-source transfer learning

Transfer learning is a well-known solution to the problem of domain shift in which source domain (training set) and target domain (test set) are drawn from di fferent distributions. In the absence of domain shift, discriminative dimensionality reduction approaches could classify target data with acceptable accuracy. However, distribution diff erence across source and target domains degrades the performance of dimensionalityreduction methods. In this paper, we propose a Discriminative Dimensionality Reduction approach for multi-source Transfer learning, DiReT, in which discrimination is exploited on transferred data. DiReT nds an embedded space, such that the distribution di erenceof the source and target domains is minimized. Moreover, DiReT employs multiple sourcedomains and semi-supervised target domain to transfer knowledge from multiple resources,and it also bridges across source and target domains to nd common knowledge in anembedded space. Empirical evidence of real and arti cial datasets indicates that DiReTmanages to improve substantially over dimensionality reduction approaches.