Discriminative Subspace Research Articles (Page 1)

Analysis–synthesis dictionary learning, which jointly learns synthesis and analysis dictionaries, is widely used in image classification. However, the performance of their algorithms remains unsatisfactory, primarily because the salient features in the samples are not effectively captured and the data contain redundant information. To address such problems, we propose an image classification algorithm based on sample projection jointly with incoherent robust adaptive dictionary pair learning. In this paper, we first project the samples into a discriminative subspace via least squares regression. Second, we integrate the joint learning of coefficients and salient features, together with their constraints, into a single model for training. Later, we impose an [Formula: see text]-norm constraint on the analysis dictionary and introduce an analysis incoherence promoting function to constrain the synthesis dictionary. Finally, we incorporate adaptive reconstruction weight learning to preserve the local structure of the training samples. We evaluated our method on the AR, Extended Yale B, ORL, CMU PIE, GT and COIL 20 datasets, achieving recognition rates of 99.67%, 98.41%, 96.50%, 98.73%, 78.50% and 99.87%, respectively.

Background/Objectives: An ASD diagnosis from EEG is challenging due to non-stationary, low-SNR signals and small cohorts. We propose a compact, interpretable pipeline that pairs a shift-invariant Stationary Wavelet Transform (SWT) with Fisher’s Linear Discriminant (FLDA) as a supervised projection method, delivering band-level insight and subject-wise evaluation suitable for resource-constrained clinics. Methods: EEG from the KAU dataset (eight ASD, eight controls; 256 Hz) was decomposed with SWT (db4). We retained levels 3, 4, and 6 (γ/β/θ) as features. FLDA learned a low-dimensional discriminant subspace, followed by a linear decision rule. Evaluation was conducted using a subject-wise 70/30 split (no subject overlap) with accuracy, precision, recall, F1, and confusion matrices. Results: The β band (Level 4) achieved the best performance (accuracy/precision/recall/F1 = 0.95), followed by γ (0.92) and θ (0.85). Despite partial overlap in FLDA scores, the projection maximized between-class separation relative to within-class variance, yielding robust linear decisions. Conclusions: Unlike earlier FLDA-only pipelines and wavelet–entropy–ANN approaches, our study (1) employs SWT (undecimated, shift-invariant) rather than DWT to stabilize sub-band features on short resting segments, (2) uses FLDA as a supervised projection to mitigate small-sample covariance pathologies before classification, (3) provides band-specific discriminative insight (β > γ/θ) under a subject-wise protocol, and (4) targets low-compute deployment. These choices yield a reproducible baseline with competitive accuracy and clear clinical interpretability. Future work will benchmark kernel/regularized discriminants and lightweight deep models as cohort size and compute permit.

Intraoperative margin assessment is crucial to ensure complete tumor removal and minimize the risk of cancer recurrence during breast-conserving surgery. The Intelligent Knife (iKnife), a mass spectrometry device that analyzes surgical smoke, shows promise in near-real-time margin evaluation. However, current AI models depend on labeled ex-vivo datasets, which are costly and time-consuming to produce. This research explores the potential of machine learning anomaly detection models to reduce reliance on labeled ex-vivo datasets by utilizing unlabeled intraoperative spectra. iKnife spectra were collected intraoperatively from 15 breast cancer surgeries. Ex-vivo samples were recorded from the resected specimen by a pathologist. Healthy samples were from the margin, and tumor samples were from the cross-section. We trained four anomaly detection methods, Isolation Forest (iForest), One Class Principal Component Analysis (OCPCA), Generalized One Class Discriminative Subspaces (GODS), and its Kernelized extension (KGODS), under two strategies: (i) intraoperative data only and (ii) intraoperative data plus healthy ex-vivo data. Performance was evaluated via four-fold cross-validation on labeled ex-vivo samples, with an additional ensemble approach on a held-out set. We compared the models to benchmark supervised classifiers and explored intraoperative feasibility with a retrospective case. Using intraoperative data alone, the average balanced accuracies were 70% (iForest), 81% (OC-PCA), 77% (GODS), and 81% (KGODS) during four-fold cross-validation. Adding healthy ex-vivo data improved performance across all models; however, OC-PCA remained competitive without ex-vivo labels. On the held-out set, OC-PCA trained only on intraoperative data achieved 81% balanced accuracy, 90% sensitivity, and 72% specificity. OC-PCA was selected for intraoperative feasibility and correctly detected the tumor breach with one false positive. Anomaly detection models, particularly OC-PCA, can identify positive breast cancer margins with no labeled ex-vivo data. Though slightly lower in performance than supervised classifiers, they offer a promising low-resource alternative for intraoperative label generation and semi-supervised training, which can enhance clinical deployment.

Enhanced finger-knuckle identification using discriminative color subspace segmentation and binarized statistical image feature extraction

Gesture recognition based on conventional machine learning is the main control approach for advanced prosthetic hand systems. Its primary limitation is the need for feature extraction, which must meet real-time control requirements. On the other hand, deep learning models could potentially overfit when trained on small datasets. For these reasons, we propose a hybrid Linear Discriminant Analysis–convolutional neural network (LDA-CNN) framework to improve the gesture recognition performance of sEMG-based prosthetic hand control systems. Within this framework, 1D-CNN filters are trained to generate latent representation that closely approximates Fisher’s (LDA’s) discriminant subspace, constructed from handcrafted features. Under the train-one-test-all evaluation scheme, our proposed hybrid framework consistently outperformed the 1D-CNN trained with cross-entropy loss only, showing improvements from 4% to 11% across two public datasets featuring hand gestures recorded under various limb positions and arm muscle contraction levels. Furthermore, our framework exhibited advantages in terms of induced spectral regularization, which led to a state-of-the-art recognition error of 22.79% with the extended 23 feature set when tested on the multi-limb position dataset. The main novelty of our hybrid framework is that it decouples feature extraction in regard to the inference time, enabling the future incorporation of a more extensive set of features, while keeping the inference computation time minimal.

Subspace learning and Support Vector Machine (SVM) are two critical techniques in pattern recognition, playing pivotal roles in feature extraction and classification. However, how to learn the optimal subspace such that the SVM classifier can perform the best is still a challenging problem due to the difficulty in optimization, computation, and algorithm convergence. To address these problems, this paper develops a novel method named Optimal Discriminant Support Vector Machine (ODSVM), which integrates support vector classification with discriminative subspace learning in a seamless framework. As a result, the most discriminative subspace and the corresponding optimal SVM are obtained simultaneously to pursue the best classification performance. The efficient optimization framework is designed for binary and multi-class ODSVM. Moreover, a fast sequential minimization optimization (SMO) algorithm with pruning is proposed to accelerate the computation in multi-class ODSVM. Unlike other related methods, ODSVM has a strong theoretical guarantee of global convergence, highlighting its superiority and stability. Numerical experiments are conducted on thirteen datasets and the results demonstrate that ODSVM outperforms existing methods with statistical significance.

Accurate classification of electroencephalogram (EEG) signals is challenging given the nonlinear and nonstationary nature of the data as well as subject-dependent variations. Graph signal processing (GSP) has shown promising results in the analysis of brain imaging data. In this article, a GSP-based approach is presented that exploits instantaneous amplitude and phase coupling between EEG time series to decode motor imagery (MI) tasks. A graph spectral representation of the Hilbert-transformed EEG signals is obtained, in which simultaneous diagonalization of covariance matrices provides the basis of a subspace that differentiates two classes of right hand and right foot MI tasks. To determine the most discriminative subspace, an exploratory analysis was conducted in the spectral domain of the graphs by ranking the graph frequency components using a feature selection method. The selected features are fed into a binary support vector machine that predicts the label of the test trials. The performance of the proposed approach was evaluated on brain-computer interface competition III (IVa) dataset. Experimental results reflect that brain functional connectivity graphs derived using the instantaneous amplitude and phase of the EEG signals show comparable performance with the best results reported on these data in the literature, indicating the efficiency of the proposed method compared to the state-of-the-art methods.

Abstract Electroencephalogram (EEG) signals processing has gathered increased interest from the scientific community for a long time, especially in the field of emotion recognition. The objective of this research is to propose a method that analyzes EEG signals from a publicly available dataset called SJTU Emotion EEG Dataset (SEED) to classify different emotional states, especially positive emotions. The dataset consists of EEG signals recorded from fifteen subjects while watching different film clips, each of which corresponds to a type of emotions. Initial analysis involves the extraction of features from the pre-processed data including time-domain analysis, frequency-domain analysis using Fast Fourier Transform (FFT) and nonlinear dynamics method. Machine learning algorithms are then applied as classifiers in order to determine the emotional states of each subject – categorized as positive, negative, or neutral – with the input is the extracted features. In this work, Support Vector Machine (SVM), Linear Discriminant Analysis (LDA) and Subspace Discriminant (SD) are used for training and testing the labeled data. The classification step achieved the highest accuracy level of 82.2% while the highest micro-F1 score value is 0.92 for positive emotions. The established methodology not only proposes an automated model for determining positive emotional states but also plays an important role at the beginning of the investigation further into emotion recognition by providing better visualization of how different emotional states are determined.

In this article, we propose a new unsupervised feature selection method named pseudo-label guided structural discriminative subspace learning (PSDSL). Unlike the previous methods that perform the two stages independently, it introduces the construction of probability graph into the feature selection learning process as a unified general framework, and therefore the probability graph can be learned adaptively. Moreover, we design a pseudo-label guided learning mechanism, and combine the graph-based method and the idea of maximizing the between-class scatter matrix with the trace ratio to construct an objective function that can improve the discrimination of the selected features. Besides, the main existing strategies of selecting features are to employ -norm for feature selection, but this faces the challenges of sparsity limitations and parameter tuning. For addressing this issue, we employ the -norm constraint on the learned subspace to ensure the row sparsity of the model and make the selected feature more stable. Effective optimization strategy is given to solve such NP-hard problem with the determination of parameters and complexity analysis in theory. Ultimately, extensive experiments conducted on nine real-world datasets and three biological ScRNA-seq genes datasets verify the effectiveness of the proposed method on the data clustering downstream task.

Sorting spikes from extracellular recordings, obtained by sensing neuronal activity around an electrode tip, is essential for unravelling the complexities of neural coding and its implications across diverse neuroscientific disciplines. However, the presence of overlapping spikes, originating from neurons firing simultaneously or within a short delay, has been overlooked because of the difficulty in identifying individual neurons due to the lack of ground truth. In this study, we propose a method to identify overlapping spikes in extracellular recordings and to recover hidden spikes by decomposing them. We initially estimate spike waveform templates through a series of steps, including discriminative subspace learning and the isolation forest algorithm. By leveraging these estimated templates, we generate synthetic spikes and train a classifier using their feature components to identify overlapping spikes from observed spike data. The identified overlapping spikes are then decomposed into individual hidden spikes using a particle swarm optimization. Results from the testing of the proposed approach, using the simulation dataset we generated, demonstrated that employing synthetic spikes in the overlapping spike classifier accurately identifies overlapping spikes among the detected ones (the maximum F1 score of 0.88). Additionally, the approach can infer the synchronization between hidden spikes by decomposing the overlapped spikes and reallocating them into distinct clusters. This study advances spike sorting by accurately identifying overlapping spikes, providing a more precise tool for neural activity analysis.

Nonconvex and discriminative transfer subspace learning for unsupervised domain adaptation

Unlike in the field of visual scene recognition, where tremendous advances have taken place due to the availability of very large datasets to train deep neural networks, inference from medical images is often hampered by the fact that only small amounts of data may be available. When working with very small dataset problems, of the order of a few hundred items of data, the power of deep learning may still be exploited by using a pre-trained model as a feature extractor and carrying out classic pattern recognition techniques in this feature space, the so-called few-shot learning problem. However, medical images are highly complex and variable, making it difficult for few-shot learning to fully capture and model these features. To address these issues, we focus on the intrinsic characteristics of the data. We find that, in regimes where the dimension of the feature space is comparable to or even larger than the number of images in the data, dimensionality reduction is a necessity and is often achieved by principal component analysis or singular value decomposition (PCA/SVD). In this paper, noting the inappropriateness of using SVD for this setting we explore two alternatives based on discriminant analysis (DA) and non-negative matrix factorization (NMF). Using 14 different datasets spanning 11 distinct disease types we demonstrate that at low dimensions, discriminant subspaces achieve significant improvements over SVD-based subspaces and the original feature space. We also show that at modest dimensions, NMF is a competitive alternative to SVD in this setting. The implementation of the proposed method is accessible via the following link.

Abstract Many subspace learning methods based on low-rank representation employ the nearest neighborhood graph to preserve the local structure. However, in these methods, the nearest neighborhood graph is a binary matrix, which fails to precisely capture the similarity between distinct samples. Additionally, these methods need to manually select an appropriate number of neighbors, and they cannot adaptively update the similarity graph during projection learning. To tackle these issues, we introduce Discriminative Subspace Learning with Adaptive Graph Regularization (DSL_AGR), an innovative unsupervised subspace learning method that integrates low-rank representation, adaptive graph learning and nonnegative representation into a framework. DSL_AGR introduces a low-rank constraint to capture the global structure of the data and extract more discriminative information. Furthermore, a novel graph regularization term in DSL_AGR is guided by nonnegative representations to enhance the capability of capturing the local structure. Since closed-form solutions for the proposed method are not easily obtained, we devise an iterative optimization algorithm for its resolution. We also analyze the computational complexity and convergence of DSL_AGR. Extensive experiments on real-world datasets demonstrate that the proposed method achieves competitive performance compared with other state-of-the-art methods.

We present the first unsupervised classification of spaxels in hyperspectral images of individual galaxies. Classes identify regions by spectral similarity and thus take all the information into account that is contained in the data cubes (spatial and spectral). We used Gaussian mixture models in a latent discriminant subspace to find clusters of spaxels. The spectra were corrected for small-scale motions within the galaxy based on emission lines with an automatic algorithm. Our data consist of two MUSE/VLT data cubes of JKB 18 and NGC 1068 and one NIRSpec/JWST data cube of NGC 4151. Our classes identify many regions that are most often easily interpreted. Most of the 11 classes that we find for JKB 18 are identified as photoionised by stars. Some of them are known regions, but we mapped them as extended, with gradients of ionisation intensities. One compact structure has not been reported before, and according to diagnostic diagrams, it might be a planetary nebula or a denser region. For NGC 1068, our 16 classes are of active galactic nucleus-type (AGN) or star-forming regions. Their spatial distribution corresponds perfectly to well-known structures such as spiral arms and a ring with giant molecular clouds. A subclassification in the nuclear region reveals several structures and gradients in the AGN spectra. Our unsupervised classification of the MUSE data of NGC 1068 helps visualise the complex interaction of the AGN and the jet with the interstellar medium in a single map. The centre of NGC 4151 is very complex, but our classes can easily be related to ionisation cones, the jet, or H$_2$ emission. We find a new elongated structure that is ionised by the AGN along the N-S axis perpendicular to the jet direction. It is rotated counterclockwise with respect to the axis of the H$_2$ emission. Our work shows that the unsupervised classification of spaxels takes full advantage of the richness of the information in the data cubes by presenting the spectral and spatial information in a combined and synthetic way.

A joint learning framework for optimal feature extraction and multi-class SVM

Cross-dataset discriminant subspace learning algorithm for apple leaf diseases identification

For high-dimensional classification, interpolation of training data manifests as the data piling phenomenon, in which linear projections of data vectors from each class collapse to a single value. Recent research has revealed an additional phenomenon known as the ‘second data piling’ for independent test data in binary classification, providing a theoretical understanding of asymptotically perfect classification. This paper extends these findings to multi-category classification and provides a comprehensive characterization of the double data piling phenomenon. We define the maximal data piling subspace, which maximizes the sum of pairwise distances between piles of training data in multi-category classification. Furthermore, we show that a second data piling subspace that induces data piling for independent data exists and can be consistently estimated by projecting the negatively-ridged discriminant subspace onto an estimated ‘signal’ subspace. By leveraging this second data piling phenomenon, we propose a bias-correction strategy for class assignments, which asymptotically achieves perfect classification. The present research sheds light on benign overfitting and enhances the understanding of perfect multi-category classification of high-dimensional discrimination with a help of high-dimensional asymptotics.

Discriminative sparse subspace learning with manifold regularization

Partial multi-view clustering is a challenging and practical research problem for data analysis in real-world applications, due to the potential data missing issue in different views. However, most existing methods have not fully explored the correlation information among various incomplete views. In addition, these existing clustering methods always ignore discovering discriminative features inside the data itself in this unsupervised task. To tackle these challenges, we propose Partial Multi-View Clustering via Self-Supervised \textbf{N}etwork (PVC-SSN) in this paper. Specifically, we employ contrastive learning to obtain a more discriminative and consistent subspace representation, which is guided by a self-supervised module. Self-supervised learning can exploit effective cluster information through the data itself to guide the learning process of clustering tasks. Thus, it can pull together embedding features from the same cluster and push apart these from different clusters. Extensive experiments on several benchmark datasets show that the proposed PVC-SCN method outperforms several state-of-the-art clustering methods.

State-of-the-art image object detection computational models require an intensive parameter fine-tuning stage (using deep convolution network, etc). with tens or hundreds of training examples. In contrast, human intelligence can robustly learn a new concept from just a few instances (i.e., few-shot detection). The distinctive perception mechanisms between these two families of systems enlighten us to revisit classical handcraft local descriptors (e.g., SIFT, HOG, etc.) as well as non-parametric visual models, which innately require no learning/training phase. Herein, we claim that the inferior performance of these local descriptors mainly results from a lack of global structure sense. To address this issue, we refine local descriptors with spatial contextual attention of neighbor affinities and then embed the local descriptors into discriminative subspace guided by Kernel-InfoNCE loss. Differing from conventional quantization of local descriptors in high-dimensional feature space or isometric dimension reduction, we actually seek a brain-inspired few-shot feature representation for the object manifold, which combines data-independent primitive representation and semantic context learning and thus helps with generalization. The obtained embeddings as pattern vectors/tensors permit us an accelerated but non-parametric visual similarity computation as the decision rule for final detection. Our approach to few-shot object detection is nearly learning-free, and experiments on remote sensing imageries (approximate 2-D affine space) confirm the efficacy of our model.