Subspace Learning Research Articles

Ensemble-Enhanced Semi-Supervised Learning with Optimized Graph Construction for High-Dimensional Data.

Graph-based methods have demonstrated exceptional performance in semi-supervised classification. However, existing graph-based methods typically construct either a predefined graph in the original space or an adaptive graph within the output space, which often limits their ability to fully utilize prior information and capture the optimal intrinsic data distribution, particularly in high-dimensional data with abundant redundant and noisy features. This paper introduces a novel approach: Semi-Supervised Classification with Optimized Graph Construction (SSC-OGC). SSC-OGC leverages both predefined and adaptive graphs to explore intrinsic data distribution and effectively employ prior information. Additionally, a graph constraint regularization term (GCR) and a collaborative constraint regularization term (CCR) are incorporated to further enhance the quality of the adaptive graph structure and the learned subspace, respectively. To eliminate the negative effect of constructing a predefined graph in the original data space, we further propose a Hybrid Subspace Ensemble-enhanced framework based on the proposed Optimized Graph Construction method (HSE-OGC). Specifically, we construct multiple hybrid subspaces, which consist of meticulously chosen features from the original data to achieve high-quality and diverse space representations. Then, HSE-OGC constructs multiple predefined graphs within hybrid subspaces and trains multiple SSC-OGC classifiers to complement each other, significantly improving the overall performance. Experimental results conducted on various high-dimensional datasets demonstrate that HSE-OGC exhibits outstanding performance.

Asymptotically faster estimation of high‐dimensional additive models using subspace learning

AbstractAs a commonly used nonparametric model to overcome the curse of dimensionality, the additive model continually attracts attentions of researchers. Our recent work (He et al., 2022) proposed to reduce the number of unknown functions to be estimated through learning an adaptive subspace shared by the additive component functions. Equipped with an efficient algorithm, our proposed reduced additive model outperforms the state‐of‐the‐art alternatives in numerical studies. However, the asymptotic properties of the proposed estimators have not been explored and the empirical findings are short of theoretical explanation. In this work, we fill in the theoretical gap by showing the resulting estimator has faster convergence rate than the estimation without subspace learning; and this is true even when the reduced additive model is only an approximation, provided that the subspace approximation error is small. Moreover, the proposed method is able to consistently identify the relevant predictors. The developed theoretical results back up the earlier empirical findings.

Iterative Multiview Subspace Learning for Unpaired Multiview Clustering.

In real applications, several unpredictable or uncertain factors could result in unpaired multiview data, i.e., the observed samples between views cannot be matched. Since joint clustering among views is more effective than individual clustering in each view, we investigate unpaired multiview clustering (UMC), which is a valuable but insufficiently studied problem. Due to lack of matched samples between views, we could fail to build the connection between views. Therefore, we aim to learn the latent subspace shared by views. However, existing multiview subspace learning methods usually rely on the matched samples between views. To address this issue, we propose an iterative multiview subspace learning strategy [iterative unpaired multiview clustering (IUMC)], aiming to learn a complete and consistent subspace representation among views for UMC. Moreover, based on IUMC, we design two effective UMC methods: 1) Iterative unpaired multiview clustering via covariance matrix alignment (IUMC-CA) that further aligns the covariance matrix of subspace representations and then performs clustering on the subspace and 2) iterative unpaired multiview clustering via one-stage clustering assignments (IUMC-CY) that performs one-stage multiview clustering (MVC) by replacing the subspace representations with clustering assignments. Extensive experiments show the excellent performance of our methods for UMC, compared with the state-of-the-art methods. Also, the clustering performance of observed samples in each view can be considerably improved by those observed samples from the other views. In addition, our methods have good applicability in incomplete MVC.

Enhancing plant disease detection: a novel CNN-based approach with tensor subspace learning and HOWSVD-MDA

Enhancing plant disease detection: a novel CNN-based approach with tensor subspace learning and HOWSVD-MDA

Cancer subtype identification by multi-omics clustering based on interpretable feature and latent subspace learning

In recent years, multi-omics clustering has become a powerful tool in cancer research, offering a comprehensive perspective on the diverse molecular characteristics inherent to various cancer subtypes. However, most existing multi-omics clustering methods directly integrate heterogeneous features from different omics, which may struggle to deal with the noise or redundancy of multi-omics data and lead to poor clustering results. Therefore, we propose a novel multi-omics clustering method to extract interpretable and discriminative features from various omics before data integration. The clinical information is used to supervise the process of feature extraction based on SHAP (SHapley Additive exPlanation) values. Singular value decomposition (SVD) is then applied to integrate the extracted features of different omics by constructing a latent subspace. Finally, we utilize shared nearest neighbor-based spectral clustering on the latent representation to obtain the clustering result. The proposed method is evaluated on several cancer datasets across three levels of omics, in comparison to several state-of-the-art multi-omics clustering methods. The comparison results demonstrate the superior performance of the proposed method in multi-omics data analysis for cancer subtyping. Additionally, experiments reveal the efficacy of utilizing clinical information based on SHAP values for feature extraction, enhancing the performance of clustering analyses. Moreover, enrichment analysis of the identified gene signatures in different subtypes is also performed to further demonstrate the effectiveness of the proposed method.Availability: The proposed method can be freely accessible at https://github.com/Tianyi-Shi-Tsukuba/Multi-omics-clustering-based-on-SHAP. Data will be made available on request.

SCAE: Structural Contrastive Auto-Encoder for Incomplete Multi-View Representation Learning

Describing an object from multiple perspectives often leads to incomplete data representation. Consequently, learning consistent representations for missing data from multiple views has emerged as a key focus in the realm of Incomplete Multi-view Representation Learning (IMRL). In recent years, various strategies, such as subspace learning, matrix decomposition, and deep learning, have been harnessed to develop numerous IMRL methods. In this article, our primary research revolves around IMRL, with a particular emphasis on addressing two main challenges. Firstly, we delve into the effective integration of intra-view similarity and contextual structure into a unified framework. Secondly, we explore the effective facilitation of information exchange and fusion across multiple views. To tackle these issues, we propose a deep learning approach known as Structural Contrastive Auto-Encoder (SCAE) to solve the challenges of IMRL. SCAE comprises two major components: intra-view structural representation learning and inter-view contrastive representation learning. The former involves capturing intra-view similarity by minimizing the Dirichlet energy of the feature matrix, while also applying spatial dispersion regularization to capture intra-view contextual structure. The latter encourages maximizing the mutual information of inter-view representations, facilitating information exchange and fusion across views. Experimental results demonstrate the efficacy of our approach in significantly enhancing model accuracy and robustly addressing IMRL problems. The code is available at https://github.com/limengran98/SCAE .

Fast Unsupervised Cross-modal Hashing With Robust Factorization and Dual Projection

Unsupervised hashing has attracted extensive attention in effectively and efficiently tackling large-scale cross-modal retrieval task. Existing methods typically try to mine the latent common subspace across multimodal data without any category annotation. Despite the exciting progress, there are still three challenges that need to be further addressed: 1) efficiently improving the robustness during latent common subspace learning; 2) harmoniously embedding the intra-modal inherence and inter-modal relevance of multimodal data into Hamming space; and 3) effectively reducing the training time complexity and making the model scalable for large-scale datasets. To well address the above challenges, this study proposes a method named Fast Unsupervised Cross-modal Hashing (FUCH). Specifically, FUCH proposes a semantic-aware collective matrix factorization to learn robust representation via exploiting latent category-specific attributes, and introduces Cauchy loss to measure the factorization process. Accordingly, the above process can effectively embed potential discriminative information into common space, while making the model insensitive for outliers. Moreover, FUCH designs a dual projection learning scheme, which not only learns modality-unique hash functions to excavate individual properties, but also learns modality-mutual hash functions to multimodal correlational properties. Experimental results on three benchmark datasets verify the effectiveness of FUCH under various scenarios.

Improved-MUSIC Algorithm Based on Compressive Subspace Learning for Multiantenna Cognitive Radio

Improved-MUSIC Algorithm Based on Compressive Subspace Learning for Multiantenna Cognitive Radio

Cross-domain manifold structure preservation for transferable and cross-machine fault diagnosis

To address the decline or failure in the autonomous learning capability of traditional transfer learning methods when training and test samples come from different machines, resulting in low cross-machine fault diagnosis rates, we propose a cross-domain manifold structure preservation (CDMSP) method for diagnosing rolling bearing faults across machines. The CDMSP method can induce the manifold space projection matrices of the source and target domains more effectively. This method maps high-dimensional features into a low-dimensional manifold, preserving non-linear relationships and aligning distribution differences while maintaining cross-domain manifold structure consistency. Additionally, highly confidently labeled target domain samples are selected from each mapping result and added to the training dataset to enhance subspace learning in subsequent iterations. The CDMSP method is both simple and effective at capturing the underlying structures and patterns in the data. The CWRU dataset and our self-built test platform dataset were used to validate this method. Experimental results show that CDMSP, as a non-deep domain adaptation method of transfer learning, outperforms similar methods in cross-machine fault identification, achieving a maximum fault identification accuracy of 100 % with excellent convergence performance. Furthermore, simulated diagnostic experiments under noise interference indicate that CDMSP maintains high fault identification accuracy, even in noisy environments. Overall, CDMSP is an efficient and reliable new method for diagnosing cross-machine bearing faults.

Exploring corpus-invariant emotional acoustic feature for cross-corpus speech emotion recognition

Unsupervised cross-corpus speech emotion recognition (SER) is the task where the labeled training (source) and unlabeled testing (target) speech come from different corpora. Subspace transfer learning is one of the mainstream technologies for tackling cross-corpus SER challenges, which is achieved by utilizing projection matrices to learn common corpus-invariant feature representations between the source and target corpus. However, these methods mainly focus on mapping modeling from the input low-level descriptor (LLD) feature space to the corpus-invariant emotional feature space, which is an implicit feature mapping process that lacks interpretability for the selected features. This omission leads to an inability to pinpoint which acoustic features possess corpus invariance. To bridge this gap, we first propose a new transfer subspace learning framework with feature selection capabilities, i.e., the Corpus-Invariant Emotional Acoustic Feature Seeker (CAFS). Specifically, the CAFS integrates two core terms into the transfer regression loss function: (1) the emotion preservation term: This term includes emotional regression and the l2,1 norm, which is mainly used to select features and ensure that these features are related to emotions. (2) the corpus invariance preservation term: This item is mainly used to measure the difference in feature distribution between the source and target corpora. Minimizing this term bridges the gap between the source and target domains, ensuring that the chosen acoustic features are corpus-invariant. Subsequently, we conducted extensive cross-corpus SER experiments to explore corpus-invariant emotional acoustic features under various commonly used acoustic feature sets (IS09 and eGeMAPS). Through statistical analysis of the acoustic features sought by the CAFS framework, some acoustic features (e.g., Mel-Frequency Cepstral Coefficients (MFCC) and Formant) reveal their corpus-invariant properties, which could provide insights for feature selection in cross-corpus SER. These findings also lay the groundwork for a solid theoretical and empirical foundation for future research and applications in cross-corpus SER.

TAKAGI–SUGENO–KANG FUZZY SYSTEM MODELING BASED ON LOW-RANK SPARSE SUBSPACE LEARNING FOR MOTOR IMAGERY ELECTROENCEPHALOGRAM SIGNAL CLASSIFICATION

The classification of electroencephalogram (EEG) signals derived from motor imagery (MI) has always been a hot topic in the field of brain–computer interfaces. Due to its ability to handle the nonstationary and uncertain information contained in EEG signals, the Takagi–Sugeno–Kang fuzzy system (TSK-FS) has become an advantageous classification algorithm. To train a fuzzy system with strong discrimination capabilities from EEG data interspersed with redundant information, this paper proposes a TSK-FS modeling method based on low-rank sparse subspace learning (TSK-LSSL). This method focuses on consequent parameter learning, which transforms the traditional consequent parameter learning strategy into low-rank subspace and sparse subspace learning processes. Low-rank subspace learning is used to mine the global structural information of data and effectively reduce the number of fuzzy rules. During sparse subspace learning, [Formula: see text]-norm regularization is used to constrain the consequent parameters and causes the number of redundant consequent parameters to be zero, thereby simplifying the fuzzy rules. In addition, a local boundary term based on graph matrices is embedded into the objective function to mine the local structural information of the given data. TSK-LSSL simplifies the number of rules and the consequent part of the fuzzy rules. It exhibits good classification performance on two BCI Competition databases.

Robust and adaptive subspace learning for fast hyperspectral image denoising

Robust and adaptive subspace learning for fast hyperspectral image denoising

A hybrid multilinear-linear subspace learning approach for enhanced person re-identification in camera networks

Several tasks in surveillance systems depend on camera networks, one of them is Person re-identification (PRe-ID) which has wide interest as a research topic in the computer vision field. The current techniques in this issue encounter many obstacles in handling the variability of appearance, extracting effective features, and capturing intricate data relationships, leading to suboptimal performance causing systems to have low performance. This paper presents a novel hybrid subspace learning approach for Person re-identification (PRe-ID), that leverages the strengths of both multilinear and linear techniques to significantly enhance the performance of PRe-ID. The proposed process begins by extracting distinctive effective features from pedestrian images using LOMO and GOG as shallow feature descriptors with CNN for deep features, followed by the application of two stages of subspace learning. In the first stage, we employ a multilinear subspace learning algorithm known as Tensor-based Quadratic Discriminant Analysis (TXQDA). TXQDA captures complex relationships and interactions among different features, resulting in a more discriminative feature representation. Subsequently, we employ a linear subspace learning technique called Side Information Linear Discriminant Analysis (SILD) to refine the subspace learned from TXQDA. SILD utilizes linear projections to enhance discriminability and preserve crucial inter-class variations, thereby improving the overall performance of PRe-ID. Ultimately, we implement logistic regression (LR) fusion at the score level to enhance the performance of our hybrid subspace learning approach, particularly within various tensor feature representation scenarios. Extensive experiments are conducted on three challenging datasets to evaluate the effectiveness of our hybrid subspace learning approach. The experimental results demonstrate that our method achieves superior performance compared to state-of-the-art techniques in the field of PRe-ID.

Dual-dual subspace learning with low-rank consideration for feature selection

The performance of machine learning algorithms can be affected by redundant features of high-dimensional data. Furthermore, these irrelevant features increase the time of computation for learning model. These problems can be addressed by leveraging different techniques including feature selection and dimensionality reduction. Unsupervised feature selection has drawn increased attention due to the difficulty of label collection for supervised feature selection. To this end, we developed an innovative approach based on nonnegative matrix factorization (NMF) to remove redundant information. In this technique, for the first time, the local information preserving regularization and global information preserving regularization are applied for both feature weight matrix and representation matrix which is why we called Dual-Dual regularized feature selection. Furthermore, Schatten p-norm is utilized to extract inherent low-rank properties of data. To demonstrate the effectiveness of the proposed method, experimental studies are conducted on six benchmark datasets. The computational results show that the proposed method in comparison with state-of-the-art unsupervised feature selection techniques is more efficient for feature selection.

Abstract We112: Volumetric imaging to recapitulate cardiac injury and arrhythmias

Advanced understanding of cardiac structure and function is crucial for uncovering the underlying mechanism of injury and arrhythmias. To explore cues to structural and functional abnormalities in extensively established animal models, we have developed optical imaging and computational methods tailored to investigate intact hearts of zebrafish and rodents at cellular resolution. Our cardiac light-field microscopy allows us to capture instantaneous dynamics such as calcium transients, irregular contraction, and blood flow at 200 volumes per second in zebrafish larvae 1 . To improve the spatial resolution and enable the study of rodent models, we have customized two light-sheet imaging systems. One system is empowered by the retrospective synchronization and machine learning for the study of contractile function and focal myocardial mechanics from end-systole to end-diastole 2 , while the other integrates tissue clearing and axially scanning approaches for the exploration of ventricular trabeculation and myocardial compaction 3 . These methods aim to overcome the trade-off in spatial resolution, imaging speed, field of view across the atria and ventricles with minimal photo-damage and maximal penetration depth, enabling long-term investigation of cardiac development and regenerative processes. To further interpret the large volume of datasets, we have customized a successive subspace learning and virtual reality-based platform for interactive image analysis 4 . Collectively, our multi-scale strategy has opened up new landscapes for exploring the cardiac micro-structure and contractile function, both in health and disease.

Efficient Multi-View Clustering via Unified and Discrete Bipartite Graph Learning.

Although previous graph-based multi-view clustering (MVC) algorithms have gained significant progress, most of them are still faced with three limitations. First, they often suffer from high computational complexity, which restricts their applications in large-scale scenarios. Second, they usually perform graph learning either at the single-view level or at the view-consensus level, but often neglect the possibility of the joint learning of single-view and consensus graphs. Third, many of them rely on the k -means for discretization of the spectral embeddings, which lack the ability to directly learn the graph with discrete cluster structure. In light of this, this article presents an efficient MVC approach via u nified and d iscrete b ipartite g raph l earning (UDBGL). Specifically, the anchor-based subspace learning is incorporated to learn the view-specific bipartite graphs from multiple views, upon which the bipartite graph fusion is leveraged to learn a view-consensus bipartite graph with adaptive weight learning. Furthermore, the Laplacian rank constraint is imposed to ensure that the fused bipartite graph has discrete cluster structures (with a specific number of connected components). By simultaneously formulating the view-specific bipartite graph learning, the view-consensus bipartite graph learning, and the discrete cluster structure learning into a unified objective function, an efficient minimization algorithm is then designed to tackle this optimization problem and directly achieve a discrete clustering solution without requiring additional partitioning, which notably has linear time complexity in data size. Experiments on a variety of multi-view datasets demonstrate the robustness and efficiency of our UDBGL approach. The code is available at https://github.com/huangdonghere/UDBGL.

Efficient Local Coherent Structure Learning via Self-Evolution Bipartite Graph.

Dimensionality reduction (DR) targets to learn low-dimensional representations for improving discriminability of data, which is essential for many downstream machine learning tasks, such as image classification, information clustering, etc. Non-Gaussian issue as a long-standing challenge brings many obstacles to the applications of DR methods that established on Gaussian assumption. The mainstream way to address above issue is to explore the local structure of data via graph learning technique, the methods based on which however suffer from a common weakness, that is, exploring locality through pairwise points causes the optimal graph and subspace are difficult to be found, degrades the performance of downstream tasks, and also increases the computation complexity. In this article, we first propose a novel self-evolution bipartite graph (SEBG) that uses anchor points as the landmark of subclasses, and learns anchor-based rather than pairwise relationships for improving the efficiency of locality exploration. In addition, we develop an efficient local coherent structure learning (ELCS) algorithm based on SEBG, which possesses the ability of updating the edges of graph in learned subspace automatically. Finally, we also provide a multivariable iterative optimization algorithm to solve proposed problem with strict theoretical proofs. Extensive experiments have verified the superiorities of the proposed method compared to related SOTA methods in terms of performance and efficiency on several real-world benchmarks and large-scale image datasets with deep features.

Multiview ensemble clustering of hypergraph p-Laplacian regularization with weighting and denoising

Multiview clustering has gained attention for its ability to incorporate complementary information from multiple sources of data, leading to better clustering results. However, these methods don't sufficiently mine high-dimensional information of non-linear subspace or ignore important high-level information between basic partitions (BPs), which are obtained via single-view clustering that could help to overcome differences between heterogeneous feature spaces and defined in Eq. (4). Additionally, existing multiview ensemble clustering methods neglect the noise arising from the generation phase. In light of this, we first propose a novel multiview subspace clustering method with hypergraph p-Laplacian regularization and denoising. Specifically, to utilize the high-dimensional information, a hypergraph p-Laplacian regularized term is added to the model along with low-rank subspace learning to capture the different hierarchical structures in the data. A denoising algorithm based on KMeans and K-nearest neighbor is used to minimize the noise of similarity matrix. Then utilizing this approach as a backbone, a multiview ensemble clustering of hypergraph p-Laplacian regularization with weighting and denoising method is proposed. A hybrid strategy and a novel global weighting ensemble strategy is further proposed to extract high-level information across all the BPs. By integrating hypergraph p-Laplacian operator, low-rank subspace learning, denoising, and weighting ensemble strategy in a unified framework, this ensemble approach adequately learns latent structures and complementary information. Experimental results on multiple datasets demonstrate the efficacy of this approach.

A novel robust adaptive subspace learning framework for dimensionality reduction

A novel robust adaptive subspace learning framework for dimensionality reduction

Multiview Clustering With Propagating Information Bottleneck.

In many practical applications, massive data are observed from multiple sources, each of which contains multiple cohesive views, called hierarchical multiview (HMV) data, such as image-text objects with different types of visual and textual features. Naturally, the inclusion of source and view relationships offers a comprehensive view of the input HMV data and achieves an informative and correct clustering result. However, most existing multiview clustering (MVC) methods can only process single-source data with multiple views or multisource data with single type of feature, failing to consider all the views across multiple sources. Observing the rich closely related multivariate (i.e., source and view) information and the potential dynamic information flow interacting among them, in this article, a general hierarchical information propagation model is first built to address the above challenging problem. It describes the process from optimal feature subspace learning (OFSL) of each source to final clustering structure learning (CSL). Then, a novel self-guided method named propagating information bottleneck (PIB) is proposed to realize the model. It works in a circulating propagation fashion, so that the resulting clustering structure obtained from the last iteration can "self-guide" the OFSL of each source, and the learned subspaces are in turn used to conduct the subsequent CSL. We theoretically analyze the relationship between the cluster structures learned in the CSL phase and the preservation of relevant information propagated from the OFSL phase. Finally, a two-step alternating optimization method is carefully designed for optimization. Experimental results on various datasets show the superiority of the proposed PIB method over several state-of-the-art methods.