Subspace Learning Model Research Articles

Bridging Modalities: A Survey of Cross-Modal Image-Text Retrieval

The rapid advancement of Internet technology, driven by social media and e-commerce platforms, has facilitated the generation and sharing of multimodal data, leading to increased interest in efficient cross-modal retrieval systems. Cross-modal image-text retrieval, encompassing tasks such as image query text (IqT) retrieval and text query image (TqI) retrieval, plays a crucial role in semantic searches across modalities. This paper presents a comprehensive survey of cross-modal image-text retrieval, addressing the limitations of previous studies that focused on single perspectives such as subspace learning or deep learning models. We categorize existing models into single-tower, dual-tower, real-value representation, and binary representation models based on their structure and feature representation. Additionally, we explore the impact of multimodal Large Language Models (MLLMs) on cross-modal retrieval. Our study also provides a detailed overview of common datasets, evaluation metrics, and performance comparisons of representative methods. Finally, we identify current challenges and propose future research directions to advance the field of cross-modal image-text retrieval.

Partial Tubal Nuclear Norm-Regularized Multiview Subspace Learning.

In this article, a unified multiview subspace learning model, called partial tubal nuclear norm-regularized multiview subspace learning (PTN2MSL), was proposed for unsupervised multiview subspace clustering (MVSC), semisupervised MVSC, and multiview dimension reduction. Unlike most of the existing methods which treat the above three related tasks independently, PTN2MSL integrates the projection learning and the low-rank tensor representation to promote each other and mine their underlying correlations. Moreover, instead of minimizing the tensor nuclear norm which treats all singular values equally and neglects their differences, PTN2MSL develops the partial tubal nuclear norm (PTNN) as a better alternative solution by minimizing the partial sum of tubal singular values. The PTN2MSL method was applied to the above three multiview subspace learning tasks. It demonstrated that these tasks organically benefited from each other and PTN2MSL has achieved better performance in comparison to state-of-the-art methods.

Robust two-dimensional PCANet with F-norm distance metric

The principal component analysis network (PCANet) is a simplified convolutional neural network (CNN) implemented using a deep subspace learning model. Since its typical PCA filters utilize the squared Frobenius-norm (F-norm) as the distance metric, it exhibits extreme sensitivity to outliers hidden within the data. Furthermore, PCANet faces challenges in extracting structural information in the row and column directions of images as the filters need to be pre-vectorized. To address these issues, we propose a robust bi-directional two-dimensional PCANet (RBDPCANet), wherein the filters utilize the F-norm instead of the squared F-norm to mitigate the impact of outliers. Additionally, the relationship between reconstruction errors and projection distances is simultaneously considered in the objective function. After a concise analytical description of the PCANet concept and related work, we present the methodology for generating three convolutional kernels in the form of RBDPCANet-1, RBDPCANet-2, and RBDPCANet-3 algorithms. These convolution kernels are iteratively obtained and are suitable for numerical implementation. Each of the three algorithms uses another optimization objective function. It has been shown that in image recognition tasks, the developed methods are characterized by better resistance to noise compared to the classic PCANet approach. The rotation invariance and convergence of RBDPCANet are demonstrated according to theoretical principles. The efficacy of the proposed method is validated using images from publicly available datasets. Extensive experimental results on these datasets demonstrate that the proposed RBDPCANet not only significantly boosts the efficiency of image classification but also has strong resistance to occlusions, changes in lighting, changes in orientation, etc. Our implementation code is available at https://github.com/Wesley-li1/RBDPCANet-.

Graph adaptive semi-supervised discriminative subspace learning for EEG emotion recognition

Since Electroencephalogram (EEG) is resistant to camouflage and contains abundant neurophysiological information, it shows significant superiorities in objective emotion recognition, making EEG-based emotion recognition become a hot research field in brain-computer interface research. However, EEG is generally non-stationary and has a low signal-to-noise ratio, which is difficult to analyze. Inspired by the consensus that exploring a discriminative subspace representation usually helps to capture the semantic information of EEG data, in this paper we propose a Graph Adaptive Semi-supervised Discriminative Subspace Learning (GASDSL) model for EEG-based emotion recognition. GASDSL aims to explore a discriminative subspace in which the intra-class scatter decreases while the inter-class separability increases. The adaptive maximum entropy graph construction and semi-supervised subspace emotional state prediction are adopted to mediate the discriminative subspace learning. Extensive comparative studies on the SEED-IV and SEED-V datasets depict that 1) GASDSL achieved satisfactory emotion recognition accuracy compared with other semi-supervised learning models, 2) the discriminative abilities of both the learned maximum entropy graph and subspace are improved as the model iterates, and 3) the features extracted from the Gamma band, the left/right temporal, prefrontal, and (central) parietal lobes contributed more to emotion recognition based on the spatial-frequency pattern analysis results.

Elaborate multi-task subspace learning with discrete group constraint

In multi-task learning (MTL), multiple related tasks can be learned simultaneously under the shared information to improve the generalization performance. However, most of MTL methods assume that all the learning tasks are related indeed and appropriate for joint learning. In some real situations, this assumption may not hold and further lead to the problem of negative transfer. Therefore, in this paper, we not only focus on researching the problem of robustly learning the common feature structure shared by tasks, but also discriminate with which tasks one task should share. By combining with the idea of subspace learning, we propose an elaborate multi-task subspace learning model (EMTSL) with discrete group structure constraint, which can cluster the learned tasks into a set of groups. By introducing the Schatten p-norm instead of trace norm, our model EMTSL can better approximate the low-rank constraint and also avoid the trivial solution. Furthermore, we design an efficient algorithm based on the re-weighted method to solve the proposed model. In addition, the convergence analysis of our algorithm is given in this paper. Experimental results on both synthetic and real-world datasets demonstrate the superiority of our method.

A Covariate-Regulated Sparse Subspace Learning Model and Its Application to Process Monitoring and Fault Isolation

Multivariate functional data are increasingly common in various applications. The cross-correlation of different process variables is typically complex in that a variable might be weakly correlated or not correlated with most of the other variables, and the cross-correlation is time-varying and might be regulated by some exogenous covariates. To address these two challenges, we propose a covariate-regulated sparse subspace learning (CSSL) model. We consider the scenario that these process variables lie in multiple subspaces, and only process variables from the same subspace are cross-correlated with each other. To take into account the effect of the exogenous covariates on the subspace structure, we partition the domain of the covariates into a number of regions. In each region, the subspace structure is treated as constant and can be learned independently. An efficient decision-tree-based algorithm is then proposed to obtain the solution. The proposed method can be further applied to process monitoring and fault isolation for multivariate processes. The efficacy of this method is demonstrated by comprehensive simulations and a case study on a dataset from the supervisory control and data acquisition (SCADA) system of the wind turbine.

T-Linear Tensor Subspace Learning for Robust Feature Extraction of Hyperspectral Images

Subspace learning has been widely applied for feature extraction of hyperspectral images (HSIs) and achieved great success. However, the current methods still leave two problems that need to be further investigated. First, those methods mainly focus on finding one or multiple projection matrices for mapping the high-dimensional data into a low-dimensional subspace, which can only capture the information from each direction of high-order hyperspectral data separately. Second, the performance of feature extraction is barely satisfactory when the hyperspectral data is severely corrupted by noise. To address these issues, this article presents a t-linear tensor subspace learning (tLTSL) model for robust feature extraction of HSIs based on t-product projection. In the model, t-product projection is a new defined tensor transformation way similar to linear transformation in vector space, which can maximally capture the intrinsic structure of tensor data. The integrated tensor low-rank and sparse decomposition can effectively remove the noise corruption and the learned t-product projection can directly transform the high-order hyperspectral data into a subspace with information from all modes comprehensively considered. Moreover, a proposition related to tensor rank is proofed for interpreting the meaning of the tLTSL model. Extensive experiments are conducted on two different kinds of noise (i.e., simulated and real noise) corrupted HSI data, which validate the effectiveness of tLTSL.

Robust multi-view subspace enhanced representation based on collaborative constraints and HSIC induction

The design of effective multi-view subspace clustering (MSC) algorithms has recently garnered significant research attention. Herein, to effectively improve the recognition performance and anti-noise interference ability of an MSC model, we propose a novel MSC algorithm, termed as robust multi-view subspace enhancement representation, based on collaborative constraints and a Hilbert–Schmidt independence criterion (HSIC) induction method. To mine the complementary information between different views, we apply the HSIC as a diversity regularization term. Specifically, to enhance the diagonal block structure of a subspace representation, a new sparse constraint is introduced on the product of itself and the transpose of the subspace representation matrix in a multi-view subspace learning model. Furthermore, hypergraph regularization and a low-rank idea are considered to capture the local geometric structure and clean data. In addition, to optimize our model, we adopt an augmented Lagrangian multiplier method and discuss the convergence of the model. Extensive experiments on six challenging datasets reveal that the proposed method achieves a highly competent objective performance with and without noisy views, as compared with several state-of-the-art multi-view clustering methods.

Trace ratio optimization with an application to multi-view learning

A trace ratio optimization problem over the Stiefel manifold is investigated from the perspectives of both theory and numerical computations. Necessary conditions in the form of nonlinear eigenvalue problem with eigenvector dependency (NEPv) are established and a numerical method based on the self-consistent field (SCF) iteration with a postprocessing step is designed to solve the NEPv and the method is proved to be always convergent. As an application to multi-view subspace learning, a new framework and its instantiated concrete models are proposed and demonstrated on real world data sets. Numerical results show that the efficiency of the proposed numerical methods and effectiveness of the new orthogonal multi-view subspace learning models.

Multi-Stage Graph Fusion Networks for Major Depressive Disorder Diagnosis

Major depressive disorder (MDD) is a common and severe psychiatric illness marked by loss of interest and low energy, which result in the highest burden of disability among all mental disorders. Clinical MDD diagnosis still utilizes the phenomenological approach of syndrome-based interview, which leads to a high rate of misdiagnosis. Therefore, it is highly imperative to explore effective biomarkers to enable precise personalized diagnosis. There still exist two main challenges due to complexity of MDD and individual differences. On the one hand, discriminative features need to be investigated to better reflect the characteristics of MDD. On the other hand, the performance from shallow and static learning models is still not satisfactory. To overcome these issues, we propose a novel Multi-Stage Graph Fusion Networks (MSGFN) for major depressive disorder diagnosis. At first, functional connectivity is calculated to better characterize interactions between white matter and gray matter. Second, multi-stage features are obtained by a deep subspace learning model, and a number of graphs are constructed under the self-expression constraints at each stage. Finally, a novel graph convolutional fusion module is proposed with graph convolutional operations to integrate features and graph at each stage. Extensive experiments demonstrate the superior performance of the proposed framework. Our source code is available on: <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/LIST-KONG/MSGFN-master</uri> .

Online subspace learning and imputation by Tensor-Ring decomposition

This paper considers the completion problem of a partially observed high-order streaming data, which is cast as an online low-rank tensor completion problem. Though the online low-rank tensor completion problem has drawn lots of attention in recent years, most of them are designed based on the traditional decomposition method, such as CP and Tucker. Inspired by the advantages of Tensor Ring decomposition over the traditional decompositions in expressing high-order data and its superiority in missing values estimation, this paper proposes two online subspace learning and imputation methods based on Tensor Ring decomposition. Specifically, we first propose an online Tensor Ring subspace learning and imputation model by formulating an exponentially weighted least squares with Frobenium norm regularization of TR-cores. Then, two commonly used optimization algorithms, i.e. alternating recursive least squares and stochastic-gradient algorithms, are developed to solve the proposed model. Numerical experiments show that the proposed methods are more effective to exploit the time-varying subspace in comparison with the conventional Tensor Ring completion methods. Besides, the proposed methods are demonstrated to be superior to obtain better results than state-of-the-art online methods in streaming data completion under varying missing ratios and noise.

A Cross-Media Advertising Design and Communication Model Based on Feature Subspace Learning.

This paper uses feature subspace learning and cross-media retrieval analysis to construct an advertising design and communication model. To address the problems of the traditional feature subspace learning model and make the samples effectively maintain their local structure and discriminative properties after projection into the feature space, this paper proposes a discriminative feature subspace learning model based on Low-Rank Representation (LRR), which explores the local structure of samples through Low-Rank Representation and uses the representation coefficients as similarity constraints of samples in the projection space so that the projection subspace can better maintain the local nearest-neighbor relationship of samples. Based on the common subspace learning, this paper uses the extreme learning machine method to improve the cross-modal retrieval accuracy, mining deeper data features and maximizing the correlation between different modalities, so that the learned shared subspace is more discriminative; meanwhile, it proposes realizing cross-modal retrieval by the deep convolutional generative adversarial network, using unlabeled samples to further explore the correlation of different modal data and improve the cross-modal performance. The clustering quality of images and audios is corrected in the feature subspace obtained by dimensionality reduction through an optimization algorithm based on similarity transfer. Three active learning strategies are designed to calculate the conditional probability of unannotated samples around user-annotated samples in the correlation feedback process, thus improving the efficiency of cross-media retrieval in the case of limited feedback samples. The experimental results show that the method accurately measures the cross-media relevance and effectively achieves mutual retrieval between image and audio data. Through the study of cross-media advertising design and communication models based on feature subspace learning, it is of positive significance to advance commercial advertising design by guiding designers and artists to better utilize digital media technology for artistic design activities at the level of theoretical research and applied practice.

VoxelHop: Successive Subspace Learning for ALS Disease Classification Using Structural MRI.

Deep learning has great potential for accurate detection and classification of diseases with medical imaging data, but the performance is often limited by the number of training datasets and memory requirements. In addition, many deep learning models are considered a "black-box," thereby often limiting their adoption in clinical applications. To address this, we present a successive subspace learning model, termed VoxelHop, for accurate classification of Amyotrophic Lateral Sclerosis (ALS) using T2-weighted structural MRI data. Compared with popular convolutional neural network (CNN) architectures, VoxelHop has modular and transparent structures with fewer parameters without any backpropagation, so it is well-suited to small dataset size and 3D imaging data. Our VoxelHop has four key components, including (1) sequential expansion of near-to-far neighborhood for multi-channel 3D data; (2) subspace approximation for unsupervised dimension reduction; (3) label-assisted regression for supervised dimension reduction; and (4) concatenation of features and classification between controls and patients. Our experimental results demonstrate that our framework using a total of 20 controls and 26 patients achieves an accuracy of 93.48 % and an AUC score of 0.9394 in differentiating patients from controls, even with a relatively small number of datasets, showing its robustness and effectiveness. Our thorough evaluations also show its validity and superiority to the state-of-the-art 3D CNN classification approaches. Our framework can easily be generalized to other classification tasks using different imaging modalities.

DS4L: Deep Semisupervised Shared Subspace Learning for Hyperspectral Image Classification

Hyperspectral image (HSI) classification is essential in remote sensing image analysis. The classification methods based on deep learning have attracted more and more attention. However, classification accuracy is seriously affected by the quantity of labeled data and redundant information. Therefore, a deep semisupervised shared subspace learning (DS <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">4</sup> L) model is developed to overcome these problems in this article. DS <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">4</sup> L is composed of two parts. First, the basic feature extraction (BFE) network is constructed to preliminary extract high-dimensional-space features of multiscale data and fusion them to one shared subspace. Then, a deep shared subspace learning (DSSL) network is proposed to obtain a deeper and more representative low-dimensional subspace. Moreover, to obtain a more representative subspace and alleviate dependence on labeled samples, the regular, irregular constraint, and cross-entropy (CE) loss are integrated into the model. The regular constraint is adopted to reconstruct the multiscale patches to ensure the quality of the subspace in an unsupervised manner. The irregular constraint can well embed labeled and unlabeled samples into the procedure of subspace learning (SL). Then, the CE loss is used to extract more discriminative subspace using the limited labeled samples. Finally, we perform experiments on three widely used HSI datasets. Compared with the basic SL model, the DS <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">4</sup> L’s classification accuracy on the popular Salinas, Indian Pines, and PaviaU datasets are increased by 4.68%, 4.25%, and 1.57%, respectively.

Sketched Multiview Subspace Learning for Hyperspectral Anomalous Change Detection

In recent years, multi-view subspace learning has been garnering increasing attention. It aims to capture the inner relationships of the data that are collected from multiple sources by learning a unified representation. In this way, comprehensive information from multiple views is shared and preserved for the generalization processes. As a special branch of temporal series hyperspectral image (HSI) processing, the anomalous change detection task focuses on detecting very small changes among different temporal images. However, when the volume of datasets is very large or the classes are relatively comprehensive, existing methods may fail to find those changes between the scenes, and end up with terrible detection results. In this paper, inspired by the sketched representation and multi-view subspace learning, a sketched multi-view subspace learning (SMSL) model is proposed for HSI anomalous change detection. The proposed model preserves major information from the image pairs and improves computational complexity by using a sketched representation matrix. Furthermore, the differences between scenes are extracted by utilizing the specific regularizer of the self-representation matrices. To evaluate the detection effectiveness of the proposed SMSL model, experiments are conducted on a benchmark hyperspectral remote sensing dataset and a natural hyperspectral dataset, and compared with other state-of-the art approaches.

Weighted multi-view common subspace learning method

• A novel multi-view common subspace learning model is proposed. • The model can obtain a unified common subspace by using the discriminant information within and between views. • The discriminant information within and between views can be adjusted by the weighted coefficient. • Our model adopts the maximize scatter difference criterion as the metric between and within the views after projection. How to use multi-view data effectively has become one of the challenging problems in the computer vision community. The existing multi-view learning methods are mainly based on common subspace learning which aim to explore the discriminative information between multi-view data and find its potential common subspace. Most of the existing multi-view subspace learning methods rely on the within-class scatter matrix and between-class scatter matrix while capturing the discriminative information of multiple views. However, these methods just roughly minimize the within-class distance and maximize the between-class distance, and do not make full use of the intra-view and inter-view information. To address this problem, we propose a weighted common subspace learning method, which can effectively adjust the contribution ratio of between-class and within-class information through a weighted parameter, so that an optimized common subspace can be obtained. And we use the maximum scatter difference criterion as the metric of inter-view and intra-view after projection. Extensive experiments on the public data sets show the superiority of this method.

NSL2CD: identifying potential circRNA-disease associations based on network embedding and subspace learning.

Many studies have evidenced that circular RNAs (circRNAs) are important regulators in various pathological processes and play vital roles in many human diseases, which could serve as promising biomarkers for disease diagnosis, treatment and prognosis. However, the functions of most of circRNAs remain to be unraveled, and it is time-consuming and costly to uncover those relationships between circRNAs and diseases by conventional experimental methods. Thus, identifying candidate circRNAs for human diseases offers new opportunities to understand the functional properties of circRNAs and the pathogenesis of diseases. In this study, we propose a novel network embedding-based adaptive subspace learning method (NSL2CD) for predicting potential circRNA-disease associations and discovering those disease-related circRNA candidates. The proposed method first calculates disease similarities and circRNA similarities by fully utilizing different data sources and learns low-dimensional node representations with network embedding methods. Then, we adopt an adaptive subspace learning model to discover potential associations between circRNAs and diseases. Meanwhile, an integrated weighted graph regularization term is imposed to preserve local geometric structures of data spaces, and L1,2-norm constraint is also incorporated into the model to realize the smoothness and sparsity of projection matrices. The experiment results show that NSL2CD achieves comparable performance under different evaluation metrics, and case studies further confirm its ability to discover potential candidate circRNAs for human diseases.

Local Manifold Embedding Cross-Domain Subspace Learning for Drift Compensation of Electronic Nose Data

The gas sensor drift problem arises from the bias of data, which is known as a significant problem in the artificial olfactory community. Traditionally, hardware calibration methods are laborious and ineffective due to frequent recalibration actions involving different gases, and some calibration transfer and baseline calibration methods are not effective enough. In this work, a local manifold embedding cross-domain subspace learning (LME-CDSL) model is proposed based on domain distribution alignment. It is a unified subspace learning model combined with manifold learning and domain adaptation, which tends to explore a latent transform matrix that not only enforces the drifted target domain data to learn the manifold of nondrifted source domain data but also adopts the domain adaptation method to align the domain data distribution. In general, the LME-CDSL model has three features: 1) the unsupervised and adaptation distribution subspace projection can be easily computed through eigenvector decomposition; 2) the local linear manifold learns to achieve the compact representations of high-dimensional data and is capable of preserving the local features of nondrifted samples; and 3) the domain adaptation part utilizes the maximum mean discrepancy (MMD) and variance maximization to make the sample distributions of different domains more similar and preserve the intrinsic properties. For long-term and short-term drift compensation on a single E-nose system, the local manifold embedding cross-domain subspace learning (LME-CDSL) model obtains the average recognition accuracy of 70.95% and 74.09%, respectively, while 71.71% and 73.96%, respectively for multiple identical E-nose systems with both long-term and interplate drift, which are higher than several comparative methods and proves the its effectiveness and superiority on anti-drift and gas recognition.

Continual Multiview Task Learning via Deep Matrix Factorization.

The state-of-the-art multitask multiview (MTMV) learning tackles a scenario where multiple tasks are related to each other via multiple shared feature views. However, in many real-world scenarios where a sequence of the multiview task comes, the higher storage requirement and computational cost of retraining previous tasks with MTMV models have presented a formidable challenge for this lifelong learning scenario. To address this challenge, in this article, we propose a new continual multiview task learning model that integrates deep matrix factorization and sparse subspace learning in a unified framework, which is termed deep continual multiview task learning (DCMvTL). More specifically, as a new multiview task arrives, DCMvTL first adopts a deep matrix factorization technique to capture hidden and hierarchical representations for this new coming multiview task while accumulating the fresh multiview knowledge in a layerwise manner. Then, a sparse subspace learning model is employed for the extracted factors at each layer and further reveals cross-view correlations via a self-expressive constraint. For model optimization, we derive a general multiview learning formulation when a new multiview task comes and apply an alternating minimization strategy to achieve lifelong learning. Extensive experiments on benchmark data sets demonstrate the effectiveness of our proposed DCMvTL model compared with the existing state-of-the-art MTMV and lifelong multiview task learning models.

Infrared Dim and Small Target Detection via Multiple Subspace Learning and Spatial-Temporal Patch-Tensor Model

Robust detection of infrared small and dim targets with highly heterogeneous backgrounds plays an indispensable role in infrared search and tracking (IRST) system, which is still a challenging problem. In this article, a novel method based on multisubspace learning and spatial-temporal tensor data structure is aimed to solve this problem. First, a tensor data structure is constructed to use inner correlation in spatial and temporal domain of the infrared image sequence. Second, in consideration of the complex and heterogeneous backgrounds in infrared images, the proposed method promotes the multisubspace property to tensor domain to separate the target and the background more accurately. Finally, an efficient and effective optimization algorithm based on alternating direction method of multipliers (ADMM) is designed to solve this problem. Experimental results on various and real scenes demonstrate the superiority of the proposed method compared to other five baseline methods.