Low-rank Subspace Research Articles

Observable-augmented manifold learning for multi-source turbulent flow data

This study seeks a low-rank representation of turbulent flow data obtained from multiple sources. To uncover such a representation, we consider finding a finite-dimensional manifold that captures underlying turbulent flow structures and characteristics. While nonlinear machine-learning techniques can be considered to seek a low-order manifold from flow field data, there exists an infinite number of transformations between data-driven low-order representations, causing difficulty in understanding turbulent flows on a manifold. Finding a manifold that captures turbulence characteristics becomes further challenging when considering multi-source data together due to the presence of inherent noise or uncertainties and the difference in the spatiotemporal length scale resolved in flow snapshots, which depends on approaches in collecting data. With an example of numerical and experimental data sets of transitional turbulent boundary layers, this study considers an observable-augmented nonlinear autoencoder-based compression, enabling data-driven feature extraction with prior knowledge of turbulence. We show that it is possible to find a low-rank subspace that not only captures structural features of flows across the Reynolds number but also distinguishes the data source. Along with machine-learning-based super-resolution, we further argue that the present manifold can be used to validate the outcome of modern data-driven techniques when training and evaluating across data sets collected through different techniques. The current approach could serve as a foundation for a range of analyses including reduced-complexity modelling and state estimation with multi-source turbulent flow data.

Moving target segmentation method based on tensor decomposition and graph Laplacian regularization

A new method for segmenting the target foreground from image frames is proposed by utilizing the theory of graph signal processing and the tensor decomposition model aiming at the problem that the segmentation results of the existing foreground segmentation methods in image frames under dynamic scenes are not high in accuracy. The intrinsic connection between image pixels in each frame of an image sequence is modeled as a graph, the image pixel intensities are modeled as graph signals, and the correlation between pixels is characterized by the graph model. According to the significant difference between the dynamic background and the target change in the moving foreground in the image sequence, the dynamic background region in each image frame is smoothed and suppressed, and the disturbing information of the dynamic background is transformed into the useful component information in the low-rank subspace. The connectivity between image pixels can be characterized by the graph Laplacian regularization term, and then the target foreground segmentation problem in the image sequence is equivalent to a constrained optimization problem with tensor decomposition and graph Laplacian regularization term. The alternating direction multiplier method is used to solve the optimization problem, and the simulation results on real scene data set verify the effectiveness of the algorithm.

Dynamic MRI with Locally Low-Rank Subspace Constraint: Towards 1-Second Temporal Resolution Aided by Deep Learning.

MRI is the most effective method for screening high-risk breast cancer patients. While current exams primarily rely on the qualitative evaluation of morphological features before and after contrast administration and less on contrast kinetic information, the latest developments in acquisition protocols aim to combine both. However, balancing between spatial and temporal resolution poses a significant challenge in dynamic MRI. Here, we propose a radial MRI reconstruction framework for Dynamic Contrast Enhanced (DCE) imaging, which offers a joint solution to existing spatial and temporal MRI limitations. It leverages a locally low-rank (LLR) subspace model to represent spatially localized dynamics based on tissue information. Our framework demonstrated substantial improvement in CNR, noise reduction and enables a flexible temporal resolution, ranging from a few seconds to 1-second, aided by a neural network, resulting in images with reduced undersampling penalties. Finally, our reconstruction framework also shows potential benefits for head and neck, and brain MRI applications, making it a viable alternative for a range of DCE-MRI exams.

Semi-Supervised Knee Cartilage Segmentation with Successive Eigen Noise-assisted Mean Teacher Knowledge Distillation.

Knee cartilage segmentation for Knee Osteoarthritis (OA) diagnosis is challenging due to domain shifts from varying MRI scanning technologies. Existing cross-modality approaches often use paired order matching or style translation techniques to align features. Still, these methods can sacrifice discrimination in less prominent cartilages and overlook critical higher-order correlations and semantic information. To address this issue, we propose a novel framework called Successive Eigen Noise-assisted Mean Teacher Knowledge Distillation (SEN-MTKD) for adapting 2D knee MRI images across different modalities using partially labeled data. Our approach includes the Eigen Low-rank Subspace (ELRS) module, which employs low-rank approximations to generate meaningful pseudo-labels from domain-invariant feature representations progressively. Complementing this, the Successive Eigen Noise (SEN) module introduces advanced data perturbation to enhance discrimination and diversity in small cartilage classes. Additionally, we propose a subspace-based feature distillation loss mechanism (LRBD) to manage variance and leverage rich intermediate representations within the teacher model, ensuring robust feature representation and labeling. Our framework identifies a mutual cross-domain subspace using higher-order structures and lower energy latent features, providing reliable supervision for the student model. Extensive experiments on public and private datasets demonstrate the effectiveness of our method over state-of-the-art benchmarks. The code is available at github.com/AmmarKhawer/SEN-MTKD.

TDSF-Net: Tensor Decomposition-Based Subspace Fusion Network for Multimodal Medical Image Classification.

Data from multimodalities bring complementary information for deep learning-based medical image classification models. However, data fusion methods simply concatenating features or images barely consider the correlations or complementarities among different modalities and easily suffer from exponential growth in dimensions and computational complexity when the modality increases. Consequently, this article proposes a subspace fusion network with tensor decomposition (TD) to heighten multimodal medical image classification. We first introduce a Tucker low-rank TD module to map the high-level dimensional tensor to the low-rank subspace, reducing the redundancy caused by multimodal data and high-dimensional features. Then, a cross-tensor attention mechanism is utilized to fuse features from the subspace into a high-dimension tensor, enhancing the representation ability of extracted features and constructing the interaction information among components in the subspace. Extensive comparison experiments with state-of-the-art (SOTA) methods are conducted on one self-established and three public multimodal medical image datasets, verifying the effectiveness and generalization ability of the proposed method. The code is available at https://github.com/1zhang-yi/TDSFNet.

Combining Low-Rank and Deep Plug-and-Play Priors for Snapshot Compressive Imaging.

Snapshot compressive imaging (SCI) is a promising technique that captures a 3-D hyperspectral image (HSI) by a 2-D detector in a compressed manner. The ill-posed inverse process of reconstructing the HSI from their corresponding 2-D measurements is challenging. However, current approaches either neglect the underlying characteristics, such as high spectral correlation, or demand abundant training datasets, resulting in an inadequate balance among performance, generalizability, and interpretability. To address these challenges, in this article, we propose a novel approach called LR2DP that integrates the model-driven low-rank prior and data-driven deep priors for SCI reconstruction. This approach not only captures the spectral correlation and deep spatial features of HSI but also takes advantage of both model-based and learning-based methods without requiring any extra training datasets. Specifically, to preserve the strong spectral correlation of the HSI effectively, we propose that the HSI lies in a low-rank subspace, thereby transforming the problem of reconstructing the HSI into estimating the spectral basis and spatial representation coefficient. Inspired by the mutual promotion of unsupervised deep image prior (DIP) and trained deep denoising prior (DDP), we integrate the unsupervised network and pre-trained deep denoiser into the plug-and-play (PnP) regime to estimate the representation coefficient together, aiming to explore the internal target image prior (learned by DIP) and the external training image prior (depicted by pre-trained DDP) of the HSI. An effective half-quadratic splitting (HQS) technique is employed to optimize the proposed HSI reconstruction model. Extensive experiments on both simulated and real datasets demonstrate the superiority of the proposed method over the state-of-the-art approaches.

Engineering flexible machine learning systems by traversing functionally invariant paths

Contemporary machine learning algorithms train artificial neural networks by setting network weights to a single optimized configuration through gradient descent on task-specific training data. The resulting networks can achieve human-level performance on natural language processing, image analysis and agent-based tasks, but lack the flexibility and robustness characteristic of human intelligence. Here we introduce a differential geometry framework—functionally invariant paths—that provides flexible and continuous adaptation of trained neural networks so that secondary tasks can be achieved beyond the main machine learning goal, including increased network sparsification and adversarial robustness. We formulate the weight space of a neural network as a curved Riemannian manifold equipped with a metric tensor whose spectrum defines low-rank subspaces in weight space that accommodate network adaptation without loss of prior knowledge. We formalize adaptation as movement along a geodesic path in weight space while searching for networks that accommodate secondary objectives. With modest computational resources, the functionally invariant path algorithm achieves performance comparable with or exceeding state-of-the-art methods including low-rank adaptation on continual learning, sparsification and adversarial robustness tasks for large language models (bidirectional encoder representations from transformers), vision transformers (ViT and DeIT) and convolutional neural networks.

Highly accelerated non-contrast-enhanced time-resolved 4D MRA using stack-of-stars golden-angle radial acquisition with a self-calibrated low-rank subspace reconstruction.

To develop a highly accelerated non-contrast-enhanced 4D-MRA technique by combining stack-of-stars golden-angle radial acquisition with a modified self-calibrated low-rank subspace reconstruction. A low-rank subspace reconstruction framework was introduced in radial 4D MRA (SUPER 4D MRA) by combining stack-of-stars golden-angle radial acquisition with control-label k-space subtraction-based low-rank subspace modeling. Radial 4D MRA data were acquired and reconstructed using the proposed technique on 12 healthy volunteers and 1 patient with steno-occlusive disease. The performance of SUPER 4D MRA was compared with two temporally constrained reconstruction methods (golden-angle radial sparse parallel [GRASP] and GRASP-Pro) at different acceleration rates in terms of image quality and delineation of blood dynamics. SUPER 4D MRA outperformed the other two reconstruction methods, offering superior image quality with a clear background and detailed delineation of cerebrovascular structures as well as great temporal fidelity in blood flow dynamics. SUPER 4D MRA maintained excellent performance even at higher acceleration rates. SUPER 4D MRA is a promising technique for highly accelerating 4D MRA acquisition without comprising both temporal fidelity and image quality.

Emotion recognition from multichannel EEG signals based on low-rank subspace self-representation features

Emotion recognition from multichannel EEG signals based on low-rank subspace self-representation features

TAKAGI–SUGENO–KANG FUZZY SYSTEM MODELING BASED ON LOW-RANK SPARSE SUBSPACE LEARNING FOR MOTOR IMAGERY ELECTROENCEPHALOGRAM SIGNAL CLASSIFICATION

The classification of electroencephalogram (EEG) signals derived from motor imagery (MI) has always been a hot topic in the field of brain–computer interfaces. Due to its ability to handle the nonstationary and uncertain information contained in EEG signals, the Takagi–Sugeno–Kang fuzzy system (TSK-FS) has become an advantageous classification algorithm. To train a fuzzy system with strong discrimination capabilities from EEG data interspersed with redundant information, this paper proposes a TSK-FS modeling method based on low-rank sparse subspace learning (TSK-LSSL). This method focuses on consequent parameter learning, which transforms the traditional consequent parameter learning strategy into low-rank subspace and sparse subspace learning processes. Low-rank subspace learning is used to mine the global structural information of data and effectively reduce the number of fuzzy rules. During sparse subspace learning, [Formula: see text]-norm regularization is used to constrain the consequent parameters and causes the number of redundant consequent parameters to be zero, thereby simplifying the fuzzy rules. In addition, a local boundary term based on graph matrices is embedded into the objective function to mine the local structural information of the given data. TSK-LSSL simplifies the number of rules and the consequent part of the fuzzy rules. It exhibits good classification performance on two BCI Competition databases.

Multiview ensemble clustering of hypergraph p-Laplacian regularization with weighting and denoising

Multiview ensemble clustering of hypergraph p-Laplacian regularization with weighting and denoising

Mobile Sensor Path Planning for Kalman Filter Spatiotemporal Estimation.

The estimation of spatiotemporal data from limited sensor measurements is a required task across many scientific disciplines. In this paper, we consider the use of mobile sensors for estimating spatiotemporal data via Kalman filtering. The sensor selection problem, which aims to optimize the placement of sensors, leverages innovations in greedy algorithms and low-rank subspace projection to provide model-free, data-driven estimates. Alternatively, Kalman filter estimation balances model-based information and sparsely observed measurements to collectively make better estimation with limited sensors. It is especially important with mobile sensors to utilize historical measurements. We show that mobile sensing along dynamic trajectories can achieve the equivalent performance of a larger number of stationary sensors, with performance gains related to three distinct timescales: (i) the timescale of the spatiotemporal dynamics, (ii) the velocity of the sensors, and (iii) the rate of sampling. Taken together, these timescales strongly influence how well-conditioned the estimation task is. We draw connections between the Kalman filter performance and the observability of the state space model and propose a greedy path planning algorithm based on minimizing the condition number of the observability matrix. This approach has better scalability and computational efficiency compared to previous works. Through a series of examples of increasing complexity, we show that mobile sensing along our paths improves Kalman filter performance in terms of better limiting estimation and faster convergence. Moreover, it is particularly effective for spatiotemporal data that contain spatially localized structures, whose features are captured along dynamic trajectories.

Precise feature selection via non-convex regularized graph embedding and self-representation for unsupervised learning

Precise feature selection via non-convex regularized graph embedding and self-representation for unsupervised learning

A symmetric low-rank subspace clustering method for cooperative spectrum sensing in complex environments

A symmetric low-rank subspace clustering method for cooperative spectrum sensing in complex environments

Zero-DeepSub: Zero-shot deep subspace reconstruction for rapid multiparametric quantitative MRI using 3D-QALAS.

To develop and evaluate methods for (1) reconstructing 3D-quantification using an interleaved Look-Locker acquisition sequence with T2 preparation pulse (3D-QALAS) time-series images using a low-rank subspace method, which enables accurate and rapid T1 and T2 mapping, and (2) improving the fidelity of subspace QALAS by combining scan-specific deep-learning-based reconstruction and subspace modeling. A low-rank subspace method for 3D-QALAS (i.e., subspace QALAS) and zero-shot deep-learning subspace method (i.e., Zero-DeepSub) were proposed for rapid and high fidelity T1 and T2 mapping and time-resolved imaging using 3D-QALAS. Using an ISMRM/NIST system phantom, the accuracy and reproducibility of the T1 and T2 maps estimated using the proposed methods were evaluated by comparing them with reference techniques. The reconstruction performance of the proposed subspace QALAS using Zero-DeepSub was evaluated in vivo and compared with conventional QALAS at high reduction factors of up to nine-fold. Phantom experiments showed that subspace QALAS had good linearity with respect to the reference methods while reducing biases and improving precision compared to conventional QALAS, especially for T2 maps. Moreover, in vivo results demonstrated that subspace QALAS had better g-factor maps and could reduce voxel blurring, noise, and artifacts compared to conventional QALAS and showed robust performance at up to nine-fold acceleration with Zero-DeepSub, which enabled whole-brain T1, T2, and PD mapping at 1 mm isotropic resolution within 2 min of scan time. The proposed subspace QALAS along with Zero-DeepSub enabled high fidelity and rapid whole-brain multiparametric quantification and time-resolved imaging.

Hyperspectral Compressive Snapshot Reconstruction via Coupled Low-Rank Subspace Representation and Self-Supervised Deep Network.

Coded aperture snapshot spectral imaging (CASSI) is an important technique for capturing three-dimensional (3D) hyperspectral images (HSIs), and involves an inverse problem of reconstructing the 3D HSI from its corresponding coded 2D measurements. Existing model-based and learning-based methods either could not explore the implicit feature of different HSIs or require a large amount of paired data for training, resulting in low reconstruction accuracy or poor generalization performance as well as interpretability. To remedy these deficiencies, this paper proposes a novel HSI reconstruction method, which exploits the global spectral correlation from the HSI itself through a formulation of model-driven low-rank subspace representation and learns the deep prior by a data-driven self-supervised deep learning scheme. Specifically, we firstly develop a model-driven low-rank subspace representation to decompose the HSI as the product of an orthogonal basis and a spatial representation coefficient, then propose a data-driven deep guided spatial-attention network (called DGSAN) to adaptively reconstruct the implicit spatial feature of HSI by learning the deep coefficient prior (DCP), and finally embed these implicit priors into an iterative optimization framework through a self-supervised training way without requiring any training data. Thus, the proposed method shall enhance the reconstruction accuracy, generalization ability, and interpretability. Extensive experiments on several datasets and imaging systems validate the superiority of our method. The source code and data of this article will be made publicly available at https://github.com/ChenYong1993/LRSDN.

SMCC: A Novel Clustering Method for Single- and Multi-Omics Data Based on Co-Regularized Network Fusion.

Clustering is a common technique for statistical data analysis and is essential for developing precision medicine. Numerous computational methods have been proposed for integrating multi-omics data to identify cancer subtypes. However, most existing clustering models based on network fusion fail to preserve the consistency of the distribution of the data before and after fusion. Motivated by this observation, we would like to measure and minimize the distribution difference between networks, which may not be in the same space, to improve the performance of data fusion. We were therefore motivated to develop a flexible clustering model, based on network fusion, that minimizes the distribution difference between the data before and after fusion by co-regularization; the model can be applied to both single- and multi-omics data. We propose a new network fusion model for single- and multi-omics data clustering for identifying cancer or cell subtypes based on co-regularized network fusion (SMCC). SMCC integrates low-rank subspace representation and entropy to fuse networks. In addition, it measures and minimizes the distribution difference between the similarity networks and the fusion network by co-regularization. The model can both reduce the noise interference in the source data and make the statistical characteristics of the fusion result closer to those of the source data. We evaluated the clustering performance of SMCC across 16 real single- and multi-omics dataset. The experimental results demonstrated that SMCC is superior to 17 state-of-the-art clustering methods. Moreover, it is effective for identifying cancer or cell subtypes, thereby promoting the development of precision medicine.

Online Unsupervised Domain Adaptation via Reducing Inter- and Intra-Domain Discrepancies.

Unsupervised domain adaptation (UDA) transfers knowledge from a labeled source domain to an unlabeled target domain on cross-domain object recognition by reducing a distribution discrepancy between the source and target domains (interdomain discrepancy). Prevailing methods on UDA were presented based on the premise that target data are collected in advance. However, in online scenarios, the target data often arrive in a streamed manner, such as visual image recognition in daily monitoring, which means that there is a distribution discrepancy between incoming target data and collected target data (intradomain discrepancy). Consequently, most existing methods need to re-adapt the incoming data and retrain a new model on online data. This paradigm is difficult to meet the real-time requirements of online tasks. In this study, we propose an online UDA framework via jointly reducing interdomain and intradomain discrepancies on cross-domain object recognition where target data arrive in a streamed manner. Specifically, the proposed framework comprises two phases: classifier training and online recognition phases. In the former, we propose training a classifier on a shared subspace where there is a lower interdomain discrepancy between the two domains. In the latter, a low-rank subspace alignment method is introduced to adapt incoming data to the shared subspace by reducing the intradomain discrepancy. Finally, online recognition results can be obtained by the trained classifier. Extensive experiments on DA benchmarks and real-world datasets are employed to evaluate the performance of the proposed framework in online scenarios. The experimental results show the superiority of the proposed framework in online recognition tasks.

Efficient and Effective Nonconvex Low-Rank Subspace Clustering via SVT-Free Operators

With the growing interest in convex and nonconvex low-rank matrix learning problems, the widely used singular value thresholding (SVT) operators associated with rank relaxation functions often face higher computational complexity, particularly for large-scale data matrices. To improve the efficacy of low-rank subspace clustering and overcome the issue of high computational complexity, this work proposes an efficient and effective method that avoids the need for singular value decomposition (SVD) computations in the iteration scheme. This can be achieved through the use of a computationally efficient and compact formulation, as well as automatic removal of the optimal mean, which reduces time consumption and enhances evaluation performance. A unified clustering framework based on Schatten- <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">p</i> norm regularized by ℓ <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2,<i>q</i></sub> -norm can be formulated using this processing way, where inner element suppression can be achieved by choosing appropriate <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">p, q</i> ∈ (0,1). Additionally, calculating the optimal mean enhances the robustness of the proposed method in the presence of outliers. Unlike the general iteration scheme of the alternating direction method of multiplier (ADMM) algorithms that introduce auxiliary splitting variables, the proposed alternating re-weighted least square (ARwLS) algorithm uses matrix inverse and multiplication computations to obtain analytic solutions, resulting in faster processing speeds for each sub-problem. To further investigate, we provide the computational complexity of each iteration and the theoretical analysis of the convergence property, where the derived solution is a stationary point. Experimental results on synthetic data and several benchmark datasets demonstrate the promising efficiency and efficacy of the proposed clustering method compared to classical and competing algorithms.

A multiple kinds of information extraction method for multi-view low-rank subspace clustering

A multiple kinds of information extraction method for multi-view low-rank subspace clustering