Low-rank Constraint Research Articles

Marginal Subspace Learning With Group Low-Rank for Unsupervised Domain Adaptation.

Unsupervised domain adaptation is intended to construct a reliable model for the unlabeled target samples using the well-labeled but differently distributed source samples. To tackle the domain shift issue, learning domain-invariant feature representations across domains is important, and most of the existing methods have concentrated on this goal. However, these methods rarely take into consideration the group discriminability of the feature representation, which is detrimental to the final recognition. Therefore, this article proposes a novel unsupervised domain adaptation method, named marginal subspace learning with group low-rank (MSL-GLR), to extract both domain-invariant and discriminative feature representations. Specifically, MSL-GLR uses the retargeting strategy to relax the regression matrix, such that the regression values would be forced to satisfy a margin maximization criterion for the requirement of correct classification. Moreover, MSL-GLR imposes a class-induced low-rank constraint, which enables the samples of each class to be located in their respective subspace. In this way, the distance between samples from the same class can be decreased and the discriminant ability of the projection is greatly improved. Furthermore, with the help of alternating direction method of multipliers (ADMM), an efficient algorithm is presented to solve the resulting optimization problem. Finally, the effectiveness of the proposed MSL-GLR is demonstrated by comprehensive evaluations on multiple domain adaptation benchmark datasets.

Low-rank prior-based Fast-RPCA for clutter filtering and noise suppression in non-contrast ultrasound microvascular imaging

Accurate and real-time separation of blood signal from clutter and noise signals is a critical step in clinical non-contrast ultrasound microvascular imaging. Despite the widespread adoption of singular value decomposition (SVD) and robust principal component analysis (RPCA) for clutter filtering and noise suppression, the SVD’s sensitivity to threshold selection, along with the RPCA’s limitations in undersampling conditions and heavy computational burden often result in suboptimal performance in complex clinical applications. To address those challenges, this study presents a novel low-rank prior-based fast RPCA (LP-fRPCA) approach to enhance the adaptability and robustness of clutter filtering and noise suppression with reduced computational cost. A low-rank prior constraint is integrated into the non-convex RPCA model to achieve a robust and efficient approximation of clutter subspace, while an accelerated alternating projection iterative algorithm is developed to improve convergence speed and computational efficiency. The performance of the LP-fRPCA method was evaluated against SVD with a tissue/blood threshold (SVD1), SVD with both tissue/blood and blood/noise thresholds (SVD2), and the classical RPCA based on the alternating direction method of multipliers algorithm through phantom and in vivo non-contrast experiments on rabbit kidneys. In the slow flow phantom experiment of 0.2 mm/s, LP-fRPCA achieved an average increase in contrast ratio (CR) of 10.68 dB, 9.37 dB, and 8.66 dB compared to SVD1, SVD2, and RPCA, respectively. In the in vivo rabbit kidney experiment, the power Doppler results demonstrate that the LP-fRPCA method achieved a superior balance in the trade-off between insufficient clutter filtering and excessive suppression of blood flow. Additionally, LP-fRPCA significantly reduced the runtime of RPCA by up to 94-fold. Consequently, the LP-fRPCA method promises to be a potential tool for clinical non-contrast ultrasound microvascular imaging.

Whole brain multiparametric mapping in two minutes using a dual-flip-angle stack-of-stars blipped multi-gradient-echo acquisition

A new MRI technique is presented for three-dimensional fast simultaneous whole brain mapping of myelin water fraction (MWF), T1, proton density (PD), R2*, magnetic susceptibility (QSM), and B1 transmit field (B1+). Phantom and human (N = 9) datasets were acquired using a dual-flip-angle blipped multi-gradient-echo (DFA-mGRE) sequence with a stack-of-stars (SOS) trajectory. Images were reconstructed using a subspace-based algorithm with a locally low-rank constraint. A novel joint-sparsity-constrained multicomponent T2*-T1 spectrum estimation (JMSE) algorithm is proposed to correct for the T1 saturation effect and B1+/B1− inhomogeneities in the quantification of MWF. A tissue-prior-based B1+ estimation algorithm was adapted for B1 correction in the mapping of T1 and PD. In the phantom study, measurements obtained at an acceleration factor (R) of 12 using prospectively under-sampled SOS showed good consistency (R2 > 0.997) with Cartesian reference for R2*/T1app/M0app. In the in vivo study, results of retrospectively under-sampled SOS with R = 6, 12, 18, showed good quality (structure similarity index measure > 0.95) compared with those of fully-sampled SOS. Besides, results of prospectively under-sampled SOS with R = 12 showed good consistency (intraclass correlation coefficient > 0.91) with Cartesian reference for T1/PD/B1+/MWF/QSM/R2*, and good reproducibility (coefficient of variation < 7.0 %) in the test-retest analysis for T1/PD/B1+/MWF/R2*. This study has demonstrated the feasibility of simultaneous whole brain multiparametric mapping with a two-minute scan using the DFA-mGRE SOS sequence, which may overcome a major obstacle for neurological applications of multiparametric MRI.

Comprehensive consensus representation learning for incomplete multiview subspace clustering

Incomplete multiview clustering (IMVC) aims to address clustering problems caused by missing views. While some progress has been made, the current approaches still have some problems. First, the existing IMVC methods excessively rely on the original high-dimensional multiview data, leading to suboptimal results when the data are heavily contaminated. Second, the utilization of high-dimensional data often overlooks the complementarity of and consistency across multiple views. Third, the cluster structure of the input data is not adequately considered, which is detrimental to the resulting clustering performance. To address these problems, we propose a novel method: comprehensive consensus representation learning for incomplete multiview subspace clustering (CLR-IMVC). Specifically, CLR-IMVC assumes that all views are generated from a latent representation space. We model the original incomplete multiple views and the latent space to explore their correlations. Subsequently, we use the latent space to generate complete views and construct a subspace self-representation matrix for each view. Moreover, we innovatively treat the latent representation itself as new a view and likewise extract its self-representation matrix. Furthermore, we construct a third-order tensor by using these self-representation matrices and impose low-rank constraints on the tensor to explore the high-order correlations among the matrices. In addition, a consensus representation learning term is added to explore the consistency information between the views and the latent space, ultimately generating a robust clustering structure. Finally, the proposed method can be effectively solved by the augmented Lagrangian multiplier (ALM). Extensive experiments conducted on diverse datasets demonstrate the effectiveness of CLR-IMVC.

Non-negative consistency affinity graph learning for unsupervised feature selection and clustering

Feature selection plays a crucial role in data mining and pattern recognition tasks. This paper proposes an efficient method for robust unsupervised feature selection, called joint local preserving and low-rank representation with nonnegative and symmetric constraint (JLPLRNS). This approach utilizes an indicator matrix instead of the row sparsity of the projection matrix to directly select some significant features from the original data and adaptively preserves the local geometric structure of original data into the low-dimensional embedded feature subspace via learned indicator matrix. By simultaneously imposing nonnegative symmetric and low-rank constraints on the representation coefficient matrix, it cannot only make this matrix discriminative, sparse and weight consistency for each pair of data, but also uncover the global structure of original data. These effectively will improve clustering performance. An algorithm based on the augmented Lagrange multiplier method with an alternating direction strategy is designed to resolve this model. Experimental results on various real datasets show that the proposed method can effectively identify some important features in data and outperforms many state-of-the-art unsupervised feature selection methods in terms of clustering performance.

Discriminative Subspace Learning With Adaptive Graph Regularization

Abstract Many subspace learning methods based on low-rank representation employ the nearest neighborhood graph to preserve the local structure. However, in these methods, the nearest neighborhood graph is a binary matrix, which fails to precisely capture the similarity between distinct samples. Additionally, these methods need to manually select an appropriate number of neighbors, and they cannot adaptively update the similarity graph during projection learning. To tackle these issues, we introduce Discriminative Subspace Learning with Adaptive Graph Regularization (DSL_AGR), an innovative unsupervised subspace learning method that integrates low-rank representation, adaptive graph learning and nonnegative representation into a framework. DSL_AGR introduces a low-rank constraint to capture the global structure of the data and extract more discriminative information. Furthermore, a novel graph regularization term in DSL_AGR is guided by nonnegative representations to enhance the capability of capturing the local structure. Since closed-form solutions for the proposed method are not easily obtained, we devise an iterative optimization algorithm for its resolution. We also analyze the computational complexity and convergence of DSL_AGR. Extensive experiments on real-world datasets demonstrate that the proposed method achieves competitive performance compared with other state-of-the-art methods.

Compact Model Training by Low-Rank Projection With Energy Transfer.

Low-rankness plays an important role in traditional machine learning but is not so popular in deep learning. Most previous low-rank network compression methods compress networks by approximating pretrained models and retraining. However, the optimal solution in the Euclidean space may be quite different from the one with low-rank constraint. A well-pretrained model is not a good initialization for the model with low-rank constraints. Thus, the performance of a low-rank compressed network degrades significantly. Compared with other network compression methods such as pruning, low-rank methods attract less attention in recent years. In this article, we devise a new training method, low-rank projection with energy transfer (LRPET), that trains low-rank compressed networks from scratch and achieves competitive performance. We propose to alternately perform stochastic gradient descent training and projection of each weight matrix onto the corresponding low-rank manifold. Compared to retraining on the compact model, this enables full utilization of model capacity since solution space is relaxed back to Euclidean space after projection. The matrix energy (the sum of squares of singular values) reduction caused by projection is compensated by energy transfer. We uniformly transfer the energy of the pruned singular values to the remaining ones. We theoretically show that energy transfer eases the trend of gradient vanishing caused by projection. In modern networks, a batch normalization (BN) layer can be merged into the previous convolution layer for inference, thereby influencing the optimal low-rank approximation (LRA) of the previous layer. We propose BN rectification to cut off its effect on the optimal LRA, which further improves the performance. Comprehensive experiments on CIFAR-10 and ImageNet have justified that our method is superior to other low-rank compression methods and also outperforms recent state-of-the-art pruning methods. For object detection and semantic segmentation, our method still achieves good compression results. In addition, we combine LRPET with quantization and hashing methods and achieve even better compression than the original single method. We further apply it in Transformer-based models to demonstrate its transferability. Our code is available at https://github.com/BZQLin/LRPET.

Aligned multi-view clustering for unmapped data via weighted tensor nuclear norm and adaptive graph learning

Multi-view clustering (MVC) is a prevailing clustering method that exploits consistency and complementarity among views to achieve satisfactory performance. The premise of MVC is that the samples of each view are mapped, that is, the same sample is placed in the same position among different views. However, multi-view data collected from the real world would also be unmapped, and thus conventional MVC methods are facing challenges. To address this problem, in our paper, we propose a novel aligned multi-view clustering (A-MVC) method for unmapped data. Specifically, the Calinski–Harabasz index is first employed to find the optimal view, and then the samples in other views are aligned to the samples in the optimal view with the help of the alignment matrix so that the unmapped data are converted into mapped ones. In addition, A-MVC absorbs the advantages of conventional multi-view clustering and employs both weighted tensor nuclear norm (TNN) and graph learning algorithms, which simultaneously learn the global representation and capture the local information. Finally, we integrate the alignment algorithm with a low-rank constraint based on weighted TNN and graph constraints into a unified framework. Extensive experimental results on certain unmapped multi-view datasets show that A-MVC outperforms other state-of-the-art methods. The codes and datasets are available at https://github.com/bingly/A-MVC.

Low-Rank Graph Completion-Based Incomplete Multiview Clustering.

In order to reduce the negative effect of missing data on clustering, incomplete multiview clustering (IMVC) has become an important research content in machine learning. At present, graph-based methods are widely used in IMVC, but these methods still have some defects. First, some of the methods overlook potential relationships across views. Second, most of the methods depend on local structure information and ignore the global structure information. Third, most of the methods cannot use both global structure information and potential information across views to adaptively recover the incomplete relationship structure. To address the above issues, we propose a unified optimization framework to learn reasonable affinity relationships, called low-rank graph completion-based IMVC (LRGR_IMVC). 1) Our method introduces adaptive graph embedding to effectively explore the potential relationship among views; 2) we append a low-rank constraint to adequately exploit the global structure information among views; and 3) this method unites related information within views, potential information across views, and global structure information to adaptively recover the incomplete graph structure and obtain complete affinity relationships. Experimental results on several commonly used datasets show that the proposed method achieves better clustering performance significantly than some of the most advanced methods.

Regularisation constrained denoising discriminant least squares regression for image classification

Discriminative Least Squares Regression (DLSR) is an algorithm that utilizes ε-draggings to expand the distance between classes to better solve multi-classification problems. However, this method increases the differences in regression targets within classes during the classification process, and noise in the samples adversely affects its performance. To address the above problems, we propose a novel method called Regularisation Constrained Denoising Discriminant Least Squares Regression (RCDDLSR). Firstly, the idea of matrix decomposition is introduced and sparse constraints are imposed on the noise matrix for noise elimination, thus helping to maintain the manifold structure of the raw data. Secondly, the neighbourhood relationship between samples of the same category is ensured through graph regularisation to avoid overfitting. In addition, we introduce low-rank constraints in the labelling space to better capture the local structure of the regression target within the class. Finally, our experiments show that the method performs better in various types of image datasets compared to other different algorithms.

Low-rank representation induced missing-view recovery for incomplete multi-view clustering

In practical applications, multi-view data suffers from incompleteness, which refers to the absence of some samples in each view. Incomplete multi-view clustering is developed to capture the common cluster structure across incomplete views. However, some existing IMC methods ignore the missing samples or the latent relationship between the available and the unavailable samples. We propose a novel missing-view recovery strategy to overcome these two drawbacks. Considering that each missing sample and part of the observable samples belong to the same category, we recover each missing sample via the linear combination of observed samples within each view. Based on the semantic consistency, we extract a unified self-representation graph from complete views and impose a low-rank constraint on it. Furthermore, we learn a comprehensive representation from complete multi-view data and introduce the dynamic weights to consider the importance of various views. The experimental results on several datasets demonstrate that the proposed method outperforms the state-of-the-art IMC approaches.

Low-rank reconstruction for simultaneous double half-echo 23Na and undersampled 23Na multi-quantum coherences MRI.

To develop a new sequence to simultaneously acquire Cartesian sodium (23Na) MRI and accelerated Cartesian single (SQ) and triple quantum (TQ) sodium MRI of in vivo human brain at 7 T by leveraging two dedicated low-rank reconstruction frameworks. The Double Half-Echo technique enables short echo time Cartesian 23Na MRI and acquires two k-space halves, reconstructed by a low-rank coupling constraint. Additionally, three-dimensional (3D) 23Na Multi-Quantum Coherences (MQC) MRI requires multi-echo sampling paired with phase-cycling, exhibiting a redundant multidimensional space. Simultaneous Autocalibrating and k-Space Estimation (SAKE) were used to reconstruct highly undersampled 23Na MQC MRI. Reconstruction performance was assessed against five-dimensional (5D) CS, evaluating structural similarity index (SSIM), root mean squared error (RMSE), signal-to-noise ratio (SNR), and quantification of tissue sodium concentration and TQ/SQ ratio in silico, in vitro, and in vivo. The proposed sequence enabled the simultaneous acquisition of fully sampled 23Na MRI while leveraging prospective undersampling for 23Na MQC MRI. SAKE improved TQ image reconstruction regarding SSIM by 6% and reduced RMSE by 35% compared to 5D CS in vivo. Thanks to prospective undersampling, the spatial resolution of 23Na MQC MRI was enhanced from mm3 to mm3 while reducing acquisition time from min to min. The proposed sequence, coupled with low-rank reconstructions, provides an efficient framework for comprehensive whole-brain sodium MRI, combining TSC, T2*, and TQ/SQ ratio estimations. Additionally, low-rank matrix completion enables the reconstruction of highly undersampled 23Na MQC MRI, allowing for accelerated acquisition or enhanced spatial resolution.

Multi-View Subspace Clustering via Structured Multi-Pathway Network.

Recently, deep multi-view clustering (MVC) has attracted increasing attention in multi-view learning owing to its promising performance. However, most existing deep multi-view methods use single-pathway neural networks to extract features of each view, which cannot explore comprehensive complementary information and multilevel features. To tackle this problem, we propose a deep structured multi-pathway network (SMpNet) for multi-view subspace clustering task in this brief. The proposed SMpNet leverages structured multi-pathway convolutional neural networks to explicitly learn the subspace representations of each view in a layer-wise way. By this means, both low-level and high-level structured features are integrated through a common connection matrix to explore the comprehensive complementary structure among multiple views. Moreover, we impose a low-rank constraint on the connection matrix to decrease the impact of noise and further highlight the consensus information of all the views. Experimental results on five public datasets show the effectiveness of the proposed SMpNet compared with several state-of-the-art deep MVC methods.

A traffic flow prediction method based on constrained dynamic graph convolutional recurrent networks

To enhance the efficiency of predicting future traffic flow trends within a transportation network, we propose a traffic flow prediction approach based on Constrained Dynamic Graph Convolutional Recurrent Network (C-DGCRN). Firstly, we employ matrix linear constraint and low-rank constraint optimization techniques to assist the Dynamic Graph Convolutional Recurrent Network (DGCRN) in extracting relevant internal correlations within the transportation network, leading to the formation of an affinity matrix. Secondly, we utilize the Hadamard product operation to effectively transfer weights from the affinity matrix to DGCRN, further enhancing its predictive accuracy. The experiments conducted an error assessment on the Metro Traffic Los Angeles and Performance Measurement System-Bay Area datasets. The Mean Absolute Error, Root Mean Square Error, and Mean Absolute Percentage Error of C-DGCRN were 2.66, 5.06, and 6.83, respectively. Compared to state-of-the-art methods like Adaptive Graph Convolutional Recurrent Network, C-DGCRN showed a reduction of 7%, 9%, and 11% in these metrics.

Accelerated multi-b-value multi-shot diffusion-weighted imaging based on EPI with keyhole and a low-rank tensor constraint

PurposeMulti-Shot (MS) Echo-Planar Imaging (EPI) may improve the in-plane resolution of multi-b-value DWI, yet it also considerably increases the scan time. Here we explored the combination of EPI with Keyhole (EPIK) and a calibrationless reconstruction algorithm for acceleration of multi-b-value MS-DWI. MethodsWe firstly analyzed the impact of nonuniform phase accrual in EPIK on the reconstructed image. Based on insights gained from the analysis, we developed a calibrationless reconstruction algorithm based on a Space-Contrast-Coil Locally Low-Rank Tensor (SCC-LLRT) constraint for reconstruction of EPIK-acquired data. We compared the algorithm with a modified SPatial-Angular Locally Low-Rank (SPA-LLR) algorithm through simulations, phantoms, and in vivo study. We then compared EPIK with uniformly undersampled EPI for accelerating multi-b-value DWI in 6 healthy subjects. ResultsThrough theoretical derivations, we found that the reconstruction of EPIK with a SENSE-encoding-based algorithm, such as SPA-LLR, may cause additional aliasing artifacts due to the frequency-dependent distortion of the coil sensitivity. Results from simulations, phantoms, and in vivo study verified the theoretical finding by showing that the calibrationless SCC-LLRT algorithm reduced aliasing artifacts compared with SPA-LLR. Finally, EPIK with SCC-LLRT substantially reduced the ghosting artifacts compared with uniform undersampled multi-b-value DWI, decreasing the fitting errors in ADC (0.05 ± 0.01 vs 0.10 ± 0.01, P < 0.001) and IVIM mapping (0.026 ± 0.004 vs 0.06 ± 0.006, P < 0.001). ConclusionThe SCC-LLRT algorithm reduced the aliasing artifacts of EPIK by using a calibrationless modeling of the multi-coil data. The dense sampling of k-space center offers EPIK a potential to improve image quality for acceleration of multi-b-value MS-DWI.

High-dimensional covariate-augmented overdispersed poisson factor model.

The current Poisson factor models often assume that the factors are unknown, which overlooks the explanatory potential of certain observable covariates. This study focuses on high dimensional settings, where the number of the count response variables and/or covariates can diverge as the sample size increases. A covariate-augmented overdispersed Poisson factor model is proposed to jointly perform a high-dimensional Poisson factor analysis and estimate a large coefficient matrix for overdispersed count data. A group of identifiability conditions is provided to theoretically guarantee computational identifiability. We incorporate the interdependence of both response variables and covariates by imposing a low-rank constraint on the large coefficient matrix. To address the computation challenges posed by nonlinearity, two high-dimensional latent matrices, and the low-rank constraint, we propose a novel variational estimation scheme that combines Laplace and Taylor approximations. We also develop a criterion based on a singular value ratio to determine the number of factors and the rank of the coefficient matrix. Comprehensive simulation studies demonstrate that the proposed method outperforms the state-of-the-art methods in estimation accuracy and computational efficiency. The practical merit of our method is demonstrated by an application to the CITE-seq dataset. A flexible implementation of our proposed method is available in the R package COAP.

Mixed Matrix Completion in Complex Survey Sampling under Heterogeneous Missingness

Modern surveys with large sample sizes and growing mixed-type questionnaires require robust and scalable analysis methods. In this work, we consider recovering a mixed dataframe matrix, obtained by complex survey sampling, with entries following different canonical exponential distributions and subject to heterogeneous missingness. To tackle this challenging task, we propose a two-stage procedure: in the first stage, we model the entry-wise missing mechanism by logistic regression, and in the second stage, we complete the target parameter matrix by maximizing a weighted log-likelihood with a low-rank constraint. We propose a fast and scalable estimation algorithm that achieves sublinear convergence, and the upper bound for the estimation error of the proposed method is rigorously derived. Experimental results support our theoretical claims, and the proposed estimator shows its merits compared to other existing methods. The proposed method is applied to analyze the National Health and Nutrition Examination Survey data. Supplementary materials for this article are available online.

Low-Rank Kernel Tensor Learning for Incomplete Multi-View Clustering

Incomplete Multiple Kernel Clustering algorithms, which aim to learn a common latent representation from pre-constructed incomplete multiple kernels from the original data, followed by k-means for clustering. They have attracted intensive attention due to their high computational efficiency. However, our observation reveals that the imputation of these approaches for each kernel ignores the influence of other incomplete kernels. In light of this, we present a novel method called Low-Rank Kernel Tensor Learning for Incomplete Multiple Views Clustering (LRKT-IMVC) to address the above issue. Specifically, LRKT-IMVC first introduces the concept of kernel tensor to explore the inter-view correlations, and then the low-rank kernel tensor constraint is used to further capture the consistency information to impute missing kernel elements, thereby improving the quality of clustering. Moreover, we carefully design an alternative optimization method with promising convergence to solve the resulting optimization problem. The proposed method is compared with recent advances in experiments with different missing ratios on seven well-known datasets, demonstrating its effectiveness and the advantages of the proposed interpolation method.

A novel multi-task TSK fuzzy system modeling method based on multi-task fuzzy clustering

The traditional multi-task Takagi-Sugeno-Kang (TSK) fuzzy system modeling methods pay more attention to utilizing the inter-task correlation to learn the consequent parameters but ignore the importance of the antecedent parameters of the model. To this end, we propose a novel multi-task TSK fuzzy system modeling method based on multi-task fuzzy clustering. This method first proposes a novel multi-task fuzzy c-means clustering method that learns multiple specific clustering centers for each task and some common clustering centers for all tasks. Secondly, for the consequent parameters of the fuzzy system, the novel low-rank and row-sparse constraints are proposed to better implement multi-task learning. The experimental results demonstrate that the proposed model shows better performance compared with other existing methods.

Compressive sensing ISAR imaging with low-rank constraint and anisotropic spatial total variation processing

In order to suppress random noise and remove stripe interference in ISAR imaging, a Compressive Sensing method is proposed for super-resolution ISAR imaging in this paper, which is named LR-ASTV (Low-rank and Anisotropic Spatial Total Variation) algorithm here. In this algorithm, the original echo HRRP is transformed into echo HRRP of MMV model with radial interpolation processing at first. Subsequently, an optimization objective function based on Compressive Sensing framework is formulated which includes the sparsity constraint and low-rank constraint, and this optimization problem is solved by JLRS algorithm to generate an initial ISAR image from which the random noise has been eliminated. The initial image is then subjected to further refinement using the Anisotropic Spatial Total Variation (ASTV) processing, ultimately yielding the final ISAR image with stripe interference removed. To validate the effectiveness of the LR-ASTV algorithm, ISAR imaging experiments based on simulated and measured data at different SNRs are completed, and the imaging results of the LR-ASTV algorithm are compared with those of other three algorithms including the LSM-ME2 algorithm, the JLRS algorithm and the LRPB algorithm. It can be found that the LR-ASTV algorithm has obvious superiority in suppressing random noise and removing stripe interference, and can provide ISAR images of higher clarity. The quality evaluation for ISAR images also shows that the LR-ASTV algorithm has lower image entropy index and higher image contrast index than the other three ISAR imaging algorithms.