Low-rank Model Research Articles

Nonrigid 3D motion estimation at high temporal resolution from prospectively undersampled k-space data using low-rank MR-MOTUS.

PurposeWith the recent introduction of the MR‐LINAC, an MR‐scanner combined with a radiotherapy LINAC, MR‐based motion estimation has become of increasing interest to (retrospectively) characterize tumor and organs‐at‐risk motion during radiotherapy. To this extent, we introduce low‐rank MR‐MOTUS, a framework to retrospectively reconstruct time‐resolved nonrigid 3D+t motion fields from a single low‐resolution reference image and prospectively undersampled k‐space data acquired during motion.TheoryLow‐rank MR‐MOTUS exploits spatiotemporal correlations in internal body motion with a low‐rank motion model, and inverts a signal model that relates motion fields directly to a reference image and k‐space data. The low‐rank model reduces the degrees‐of‐freedom, memory consumption, and reconstruction times by assuming a factorization of space‐time motion fields in spatial and temporal components.MethodsLow‐rank MR‐MOTUS was employed to estimate motion in 2D/3D abdominothoracic scans and 3D head scans. Data were acquired using golden‐ratio radial readouts. Reconstructed 2D and 3D respiratory motion fields were, respectively, validated against time‐resolved and respiratory‐resolved image reconstructions, and the head motion against static image reconstructions from fully sampled data acquired right before and right after the motion.ResultsResults show that 2D+t respiratory motion can be estimated retrospectively at 40.8 motion fields per second, 3D+t respiratory motion at 7.6 motion fields per second and 3D+t head‐neck motion at 9.3 motion fields per second. The validations show good consistency with image reconstructions.ConclusionsThe proposed framework can estimate time‐resolved nonrigid 3D motion fields, which allows to characterize drifts and intra and inter‐cycle patterns in breathing motion during radiotherapy, and could form the basis for real‐time MR‐guided radiotherapy.

NOise reduction with DIstribution Corrected (NORDIC) PCA in dMRI with complex-valued parameter-free locally low-rank processing

Diffusion-weighted magnetic resonance imaging (dMRI) has found great utility for a wide range of neuroscientific and clinical applications. However, high-resolution dMRI, which is required for improved delineation of fine brain structures and connectomics, is hampered by its low signal-to-noise ratio (SNR). Since dMRI relies on the acquisition of multiple different diffusion weighted images of the same anatomy, it is well-suited for denoising methods that utilize correlations across the image series to improve the apparent SNR and the subsequent data analysis. In this work, we introduce and quantitatively evaluate a comprehensive framework, NOise Reduction with DIstribution Corrected (NORDIC) PCA method for processing dMRI. NORDIC uses low-rank modeling of g-factor-corrected complex dMRI reconstruction and non-asymptotic random matrix distributions to remove signal components which cannot be distinguished from thermal noise. The utility of the proposed framework for denoising dMRI is demonstrated on both simulations and experimental data obtained at 3 Tesla with different resolutions using human connectome project style acquisitions. The proposed framework leads to substantially enhanced quantitative performance for estimating diffusion tractography related measures and for resolving crossing fibers as compared to a conventional/state-of-the-art dMRI denoising method.

Bio-Inspired Structure Representation Based Cross-View Discriminative Subspace Learning via Simultaneous Local and Global Alignment

Recently, cross-view feature learning has been a hot topic in machine learning due to the wide applications of multiview data. Nevertheless, the distribution discrepancy between cross-views leads to the fact that instances of the different views from same class are farther than those within the same view but from different classes. To address this problem, in this paper, we develop a novel cross-view discriminative feature subspace learning method inspired by layered visual perception from human. Firstly, the proposed method utilizes a separable low-rank self-representation model to disentangle the class and view structure layers, respectively. Secondly, a local alignment is constructed with two designed graphs to guide the subspace decomposition in a pairwise way. Finally, the global discriminative constraint on distribution center in each view is designed for further alignment improvement. Extensive cross-view classification experiments on several public datasets prove that our proposed method is more effective than other existing feature learning methods.

Single-Cell RNA Sequencing Data Clustering by Low-Rank Subspace Ensemble Framework.

The rapid development of single-cell RNA sequencing (scRNA-seq)technology reveals the gene expression status and gene structure of individual cells, reflecting the heterogeneity and diversity of cells. The traditional methods of scRNA-seq data analysis treat data as the same subspace, and hide structural information in other subspaces. In this paper, we propose a low-rank subspace ensemble clustering framework (LRSEC)to analyze scRNA-seq data. Assuming that the scRNA-seq data exist in multiple subspaces, the low-rank model is used to find the lowest rank representation of the data in the subspace. It is worth noting that the penalty factor of the low-rank kernel function is uncertain, and different penalty factors correspond to different low-rank structures. Moreover, the single cluster model is difficult to find the cellular structure of all datasets. To strengthen the correlation between model solutions, we construct a new ensemble clustering framework LRSEC by using the low-rank model as the basic learner. The LRSEC framework captures the global structure of data through low-rank subspaces, which has better clustering performance than a single clustering model. We validate the performance of the LRSEC framework on seven small datasets and one large dataset and obtain satisfactory results.

Low rank spectral regression via matrix factorization for efficient subspace learning

Subspace learning aims to obtain the corresponding low-dimensional representation of high dimensional data in order to facilitate the subsequent data storage and processing. Graph-based subspace learning is a kind of effective subspace learning methods by modeling the data manifold with a graph, which can be included in the general spectral regression (SR) framework. By using the least square regression form as objective function, spectral regression mathematically avoids performing eign-decomposition on dense matrices and has excellent flexibility. Recently, spectral regression has obtained promising performance in diverse applications; however, it did not take the underlying classes/tasks correlation patterns of data into consideration. In this paper, we propose to improve the performance of spectral regression by exploring the correlation among classes with low-rank modeling. The newly formulated low-rank spectral regression (LRSR) model is achieved by decomposing the projection matrix in SR by two factor matrices which were respectively regularized. The LRSR objective function can be handled by the alternating direction optimization framework. Besides some analysis on the differences between LRSR and existing related models, we conduct extensive experiments by comparing LRSR with its full rank counterpart on benchmark data sets and the results demonstrate its superiority.

MLDH-Fold: Protein fold recognition based on multi-view low-rank modeling

Protein fold recognition is critical for understanding the molecular functions of proteins and drug design. Computational predictors have been proposed to identify protein into one of the known folds based only on the protein sequence information. However, how to combine different features to improve predictive performance remains a challenging problem. In this study, two novel methods (MVLR and MLDH-Fold) were proposed for protein fold recognition. We proposed a novel multi-view learning framework to combine the different views of protein sequences. Each view represents the similarity scores between the target sequences and template sequences calculated by the threading method. The proposed method extracts the low-rank principal features to precisely represent the similarity scores of each view and constructs the latent subspace with the common information of different views to predict the target proteins. Furthermore, we proposed an ensemble method called MLDH-Fold to combine the MVLR with the template-based methods. Predictive results on the two widely used datasets (LE and YK) show that the proposed computational methods outperform other computational predictors, indicating that the MVLR and MLDH-Fold are useful tools for protein fold recognition.

Accelerating T2 mapping of the brain by integrating deep learning priors with low-rank and sparse modeling.

To accelerate T2 mapping with highly sparse sampling by integrating deep learning image priors with low-rank and sparse modeling. The proposed method achieves high-speed T2 mapping by highly sparsely sampling (k, TE)-space. Image reconstruction from the undersampled data was done by exploiting the low-rank structure and sparsity in the T2 -weighted image sequence and image priors learned from training data. The image priors for a single TE were generated from the public Human Connectome Project data using a tissue-based deep learning method; the image priors were then transferred to other TEs using a generalized series-based method. With these image priors, the proposed reconstruction method used a low-rank model and a sparse model to capture subject-dependent novel features. The proposed method was evaluated using experimental data obtained from both healthy subjects and tumor patients using a turbo spin-echo sequence. High-quality T2 maps at the resolution of 0.9 × 0.9 × 3.0 mm3 were obtained successfully from highly undersampled data with an acceleration factor of 8. Compared with the existing compressed sensing-based methods, the proposed method produced significantly reduced reconstruction errors. Compared with the deep learning-based methods, the proposed method recovered novel features better. This paper demonstrates the feasibility of learning T2 -weighted image priors for multiple TEs using tissue-based deep learning and generalized series-based learning. A new method was proposed to effectively integrate these image priors with low-rank and sparse modeling to reconstruct high-quality images from highly undersampled data. The proposed method will supplement other acquisition-based methods to achieve high-speed T2 mapping.

Nonstationary Modeling With Sparsity for Spatial Data via the Basis Graphical Lasso

Many modern spatial models express the stochastic variation component as a basis expansion with random coefficients. Low rank models, approximate spectral decompositions, multiresolution representations, stochastic partial differential equations, and empirical orthogonal functions all fall within this basic framework. Given a particular basis, stochastic dependence relies on flexible modeling of the coefficients. Under a Gaussianity assumption, we propose a graphical model family for the stochastic coefficients by parameterizing the precision matrix. Sparsity in the precision matrix is encouraged using a penalized likelihood framework—we term this approach the basis graphical lasso. Computations follow from a majorization-minimization (MM) approach, a byproduct of which is a connection to the standard graphical lasso. The result is a flexible nonstationary spatial model that is adaptable to very large datasets with multiple realizations. We apply the model to two large and heterogeneous spatial datasets in statistical climatology and recover physically sensible graphical structures. Moreover, the model performs competitively against the popular LatticeKrig model in predictive cross-validation but improves the Akaike information criterion score and a log score for the quality of the joint predictive distribution.

Multi-View Spectral Clustering Tailored Tensor Low-Rank Representation

This paper explores the problem of multi-view spectral clustering (MVSC) based on tensor low-rank modeling. Unlike the existing methods that all adopt an off-the-shelf tensor low-rank norm without considering the special characteristics of the tensor in MVSC, we design a novel structured tensor low-rank norm tailored to MVSC. Specifically, we explicitly impose a symmetric low-rank constraint and a structured sparse low-rank constraint on the frontal and horizontal slices of the tensor to characterize the intra-view and inter-view relationships, respectively. Moreover, the two constraints could be jointly optimized to achieve mutual refinement. On basis of the novel tensor low-rank norm, we formulate MVSC as a convex low-rank tensor recovery problem, which is then efficiently solved with an augmented Lagrange multiplier-based method iteratively. Extensive experimental results on seven commonly used benchmark datasets show that the proposed method outperforms state-of-the-art methods to a significant extent. Impressively, our method is able to produce perfect clustering. In addition, the parameters of our method can be easily tuned, and the proposed model is robust to different datasets, demonstrating its potential in practice. The code is available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/jyh-learning/MVSC-TLRR</uri> .

Rain streaks removal from single image based on texture constraint of background scene

Removing rain streaks from single image is mathematically ill-defined and the known strategies tend to over-smooth the background and remove image details. To tackle this problem, this paper proposes a novel two-stage system. Firstly, in view of the frequency and direction of rain streaks, Gabor filter is first used to suppress rain streaks. Based on the bases extracted from natural images, several Gabor filtered images extracted from the input rain image are fused by a new proposed method, and the fused result provides a texture constraint for the non-rain part. Secondly, a low-rank model is adopted to describe rain streaks as they take on similar and repeated patterns in an imaging scene. By combining the texture constraint and the low-rank model, a global cost function is constructed. Finally, an optimization strategy is proposed based on the split Bregman technique, which solves the function with high performance. A large number of experimental results show the competitive performance and efficiency of the proposed method in comparison with the state-of-the-art methods, making it more applicable in real practices.

Infrared Search and Track With Unbalanced Optimal Transport Dynamics Regularization

Accurate detection of small and dim targets in infrared imagery is a crucial component in infrared search and track which has broad utility in military and remote sensing applications. Low-rank models have enjoyed state-of-the-art performance in infrared tracking applications, but many approaches underutilize dynamics information which has the potential to improve performance in challenging tracking scenarios. We present two algorithms, robust principal components analysis with patched unbalanced optimal transport (RPCA + PUOT) and robust alignment by sparse and low-rank with patched unbalanced optimal transport (RASL + PUOT), which incorporate optimal transport dynamics regularization and demonstrate improved performance on realistic data.

SAR RFI Suppression for Extended Scene Using Interferometric Data via Joint Low-Rank and Sparse Optimization

Radio frequency interference (RFI) can significantly pollute synthetic aperture radar (SAR) data and images, which is also harmful to SAR interferometry (InSAR) for retrieving elevational information. To address this issue, in recent years, a class of advanced RFI suppression methods has been proposed based on narrowband properties of RFI and sparsity assumptions of radar echoes or target reflectivity. However, for SAR echoes and the associated scene reflectivity, these assumptions are usually not feasible when the imaged scene is spatially extended. In view of these problems, this study proposes an InSAR-based RFI suppression method for the case of extended scenes. For this task, we combine the RFI-polluted SAR data with RFI-free interferometric data to form an interferometric SAR data pair. We show that such an InSAR data pair embeds an interferogram having the image amplitude multiplying by a complex exponential interferometric phase. We treat the interferogram as a kind of natural image and use discrete Fourier cosine transform (DCT) for its sparse representation. Then combining the DCT-domain sparsity with low-rank modeling of RFI, we retrieve the interferogram and reconstruct the SAR image via joint low-rank and sparse optimization. Numerical simulations show that the proposed method can effectively recover SAR images and interferometric phases from RFI-polluted SAR data.

Low-rank decomposition on transformed feature maps domain for image denoising

Low-rank based models are proved outstanding for denoising on the data with strong repetitive or redundant property. However, for natural images with complex structures or rich details, the performance drops down because of the weak low-rankness of the data. A feasible solution is to transform the data into a suitable domain to further explore the underlying low-rank information. In this paper, we present a novel approach to create such a domain via a fully replicated linear autoencoder network. By applying various low-rank models to the feature maps generated by the encoder rather than the original data, and then performing inverse transformation by the decoder, their denoising performances all get enhanced. In addition, feature maps also show good sparsity, hence we introduce a new measure combining sparse and low-rank regularity, and further propose corresponding single image denoising model. Extensive experiments show the superiority of our work.

Freeness over the diagonal and outliers detection in deformed random matrices with a variance profile

Abstract We study the eigenvalue distribution of a Gaussian unitary ensemble (GUE) matrix with a variance profile that is perturbed by an additive random matrix that may possess spikes. Our approach is guided by Voiculescu’s notion of freeness with amalgamation over the diagonal and by the notion of deterministic equivalent. This allows to derive a fixed point equation to approximate the spectral distribution of certain deformed GUE matrices with a variance profile and to characterize the location of potential outliers in such models in a non-asymptotic setting. We also consider the singular values distribution of a rectangular Gaussian random matrix with a variance profile in a similar setting of additive perturbation. We discuss the application of this approach to the study of low-rank matrix denoising models in the presence of heteroscedastic noise, that is when the amount of variance in the observed data matrix may change from entry to entry. Numerical experiments are used to illustrate our results. Deformed random matrix, Variance profile, Outlier detection, Free probability, Freeness with amalgamation, Operator-valued Stieltjes transform, Gaussian spiked model, Low-rank model. 2000 Math Subject Classification: 62G05, 62H12.

Robust low rank representation via feature and sample scaling

Low-rank representation (LRR) is a very competitive technique in various real-world applications for its powerful capability in discovering latent structure of noisy or corrupted data set. However, traditional low-rank models treat each data point and feature equally so that noisy data cannot be detected and suppressed effectively and have obvious deterioration in performance, especially in heavy noisy scenario. In this paper, to address this problem, we develop a method of feature and sample scaling for low rank representation. The importance of data points and their features in both feature and sample spaces are considered, as such, clean data points and noisy data points and their features can be distinguished. In addition, based on the observation that noisy data points are usually deviated far away from the principal projection of the data set, a cosine similarity metric between data vector and the principal projection vector is developed to measure the importance of each sample. Applying our method into two classical low rank models such as Low Rank Representation (LRR) and Bilinear Factorization (BF), we can learn better low-rank structure of clean data, while the outliers or missing data being suppressed. Extensive experimental results on ORL, COIL20 and video surveillance, demonstrate that our proposed method can outperform state-of-the-art low rank methods in image clustering tasks with various levels of corruptions, especially in a heavy noisy scenario.

Hyperspectral Image Mixed Noise Removal Based on Multidirectional Low-Rank Modeling and Spatial–Spectral Total Variation

Conventional low-rank (LR)-based hyperspectral image (HSI) denoising models generally convert high-dimensional data into 2-D matrices or just treat this type of data as 3-D tensors. However, these pure LR or tensor low-rank (TLR)-based methods lack flexibility for considering different correlation information from different HSI directions, which leads to the loss of comprehensive structure information and inherent spatial-spectral relationship. To overcome these shortcomings, we propose a novel multidirectional LR modeling and spatial-spectral total variation (MLR-SSTV) model for removing HSI mixed noise. By incorporating the weighted nuclear norm, we obtain the weighted sum of weighted nuclear norm minimization (WSWNNM) and the weighted sum of weighted tensor nuclear norm minimization (WSWTNNM) to estimate the more accurate LR tensor, especially, to remove the dead-line noise better. Gaussian noise is further denoised and the local spatial-spectral smoothness is preserved effectively by SSTV regularization. We develop an efficient algorithm for solving the derived optimization based on the alternating direction method of multipliers (ADMM). Extensive experiments on both synthetic data and real data demonstrate the superior performance of the proposed MLR-SSTV model for HSI mixed noise removal.

A representation transfer learning approach for enhanced prediction of growth hormone binding proteins

Growth hormone binding proteins (GHBPs) are soluble proteins that play an important role in the modulation of signaling pathways pertaining to growth hormones. GHBPs are selective and bind non-covalently with growth hormones, but their functions are still not fully understood. Identification and characterization of GHBPs are the preliminary steps for understanding their roles in various cellular processes. As wet lab based experimental methods involve high cost and labor, computational methods can facilitate in narrowing down the search space of putative GHBPs. Performance of machine learning algorithms largely depends on the quality of features that it feeds on. Informative and non-redundant features generally result in enhanced performance and for this purpose feature selection algorithms are commonly used. In the present work, a novel representation transfer learning approach is presented for prediction of GHBPs. For their accurate prediction, deep autoencoder based features were extracted and subsequently SMO-PolyK classifier is trained. The prediction model is evaluated by both leave one out cross validation (LOOCV) and hold out independent testing set. On LOOCV, the prediction model achieved 89.8%% accuracy, with 89.4% sensitivity and 90.2% specificity and accuracy of 93.5%, sensitivity of 90.2% and specificity of 96.8% is attained on the hold out testing set. Further a comparison was made between the full set of sequence-based features, top performing sequence features extracted using feature selection algorithm, deep autoencoder based features and generalized low rank model based features on the prediction accuracy. Principal component analysis of the representative features along with t-sne visualization demonstrated the effectiveness of deep features in prediction of GHBPs. The present method is robust and accurate and may complement other wet lab based methods for identification of novel GHBPs.

Parametric localisation in frequency diverse array

The authors present a non-traditional, parametric method for velocity, angle, and range estimation in a frequency diverse array radar. Unlike the traditional beamforming techniques, the proposed scheme transmits an omni-directional sinusoid regardless of the target locations. They propose a simple sampling strategy, which eliminates the need for employing a bank of bandpass filters at the receiver. Under the proposed sampling scheme the received data follows a convenient low rank model. They exploit this model to design a fast and accurate parametric estimation algorithm. Their velocity and range estimation steps employ known spectral analysis techniques. For angle estimation, they propose a new grid-less sparse recovery algorithm. The resulting methods are applicable to any arbitrary array geometry. Furthermore, they propose an efficient method to mitigate jamming. They also provide necessary guidelines to avoid ambiguity and achieve the desired resolution performance. The Cramer-Rao lower bound for the estimation problem is derived. The utility of the proposed method is demonstrated via numerical simulation results.

Enhancing robustness of link prediction for noisy complex networks

In recent years, complex networks research have attracted considerable attention, in which link prediction has been taken as an effective tool for predicting missing links or identifying spurious links from available datasets. In reality, the collected networks are always noisy and represent patterns of measured structure, not the true structure of complex systems. However, most of the existing link prediction methods have been developed based on the assumption that the observed networks are reliable. Therefore, how to strengthen the robustness of link prediction methods thereby improving prediction accuracy in noisy networks becomes an important problem. In this paper, we propose a novel link prediction method which combines low-rank representation and non-negative matrix factorization for similarity matrix calculation. The low-rank module decomposes the adjacent matrix of networks into a low rank backbone structure and a sparse noise matrix, and the matrix factorization module characterizes the structural patterns of networks comprehensively with multiple perturbation mechanism. The idea behind the method is that we reshape the original network through the addition and deletion of some links identified by low-rank modeling, thus the important parts of the original network are emphasized. Based on the exaggerated but characteristic network, the potential links may be predicted more accurately. Experimental results on synthetic and real-world networks demonstrate that the proposed method performs better than state-of-the-art methods for link prediction in noisy networks.

Pursuit of Low-Rank Models of Time-Varying Matrices Robust to Sparse and Measurement Noise

In tracking of time-varying low-rank models of time-varying matrices, we present a method robust to both uniformly-distributed measurement noise and arbitrarily-distributed “sparse” noise. In theory, we bound the tracking error. In practice, our use of randomised coordinate descent is scalable and allows for encouraging results on changedetection.net, a benchmark.