Low-rank Model Research Articles

Using Low-rank Multilinear Parameter Identification for Anomaly Detection of Building Systems

The paper proposes a new method for anomaly detection based on multilinear low-rank models. No a priori knowledge about the investigated system is needed for data-driven parameter identification of these models. Multilinear parameter identification is able to cover more dynamic phenomena than linear black box identification. A minimal model of rank 1 has a tiny number of parameters which is equal to the dynamic order plus the number of inputs. These multilinear parameters are moreover directly interpretable as each parameter indicates the influence of one corresponding state or input to the next state of the MTI model. As example, the method is demonstrated by an anomaly detection with real data from the HVAC system of a test room.

Low-Rank Characteristic Tensor Density Estimation Part I: Foundations

Effective non-parametric density estimation is a key challenge in high-dimensional multivariate data analysis. In this paper,we propose a novel approach that builds upon tensor factorization tools. Any multivariate density can be represented by its characteristic function, via the Fourier transform. If the sought density is compactly supported, then its characteristic function can be approximated, within controllable error, by a finite tensor of leading Fourier coefficients, whose size de-pends on the smoothness of the underlying density. This tensor can be naturally estimated from observed realizations of the random vector of interest, via sample averaging. In order to circumvent the curse of dimensionality, we introduce a low-rank model of this characteristic tensor, which significantly improves the density estimate especially for high-dimensional data and/or in the sample-starved regime. By virtue of uniqueness of low-rank tensor decomposition, under certain conditions, our method enables learning the true data-generating distribution. We demonstrate the very promising performance of the proposed method using several measured datasets.

Unsupervised Learning for Automated Detection of Coronary Artery Disease Subgroups.

BackgroundThe promise of precision population health includes the ability to use robust patient data to tailor prevention and care to specific groups. Advanced analytics may allow for automated detection of clinically informative subgroups that account for clinical, genetic, and environmental variability. This study sought to evaluate whether unsupervised machine learning approaches could interpret heterogeneous and missing clinical data to discover clinically important coronary artery disease subgroups.Methods and ResultsThe Genetic Determinants of Peripheral Arterial Disease study is a prospective cohort that includes individuals with newly diagnosed and/or symptomatic coronary artery disease. We applied generalized low rank modeling and K‐means cluster analysis using 155 phenotypic and genetic variables from 1329 participants. Cox proportional hazard models were used to examine associations between clusters and major adverse cardiovascular and cerebrovascular events and all‐cause mortality. We then compared performance of risk stratification based on clusters and the American College of Cardiology/American Heart Association pooled cohort equations. Unsupervised analysis identified 4 phenotypically and prognostically distinct clusters. All‐cause mortality was highest in cluster 1 (oldest/most comorbid; 26%), whereas major adverse cardiovascular and cerebrovascular event rates were highest in cluster 2 (youngest/multiethnic; 41%). Cluster 4 (middle‐aged/healthiest behaviors) experienced more incident major adverse cardiovascular and cerebrovascular events (30%) than cluster 3 (middle‐aged/lowest medication adherence; 23%), despite apparently similar risk factor and lifestyle profiles. In comparison with the pooled cohort equations, cluster membership was more informative for risk assessment of myocardial infarction, stroke, and mortality.ConclusionsUnsupervised clustering identified 4 unique coronary artery disease subgroups with distinct clinical trajectories. Flexible unsupervised machine learning algorithms offer the ability to meaningfully process heterogeneous patient data and provide sharper insights into disease characterization and risk assessment.RegistrationURL: https://www.clinicaltrials.gov; Unique identifier: NCT00380185.

Cross-Domain Object Representation via Robust Low-Rank Correlation Analysis

Cross-domain data has become very popular recently since various viewpoints and different sensors tend to facilitate better data representation. In this article, we propose a novel cross-domain object representation algorithm (RLRCA) which not only explores the complexity of multiple relationships of variables by canonical correlation analysis (CCA) but also uses a low rank model to decrease the effect of noisy data. To the best of our knowledge, this is the first try to smoothly integrate CCA and a low-rank model to uncover correlated components across different domains and to suppress the effect of noisy or corrupted data. In order to improve the flexibility of the algorithm to address various cross-domain object representation problems, two instantiation methods of RLRCA are proposed from feature and sample space, respectively. In this way, a better cross-domain object representation can be achieved through effectively learning the intrinsic CCA features and taking full advantage of cross-domain object alignment information while pursuing low rank representations. Extensive experimental results on CMU PIE, Office-Caltech, Pascal VOC 2007, and NUS-WIDE-Object datasets, demonstrate that our designed models have superior performance over several state-of-the-art cross-domain low rank methods in image clustering and classification tasks with various corruption levels.

Simultaneous Multislice Brain MRI T1 Mapping with Improved Low-Rank Modeling.

To accelerate data acquisition speed in magnetic resonance imaging (MRI), multiple slices are simultaneously acquired using multiband pulses. Simultaneous multislice (SMS) imaging typically unfolds slice aliasing from the acquired collapsed slices. In this study, we extended the SMS framework to accelerated MR parameter quantification such as T1 mapping. Assuming that the slice-specific null space and signal subspace are invariant along the parameter dimension, we formulated the SMS framework as a constrained optimization problem under a joint reconstruction framework such that the noise and signal subspaces are used for slice separation and recovery, respectively. The proposed method was validated on 3T MR human brain scans. We successfully demonstrated that the proposed method outperforms competing methods in suppressing aliasing artifacts and noise at high SMS accelerations, thus leading to accurate T1 maps.

A novel reduced-order method for analysis of hyperspectral images

Hyperspectral imaging technology has been broadly applied in remote sensing because it collects echoed signals from across the electromagnetic (EM) spectrum and provides fruitfully helpful information. However, the processing or transformation of high-data-volume hyperspectral images, also viewed as snapshots varying with the EM spectrum, burdens the hardware resources, especially for the high spectral resolution and spatial resolution cases. To tackle this challenge, a novel reduced-order method based on the dynamic mode decomposition (DMD) algorithm is presented here to analyze hyperspectral images. The method decomposes the spatial-spectral hyperspectral images in terms of spatial dynamic modes and corresponding spectral patterns. Then, these spatial-spectral patterns are utilized to recover the raw hyperspectral images. Our proposed approach is benchmarked by the actual hyperspectral images measured at the Salinas scene. It is demonstrated that the proposed approach can represent the hyperspectral images with a low-rank model in spectral dimension. Our proposed approach could provide a useful tool for the model order reduction of hyperspectral images.

Recommendation System to Predict Missing Adsorption Properties of Nanoporous Materials

Nanoporous materials (NPMs) selectively adsorb and concentrate gases into their pores, and thus could be used to store, capture, and sense many different gases. Modularly synthesized classes of NPMs, such as covalent organic frameworks (COFs), offer a large number of candidate structures for each adsorption task. A complete NPM-property table, containing measurements of the relevant adsorption properties in the candidate NPMs, would enable the matching of NPMs with adsorption tasks. However, in practice the NPM-property matrix is only partially observed (incomplete); (i) many properties of any given NPM have not been measured and (ii) any given property has not been measured for all NPMs.The idea in this work is to leverage the observed (NPM, property) values to impute the missing ones. Similarly, commercial recommendation systems impute missing entries in an incomplete item-customer ratings matrix to recommend items to customers. We demonstrate a COF recommendation system to match COFs with adsorption tasks by training a low rank model of an incomplete COF--adsorption-property matrix. A low rank model, trained on the observed (COF, adsorption property) values, provides (i) predictions of the missing (COF, adsorption property) values and (ii) a map of COFs, wherein COFs with similar (dissimilar) adsorption properties congregate (separate). We find the performance of the COF recommendation system varies for different adsorption tasks and diminishes precipitously when the fraction of missing entries exceeds 60%. The concepts in our COF recommendation system can be applied broadly to many different materials and properties.

Achieving high-resolution 1H-MRSI of the human brain with compressed-sensing and low-rank reconstruction at 7 Tesla

Low sensitivity MR techniques such as magnetic resonance spectroscopic imaging (MRSI) greatly benefit from the gain in signal-to-noise provided by ultra-high field MR. High-resolution and whole-slab brain MRSI remains however very challenging due to lengthy acquisition, low signal, lipid contamination and field inhomogeneity. In this study, we propose an acquisition-reconstruction scheme that combines 1H free-induction-decay (FID)-MRSI sequence, short TR acquisition, compressed sensing acceleration and low-rank modeling with total-generalized-variation constraint to achieve metabolite imaging in two and three dimensions at 7 Tesla. The resulting images and volumes reveal highly detailed distributions that are specific to each metabolite and follow the underlying brain anatomy. The MRSI method was validated in a high-resolution phantom containing fine metabolite structures, and in five healthy volunteers. This new application of compressed sensing acceleration paves the way for high-resolution MRSI in clinical setting with acquisition times of 5 min for 2D MRSI at 2.5 mm and of 20 min for 3D MRSI at 3.3 mm isotropic.

A Novel Low-Rank and Sparse Decomposition Model and Its Application in Moving Objects Detection

A Novel Low-Rank and Sparse Decomposition Model and Its Application in Moving Objects Detection

Learning Mixtures of Low-Rank Models

We study the problem of learning mixtures of low-rank models, i.e. reconstructing multiple low-rank matrices from unlabelled linear measurements of each. This problem enriches two widely studied settings - low-rank matrix sensing and mixed linear regression - by bringing latent variables (i.e. unknown labels) and structural priors (i.e. low-rank structures) into consideration. To cope with the non-convexity issues arising from unlabelled heterogeneous data and low-complexity structure, we develop a three-stage meta-algorithm that is guaranteed to recover the unknown matrices with near-optimal sample and computational complexities under Gaussian designs. In addition, the proposed algorithm is provably stable against random noise. We complement the theoretical studies with empirical evidence that confirms the efficacy of our algorithm.

Rank-Adaptive Tensor Methods for High-Dimensional Nonlinear PDEs

We present a new rank-adaptive tensor method to compute the numerical solution of high-dimensional nonlinear PDEs. The method combines functional tensor train (FTT) series expansions, operator splitting time integration, and a new rank-adaptive algorithm based on a thresholding criterion that limits the component of the PDE velocity vector normal to the FTT tensor manifold. This yields a scheme that can add or remove tensor modes adaptively from the PDE solution as time integration proceeds. The new method is designed to improve computational efficiency, accuracy and robustness in numerical integration of high-dimensional problems. In particular, it overcomes well-known computational challenges associated with dynamic tensor integration, including low-rank modeling errors and the need to invert covariance matrices of tensor cores at each time step. Numerical applications are presented and discussed for linear and nonlinear advection problems in two dimensions, and for a four-dimensional Fokker–Planck equation.

A Novel Ensemble Machine Learning Approach for Bioarchaeological Sex Prediction

I present a novel machine learning approach to predict sex in the bioarchaeological record. Eighteen cranial interlandmark distances and five maxillary dental metric distances were recorded from n = 420 human skeletons from the necropolises at Alfedena (600–400 BCE) and Campovalano (750–200 BCE and 9–11th Centuries CE) in central Italy. A generalized low rank model (GLRM) was used to impute missing data and Area under the Curve—Receiver Operating Characteristic (AUC-ROC) with 20-fold stratified cross-validation was used to evaluate predictive performance of eight machine learning algorithms on different subsets of the data. Additional perspectives such as this one show strong potential for sex prediction in bioarchaeological and forensic anthropological contexts. Furthermore, GLRMs have the potential to handle missing data in ways previously unexplored in the discipline. Although results of this study look promising (highest AUC-ROC = 0.9722 for predicting binary male/female sex), the main limitation is that the sexes of the individuals included were not known but were estimated using standard macroscopic bioarchaeological methods. However, future research should apply this machine learning approach to known-sex reference samples in order to better understand its value, along with the more general contributions that machine learning can make to the reconstruction of past human lifeways.

Image compressive sensing recovery via group residual based nonlocal low-rank regularization

To address the ill-posed nature of image compressive sensing (CS) recovery, it is of great importance to properly exploit image priors. In this paper, we propose a novel group residual based nonlocal low-rank regularization for image CS recovery, namely GRNLR-CS, which performs nonlocal low-rank modeling based on the residual of patch group rather than on the patch group itself as conventional methods. In our GRNLR-CS, the weighted Schatten p-norm is adopted as a non-convex surrogate function to estimate the rank of the group residual. To make our GRNLR-CS method tractable and robust, the split Bregman iteration technique is employed to develop an efficient algorithm to solve the optimization problem of GRNLR-CS. Extensive experimental results demonstrate that our GRNLR-CS method outperforms some state-of-the-art optimization based or deep learning based methods for image CS recovery.

An improved unsupervised learning approach for potential human microRNA–disease association inference using cluster knowledge

Recently, microRNAs (miRNAs) have been shown to play significant roles in the progression of major human diseases. Identifying associations between human diseases and miRNAs using computational tools have attracted considerable attention. Experimental identification and validation of disease–miRNA associations are rare and time consuming. We propose a computational method for predicting associations to understand pathogenicity and uncover prognosis markers. Existing methods employed approaches based on network and machine learning to predict miRNA disease. However, these approaches do not consider the cluster information between miRNAs. Performance results of these methods’ predictions are not satisfactory due to more number of false positives and false negatives. The proposed methodology comprises of two phases: first, we use miRNA cluster information in addition to miRNA-functional similarity and disease semantic similarity. MiRNAs and diseases are placed in the same cluster by transforming the miRNA–disease association data matrix into a low-rank model using Principal Component Analysis (PCA). Second, the Adamic/Adar index was applied that computes the closeness of miRNAs and diseases based on shared neighbors, improving the prediction results. The problem of overestimation is resolved by incorporating similarity information about miRNA and disease. Case studies on Leukemia, Carcinoma, Glioma, Pancreatic Neoplasms, and Melanoma exhibit satisfactory performance with Area Under Curve (AUC) values ranging from 0.736 to 0.834. miRNAs predicted using the proposed method are ascertained to be matching the known associations found in the database of HMDD V3.2, dbDEMC V2.0, and PhenomiR V2.0.

Fast Tensor Nuclear Norm for Structured Low-Rank Visual Inpainting

Low-rank modeling has achieved great success in visual data completion. However, the low-rank assumption of original visual data may be in approximate mode, which leads to suboptimality for the recovery of underlying details, especially when the missing rate is extremely high. In this paper, we go further by providing a detailed analysis about the rank distributions in Hankel structured and clustered cases, and figure out both non-local similarity and patch-based structuralization play a positive role. This motivates us to develop a new Hankel low-rank tensor recovery method that is competent to truthfully capture the underlying details with sacrifice of slightly more computational burden. First, benefiting from the correlation of different spectral bands and the smoothness of local spatial neighborhood, we divide the visual data into overlapping 3D patches and group the similar ones into individual clusters exploring the non-local similarity. Second, the 3D patches are individually mapped to the structured Hankel tensors for better revealing low-rank property of the image. Finally, we solve the tensor completion model via the well-known alternating direction method of multiplier (ADMM) optimization algorithm. Due to the fact that size expansion happens inevitably in Hankelization operation, we further propose a fast randomized skinny tensor singular value decomposition (rst-SVD) to accelerate the per-iteration running efficiency. Extensive experimental results on real world datasets verify the superiority of our method compared to the state-of-the-art visual inpainting approaches.

Low complexity channel estimation algorithm using paired spatial signatures for UAV 3D MIMO systems

This paper proposes a low complexity channel estimation algorithm for unmanned aerial vehicle three dimension multi-user multiple-input-multiple-output (3D MIMO) systems with the uniform planar array (UPA) at base station using paired spatial signatures. With the aid of antenna array theory and array signal processing, 3D channel is firstly modeled based on the angles between the direction of arrival along x- and y-axis of the UPA. And 3D MIMO channels can be projected onto the x- and y-directions, respectively. Then, channel estimation for multi-user uplinks using small amount of training resources is divided into two phases. At the first uplink preamble phase, each user is assigned the orthogonal pilot, and the paired spatial signatures and optimal rotation angle of each user through the same pilot sequence are obtained. We also propose a user grouping strategy based on three-dimension angle-division multiple access (3D-ADMA) to ensure that the user's spatial signatures do not overlap. At the second phase during several coherence times, the same pilot sequence within a group and orthogonal pilot sequences between groups are assigned, then, the channel state information of the user's x- and y-directions are recovered by the paired space signatures and optimal rotation angle of each user obtained in the preamble phase, respectively. And dynamically updating the user's paired spatial signatures and optimal rotation angle utilizes the obtained channel parameter of x- and y-directions. Finally, the channel parameter of the x- and y-directions are reconstructed by the updated user's space signatures and the optimal rotation angle, and the 3D MIMO channel estimation is obtained through the Kronecker product. Compared with the conventional channel estimation method of the 3D MIMO system under UPA using a low-rank model, the proposed methods reduce the computational complexity without degrading the estimated performance to a large extent. Furthermore, it is carried out with limited training resources, and the pilot resource overhead of the system is greatly reduced by the 3D-ADMA packet and the two-stage pilot allocation. Simulation results verified that the proposed algorithm is effective and feasible.

Modal-Regression-Based Structured Low-Rank Matrix Recovery for Multiview Learning.

Low-rank Multiview Subspace Learning (LMvSL) has shown great potential in cross-view classification in recent years. Despite their empirical success, existing LMvSL-based methods are incapable of handling well view discrepancy and discriminancy simultaneously, which, thus, leads to performance degradation when there is a large discrepancy among multiview data. To circumvent this drawback, motivated by the block-diagonal representation learning, we propose structured low-rank matrix recovery (SLMR), a unique method of effectively removing view discrepancy and improving discriminancy through the recovery of the structured low-rank matrix. Furthermore, recent low-rank modeling provides a satisfactory solution to address the data contaminated by the predefined assumptions of noise distribution, such as Gaussian or Laplacian distribution. However, these models are not practical, since complicated noise in practice may violate those assumptions and the distribution is generally unknown in advance. To alleviate such a limitation, modal regression is elegantly incorporated into the framework of SLMR (termed MR-SLMR). Different from previous LMvSL-based methods, our MR-SLMR can handle any zero-mode noise variable that contains a wide range of noise, such as Gaussian noise, random noise, and outliers. The alternating direction method of multipliers (ADMM) framework and half-quadratic theory are used to optimize efficiently MR-SLMR. Experimental results on four public databases demonstrate the superiority of MR-SLMR and its robustness to complicated noise.

On the Synergy between Nonconvex Extensions of the Tensor Nuclear Norm for Tensor Recovery

Low-rank tensor recovery has attracted much attention among various tensor recovery approaches. A tensor rank has several definitions, unlike the matrix rank—e.g., the CP rank and the Tucker rank. Many low-rank tensor recovery methods are focused on the Tucker rank. Since the Tucker rank is nonconvex and discontinuous, many relaxations of the Tucker rank have been proposed, e.g., the sum of nuclear norm, weighted tensor nuclear norm, and weighted tensor schatten-p norm. In particular, the weighted tensor schatten-p norm has two parameters, the weight and p, and the sum of nuclear norm and weighted tensor nuclear norm are special cases of these parameters. However, there has been no detailed discussion of whether the effects of the weighting and p are synergistic. In this paper, we propose a novel low-rank tensor completion model using the weighted tensor schatten-p norm to reveal the relationships between the weight and p. To clarify whether complex methods such as the weighted tensor schatten-p norm are necessary, we compare them with a simple method using rank-constrained minimization. It was found that the simple methods did not outperform the complex methods unless the rank of the original tensor could be accurately known. If we can obtain the ideal weight, p=1 is sufficient, although it is necessary to set p<1 when using the weights obtained from observations. These results are consistent with existing reports.

DLRF-Net: A Progressive Deep Latent Low-Rank Fusion Network for Hierarchical Subspace Discovery

Low-rank coding-based representation learning is powerful for discovering and recovering the subspace structures in data, which has obtained an impressive performance; however, it still cannot obtain deep hidden information due to the essence of single-layer structures. In this article, we investigate the deep low-rank representation of images in a progressive way by presenting a novel strategy that can extend existing single-layer latent low-rank models into multiple layers. Technically, we propose a new progressive Deep Latent Low-Rank Fusion Network (DLRF-Net) to uncover deep features and the clustering structures embedded in latent subspaces. The basic idea of DLRF-Net is to progressively refine the principal and salient features in each layer from previous layers by fusing the clustering and projective subspaces, respectively, which can potentially learn more accurate features and subspaces. To obtain deep hidden information, DLRF-Net inputs shallow features from the last layer into subsequent layers. Then, it aims at recovering the hierarchical information and deeper features by respectively congregating the subspaces in each layer of the network. As such, one can also ensure the representation learning of deeper layers to remove the noise and discover the underlying clean subspaces, which will be verified by simulations. It is noteworthy that the framework of our DLRF-Net is general and is applicable to most existing latent low-rank representation models, i.e., existing latent low-rank models can be easily extended to the multilayer scenario using DLRF-Net. Extensive results on real databases show that our framework can deliver enhanced performance over other related techniques.

Background Modeling Based on Statistical Clustering Partitioning

In order to effectively detect dim-small targets in complex scenes, background suppression is applied to highlight the targets. This paper presents a statistical clustering partitioning low-rank background modeling algorithm (SCPLBMA), which clusters the image into several patches based on image statistics. The image matrix of each patch is decomposed into low-rank matrix and sparse matrix in the SCPLBMA. The background of the original video frames is reconstructed from the low-rank matrices, and the targets can be obtained by subtracting the background. Experiments on different scenes show that the SCPLBMA can effectively suppress the background and textures and equalize the residual noise with gray levels significantly lower than that of the targets. Thus, the difference images obtain good stationary characteristics, and the contrast between the targets and the residual backgrounds is significantly improved. Compared with six other algorithms, the SCPLBMA significantly improved the target detection rates of single-frame threshold segmentation.