Low-rank Model Research Articles

Coherent Set Identification Via Direct Low Rank Maximum Likelihood Estimation

We analyze connections between two low rank modeling approaches from the last decade for treating dynamical data. The first one is the coherence problem (or coherent set approach), where groups of states are sought that evolve under the action of a stochastic transition matrix in a way maximally distinguishable from other groups. The second one is a low rank factorization approach for stochastic matrices, called direct Bayesian model reduction (DBMR), which estimates the low rank factors directly from observed data. We show that DBMR results in a low rank model that is a projection of the full model, and exploit this insight to infer bounds on a quantitative measure of coherence within the reduced model. Both approaches can be formulated as optimization problems, and we also prove a bound between their respective objectives. On a broader scope, this work relates the two classical loss functions of nonnegative matrix factorization, namely the Frobenius norm and the generalized Kullback–Leibler divergence, and suggests new links between likelihood-based and projection-based estimation of probabilistic models.

Spatiotemporal implicit neural representation as a generalized traffic data learner

Spatiotemporal Traffic Data (STTD) measures the complex dynamical behaviors of the multiscale transportation system. Existing methods aim to reconstruct STTD using low-dimensional models. However, they are limited to data-specific dimensions or source-dependent patterns, restricting them from unifying representations. Here, we present a novel paradigm to address the STTD learning problem by parameterizing STTD as an implicit neural representation. To discern the underlying dynamics in low-dimensional regimes, coordinate-based neural networks that can encode high-frequency structures are employed to directly map coordinates to traffic variables. To unravel the entangled spatial–temporal interactions, the variability is decomposed into separate processes. We further enable modeling in irregular spaces such as sensor graphs using spectral embedding. Through continuous representations, our approach enables the modeling of a variety of STTD with a unified input, thereby serving as a generalized learner of the underlying traffic dynamics. It is also shown that it can learn implicit low-rank priors and smoothness regularization from the data, making it versatile for learning different dominating data patterns. We validate its effectiveness through extensive experiments in real-world scenarios, showcasing applications from corridor to network scales. Empirical results not only indicate that our model has significant superiority over conventional low-rank models, but also highlight that the versatility of the approach extends to different data domains, output resolutions, and network topologies. Comprehensive model analyses provide further insight into the inductive bias of STTD. We anticipate that this pioneering modeling perspective could lay the foundation for universal representation of STTD in various real-world tasks. PyTorch implementations of this project is publicly available at:https://github.com/tongnie/traffic_dynamics.

Effective modeling of open quantum systems by low-rank discretization of structured environments.

The accurate description of the interaction of a quantum system with its environment is a challenging problem ubiquitous across all areas of physics and lies at the foundation of quantum mechanics theory. Here, we pioneer a new strategy to create discrete low-rank models of the system-environment interaction, by exploiting the frequency and time domain information encoded in the fluctuation-dissipation relation connecting the system-bath correlation function and the spectral density. We demonstrate the effectiveness of our methodology by combining it with tensor-network methodologies and simulating the quantum dynamics of complex excitonic systems in a highly structured bosonic environment. The new modeling framework sets the basis for a leap in the analysis of open quantum systems, providing controlled accuracy at significantly reduced computational costs, with benefits in all connected research areas.

A Hierarchical Low-Rank Denoising Model for Remote Sensing Images Based on Deep Unfolding.

Recently, the low-rank representation (LRR) model has been widely used in the field of remote sensing image denoising due to its excellent noise suppression capability. However, those low-rank-based methods always discard important edge details as residuals, leading to a common issue of blurred edges in denoised results. To address this problem, we take a new look at low-rank residuals and try to extract edge information from them. Therefore, a hierarchical denoising framework was combined with a low-rank model to extract edge information from low-rank residuals within the edge subspace. A prior knowledge matrix was designed to enable the model to learn necessary structural information rather than noise. Also, such traditional model-driven approaches require multiple iterations, and the solutions may be very complex and computationally intensive. To further enhance the noise suppression performance and computing efficiency, a hierarchical low-rank denoising model based on deep unrolling (HLR-DUR) was proposed, integrating deep neural networks into the hierarchical low-rank denoising framework to expand the information capture and representation capabilities of the proposed shallow model. Sufficient experiments on optical images, hyperspectral images (HSI), and synthetic aperture radar (SAR) images showed that HLR-DUR achieved state-of-the-art (SOTA) denoising results.

Vibration shock disturbance modeling in the rotating machinery fault diagnosis: A generalized mixture Gaussian model

In real-world industrial environments, complex background noises are composed of various components, deviating from a simple Gaussian distribution like shock noise. In this work, a robust noise modeling method based on the mixture of exponential power (MoEP) distributions is developed to address this issue. To proficiently extract the fault characteristics, the signal’s 2-D representation is attained via Fast-SC, both of the fault features’ low-rankness and the complex noise are combined in a signal model. Then, a penalized function of the noise model is combined to further improve the performance of the method. The model is designated as the PMoEP enhanced low-rank model (PMoEP-LR). The Generalized Expectation–Maximization (GEM) algorithm is utilized to estimate the low-rank spectral correlation matrix and deduce all parameters of the PMoEP-LR model. Finally, the enhanced envelope spectrum (EES) is used to detect the defect characteristic. The efficacy of the proposed method is showcased by analyzing both simulated and real data.

Acoustic resolvent analysis of turbulent jets

Abstract We perform a resolvent analysis of a compressible turbulent jet, where the optimisation domain of the response modes is located in the acoustic field, excluding the hydrodynamic region, in order to promote acoustically efficient modes. We examine the properties of the acoustic resolvent and assess its potential for jet-noise modelling, focusing on the subsonic regime. Resolvent forcing modes, consistent with previous studies, are found to contain supersonic waves associated with Mach wave radiation in the response modes. This differs from the standard resolvent in which hydrodynamic instabilities dominate. We compare resolvent modes with SPOD modes educed from LES data. Acoustic resolvent response modes generally have better alignment with acoustic SPOD modes than standard resolvent response modes. For the optimal mode, the angle of the acoustic beam is close to that found in SPOD modes for moderate frequencies. However, there is no significant separation between the singular values of the leading and sub-optimal modes. Some suboptimal modes are furthermore shown to contain irrelevant structure for jet noise. Thus, even though it contains essential acoustic features absent from the standard resolvent approach, the SVD of the acoustic resolvent alone is insufficient to educe a low-rank model for jet noise. But because it identifies the prevailing mechanisms of jet noise, it provides valuable guidelines in the search of a forcing model (Karban et al. in J Fluid Mech 965:18, 2023). Graphical abstract

3D Poissonian image deblurring via patch-based tensor logarithmic Schatten-p minimization

In medical and biological image processing, multi-dimensional images are often corrupted by blur and Poisson noise. In this paper, we first propose a new tensor logarithmic Schatten-p (t-log-Sp ) low-rank measure and a tensor iteratively reweighted Schatten-p minimization algorithm for minimizing such measure. Furthermore, we adopt this low-rank measure to regularize the non-local tensors formed by similar 3D image patches and develop a patch-based non-local low-rank model. The data fidelity term of the model characterizes the Poisson noise distribution and blur operator. The optimization model is further solved by an alternating minimization technique combined with variable splitting. Experimental results tested on 3D fluorescence microscope images show that the proposed patch-based tensor logarithmic Schatten-p minimization method outperforms state-of-the-art methods in terms of image evaluation metrics and visual quality.

A theory of optimal convex regularization for low-dimensional recovery

Abstract We consider the problem of recovering elements of a low-dimensional model from under-determined linear measurements. To perform recovery, we consider the minimization of a convex regularizer subject to a data fit constraint. Given a model, we ask ourselves what is the ‘best’ convex regularizer to perform its recovery. To answer this question, we define an optimal regularizer as a function that maximizes a compliance measure with respect to the model. We introduce and study several notions of compliance. We give analytical expressions for compliance measures based on the best-known recovery guarantees with the restricted isometry property. These expressions permit to show the optimality of the $\ell ^{1}$-norm for sparse recovery and of the nuclear norm for low-rank matrix recovery for these compliance measures. We also investigate the construction of an optimal convex regularizer using the examples of sparsity in levels and of sparse plus low-rank models.

NSDIE: Noise Suppressing Dark Image Enhancement Using Multiscale Retinex and Low-Rank Minimization

It is inevitable for dark images to have crucial information obscured by low-light conditions, which are worsened by the presence of noise in these images. This work introduces a groundbreaking solution, Noise-Suppressing Dark Image Enhancement for Web Apps (NSDIE), to address the challenging task of enhancing low-light images marred by noise. The proposed work utilizes a low-rank model with simultaneous enhancement of reflectance and illumination components to improve the nighttime scenes while also eradicating the present noise of the image. The reflectance component is further processed using a multiscale retinex model to compensate for the possible color distortions while the illumination component is enhanced using the camera response model to ensure the genuineness of the scene. The proposed work is also tested for a standalone application and is presented to the user through a web portal to aid the concerns of dark image enhancement in the daily life of the user. Rigorous quantitative and qualitative analyses assert NSDIE's superiority over existing techniques, establishing its pivotal role in addressing the critical concern of dark image enhancement.

The WIPI Model Based on Multi-Scale Local Contrast Post-Processing for Infrared Small Target Detection

According to the infrared patch image (IPI) model theory, the infrared image background has a low rank and the target is sparse. The low-rank model can be used to separate the background and identify the target. However, in a noisy environment, the recognition effect will be affected. The higher the noise, the harder it would be to detect a small target. The residual strong fault and background edges could reduce the detection rate and increase false alarms. The traditional IPI model is adaptable to the background with the lower noise. This paper combines weighted nuclear norm minimization (WNNM) optimization with sparse representation based on the local IPI model. The background details are described more prominently by improving the nuclear norm weighting factor. The target is much easier to detect under the specific bright clouds and ground buildings background with high noise. At the same time, post-processing with image local contrast analysis is performed to compare traditional spatial filtering and local infrared patch image model algorithms. Our method has a good suppression effect on complex noise backgrounds and achieves a higher signal to clutter ratio gain (SCRG). It could also improve the target detection rate and reduce false alarms.

A reduced order model to assist welding parameter setup

This article demonstrates the time saving in industrial process setup using numerical reduced order modelling (ROM). The numerical simulations may supply useful information to design manufacturing processes but are often time consuming and then not suited with multi-query study such as inverse problem. ROM aims at replacing the original simulation (so-called high-fidelity (HF)) by a low rank model that will run fast according to the HF simulation time. Multi-query studies will use the ROM instead of the HF simulation to save time and deliver a solution consistent with industrial timeline. To validate the solution, the HF simulation will be used. In this paper, a problem of calibration of TIG welding parameter will illustrate this approach.

LR Aerial Photo Categorization by Cross-Resolution Perceptual Knowledge Propagation.

There are hundreds of high-and low-altitude earth observation satellites that asynchronously capture massive-scale aerial photographs every day. Generally, high-altitude satellites take low-resolution (LR) aerial pictures, each covering a considerably large area. In contrast, low-altitude satellites capture high-resolution (HR) aerial photos, each depicting a relatively small area. Accurately discovering the semantics of LR aerial photos is an indispensable technique in computer vision. Nevertheless, it is also a challenging task due to: 1) the difficulty to characterize human hierarchical visual perception and 2) the intolerable human resources to label sufficient training data. To handle these problems, a novel cross-resolution perceptual knowledge propagation (CPKP) framework is proposed, focusing on adapting the visual perceptual experiences deeply learned from HR aerial photos to categorize LR ones. Specifically, by mimicking the human vision system, a novel low-rank model is designed to decompose each LR aerial photo into multiple visually/semantically salient foreground regions coupled with the background nonsalient regions. This model can: 1) produce a gaze-shifting path (GSP) simulating human gaze behavior and 2) engineer the deep feature for each GSP. Afterward, a kernel-induced feature selection (FS) algorithm is formulated to obtain a succinct set of deep GSP features discriminative across LR and HR aerial photos. Based on the selected features, the labels from LR and HR aerial photos are collaboratively utilized to train a linear classifier for categorizing LR ones. It is worth emphasizing that, such a CPKP mechanism can effectively optimize the linear classifier training, as labels of HR aerial photos are acquired more conveniently in practice. Comprehensive visualization results and comparative study have validated the superiority of our approach.

RED-PSM: Regularization by Denoising of Factorized Low Rank Models for Dynamic Imaging

RED-PSM: Regularization by Denoising of Factorized Low Rank Models for Dynamic Imaging

A New Statistic for Testing Covariance Equality in High-Dimensional Gaussian Low-Rank Models

A New Statistic for Testing Covariance Equality in High-Dimensional Gaussian Low-Rank Models

Minimizing Low-Rank Models of High-Order Tensors: Hardness, Span, Tight Relaxation, and Applications

Minimizing Low-Rank Models of High-Order Tensors: Hardness, Span, Tight Relaxation, and Applications

Spatial Correlation-Constrained Low-Rank Modeling for SAR Image Change Detection

Spatial Correlation-Constrained Low-Rank Modeling for SAR Image Change Detection

New Theory and Faster Computations for Subspace-Based Sensitivity Map Estimation in Multichannel MRI.

Sensitivity map estimation is important in many multichannel MRI applications. Subspace-based sensitivity map estimation methods like ESPIRiT are popular and perform well, though can be computationally expensive and their theoretical principles can be nontrivial to understand. In the first part of this work, we present a novel theoretical derivation of subspace-based sensitivity map estimation based on a linear-predictability/structured low-rank modeling perspective. This results in an estimation approach that is equivalent to ESPIRiT, but with distinct theory that may be more intuitive for some readers. In the second part of this work, we propose and evaluate a set of computational acceleration approaches (collectively known as PISCO) that can enable substantial improvements in computation time (up to ∼100× in the examples we show) and memory for subspace-based sensitivity map estimation.

Deep image prior cine MR fingerprinting with B1 + spin history correction.

To develop a deep image prior (DIP) reconstruction for B1 + -corrected 2D cine MR fingerprinting (MRF). The proposed method combines low-rank (LR) modeling with a DIP to generate cardiac phase-resolved parameter maps without motion correction, employing self-supervised training to enforce consistency with undersampled spiral k-space data. Two implementations were tested: one approach (DIP) for cine T1 , T2 , and M0 mapping, and a second approach (DIP with effective B1 + estimation [DIP-B1]) that also generated an effective B1 + map to correct for errors due to RF transmit inhomogeneities, through-plane motion, and blood flow. Cine MRF data were acquired in 14 healthy subjects and four reconstructions were compared: LR, low-rank motion-corrected (LRMC), DIP, and DIP-B1. Results were compared to diastolic ECG-triggered MRF, MOLLI, and T2 -prep bSSFP. Additionally, bright-blood and dark-blood images calculated from cine MRF maps were used to quantify ventricular function and compared to reference cine measurements. DIP and DIP-B1 outperformed other cine MRF reconstructions with improved noise suppression and delineation of high-resolution details. Within-segment variability in the myocardium (reported as the coefficient of variation for T1 /T2 ) was lowest for DIP-B1 (2.3/8.3%) followed by DIP (2.7/8.7%), LRMC (3.5/10.5%), and LR (15.3/39.6%). Spatial homogeneity improved with DIP-B1 having the lowest intersegment variability (2.6/4.1%). The mean bias in ejection fraction was -1.1% compared to reference cine scans. A DIP reconstruction for 2D cine MRF enabled cardiac phase-resolved mapping of T1 , T2 , M0 , and the effective B1 + with improved noise suppression and precision compared to LR and LRMC.

Correntropy-Based Low-Rank Matrix Factorization With Constraint Graph Learning for Image Clustering.

This article proposes a novel low-rank matrix factorization model for semisupervised image clustering. In order to alleviate the negative effect of outliers, the maximum correntropy criterion (MCC) is incorporated as a metric to build the model. To utilize the label information to improve the clustering results, a constraint graph learning framework is proposed to adaptively learn the local structure of the data by considering the label information. Furthermore, an iterative algorithm based on Fenchel conjugate (FC) and block coordinate update (BCU) is proposed to solve the model. The convergence properties of the proposed algorithm are analyzed, which shows that the algorithm exhibits both objective sequential convergence and iterate sequential convergence. Experiments are conducted on six real-world image datasets, and the proposed algorithm is compared with eight state-of-the-art methods. The results show that the proposed method can achieve better performance in most situations in terms of clustering accuracy and mutual information.

Multi-level deep domain adaptive adversarial model based on tensor-train decomposition for industrial time series forecasting

The polyester industry is a complex process industry, building a time series prediction model for new production lines or equipment with new sensors can be challenging due to a lack of historical data. The time-series data collected from sensors cross-production-line often exhibit varying distributions. Current domain adaptation (DA) approaches in data-driven time series forecasting primarily concentrate on adjusting either the features or the models, neglecting the intricacies of industrial time series data. Furthermore, constructing deep neural networks for industrial data necessitates substantial computational resources and runtime due to their large and high-dimensional nature. In order to tackle these obstacles, we propose a novel Multi-level deep domain adaptive adversarial model based on tensor-train decomposition (TT-MDAM). Our model aims to strike a dynamic balance between prediction accuracy and runtime efficiency. By integrating multiple perspectives at the feature, trend, and model levels, we leverage DA to enhance the prediction accuracy of our model in the target domain. Additionally, by analyzing the weight matrix of the neural network, we generate a low-rank model to improve operational efficiency. The application of the proposed TT-MDAM approach to both the three-phase flow facility process (TPFF) dataset and a real-world polyester esterification process dataset reveals promising results, outperforming state-of-the-art methodologies in terms of prediction performance. The results indicate that the approach provides a viable solution for building time series prediction models in industrial processes with new equipment or production lines.