Low-rank Signal Research Articles

Information Criteria for Signal Extraction Using Singular Spectrum Analysis: White and Red Noise

In singular spectrum analysis, which is applied to signal extraction, it is of critical importance to select the number of components correctly in order to accurately estimate the signal. In the case of a low-rank signal, there is a challenge in estimating the signal rank, which is equivalent to selecting the model order. Information criteria are commonly employed to address these issues. However, singular spectrum analysis is not aimed at the exact low-rank approximation of the signal. This makes it an adaptive, fast, and flexible approach. Conventional information criteria are not directly applicable in this context. The paper examines both subspace-based and information criteria, proposing modifications suited to the Hankel structure of trajectory matrices employed in singular spectrum analysis. These modifications are initially developed for white noise, and a version for red noise is also proposed. In the numerical comparisons, a number of scenarios are considered, including the case of signals that are approximated by low-rank signals. This is the most similar to the case of real-world time series. The criteria are compared with each other and with the optimal rank choice that minimizes the signal estimation error. The results of numerical experiments demonstrate that for low-rank signals and noise levels within a region of stable rank detection, the proposed modifications yield accurate estimates of the optimal rank for both white and red noise cases. The method that considers the Hankel structure of the trajectory matrices appears to be a superior approach in many instances. Reasonable model orders are obtained for real-world time series. It is recommended that a transformation be applied to stabilize the variance before estimating the rank.

Singular vectors of sums of rectangular random matrices and optimal estimation of high-rank signals: The extensive spike model.

Across many disciplines spanning from neuroscience and genomics to machine learning, atmospheric science, and finance, the problems of denoising large data matrices to recover hidden signals obscured by noise, and of estimating the structure of these signals, is of fundamental importance. A key to solving these problems lies in understanding how the singular value structure of a signal is deformed by noise. This question has been thoroughly studied in the well-known spiked matrix model, in which data matrices originate from low-rank signal matrices perturbed by additive noise matrices, in an asymptotic limit where matrix size tends to infinity but the signal rank remains finite. We first show, strikingly, that the singular value structure of large finite matrices (of size ∼1000) with even moderate-rank signals, as low as 10, is not accurately predicted by the finite-rank theory, thereby limiting the application of this theory to real data. To address these deficiencies, we analytically compute how the singular values and vectors of an arbitrary high-rank signal matrix are deformed by additive noise. We focus on an asymptotic limit corresponding to an extensive spike model, in which both the signal rank and the size of the data matrix tend to infinity at a constant ratio. We map out the phase diagram of the singular value structure of the extensive spike model as a joint function of signal strength and rank. We further exploit these analytics to derive optimal rotationally invariant denoisers to recover the hidden high-rank signal from the data, as well as optimal invariant estimators of the signal covariance structure. Our extensive-rank results yield several conceptual differences compared to the finite-rank case: (1) as signal strength increases, the singular value spectrum does not directly transition from a unimodal bulk phase to a disconnected phase, but instead there is a bimodal connected regime separating them; (2) the signal singular vectors can be partially estimated even in the unimodal bulk regime, and thus the transitions in the data singular value spectrum do not coincide with a detectability threshold for the signal singular vectors, unlike in the finite-rank theory; (3) signal singular values interact nontrivially to generate data singular values in the extensive-rank model, whereas they are noninteracting in the finite-rank theory; and (4) as a result, the more sophisticated data denoisers and signal covariance estimators we derive, which take into account these nontrivial extensive-rank interactions, significantly outperform their simpler, noninteracting, finite-rank counterparts, even on data matrices of only moderate rank. Overall, our results provide fundamental theory governing how high-dimensional signals are deformed by additive noise, together with practical formulas for optimal denoising and covariance estimation.

Low-rank latent matrix-factor prediction modeling for generalized high-dimensional matrix-variate regression.

Motivated by diagnosing the COVID-19 disease using two-dimensional (2D) image biomarkers from computed tomography (CT) scans, we propose a novel latent matrix-factor regression model to predict responses that may come from an exponential distribution family, where covariates include high-dimensional matrix-variate biomarkers. A latent generalized matrix regression (LaGMaR) is formulated, where the latent predictor is a low-dimensional matrix factor score extracted from the low-rank signal of the matrix variate through a cutting-edge matrix factor model. Unlike the general spirit of penalizing vectorization plus the necessity of tuning parameters in the literature, instead, our prediction modeling in LaGMaR conducts dimension reduction that respects the geometric characteristic of intrinsic 2D structure of the matrix covariate and thus avoids iteration. This greatly relieves the computation burden, and meanwhile maintains structural information so that the latent matrix factor feature can perfectly replace the intractable matrix-variate owing to high-dimensionality. The estimation procedure of LaGMaR is subtly derived by transforming the bilinear form matrix factor model onto a high-dimensional vector factor model, so that the method of principle components can be applied. We establish bilinear-form consistency of the estimated matrix coefficient of the latent predictor and consistency of prediction. The proposed approach can be implemented conveniently. Through simulation experiments, the prediction capability of LaGMaR is shown to outperform some existing penalized methods under diverse scenarios of generalized matrix regressions. Through the application to a real COVID-19 dataset, the proposed approach is shown to predict efficiently the COVID-19.

On Best Low Rank Approximation of Positive Definite Tensors

Tensors, or multi-indexed arrays, play an important role in machine learning and signal processing. These higher-order generalizations of matrices allow for preservation of higher-order structure in data, and low rank decompositions of tensors allow for recovery of underlying information. In many cases, the underlying tensor of interest is positive (semi)definite, i.e., the homogeneous polynomial associated to the tensor has nonnegative evaluation on all inputs. An archetypal problem is that one has a noisy measurement of some low rank signal tensor of interest. This measurement itself does not have low rank, so one must compute a best low rank approximation of the measured tensor to recover component information. As it turns out, the set of tensors of rank less than or equal to is, in general, not closed when , and, as a consequence, best low rank approximations can fail to exist. In the case when a best low rank approximation does not exist, near optimal low rank approximations suffer numerical issues and cannot be used to reliably approximate underlying component information. As a consequence, existence guarantees for best low rank tensor approximations are of great practical and theoretical interest. This article develops deterministic guarantees for the existence of best low rank approximations of tensors which are positive semidefinite. In particular, we show that the set of low rank positive semidefinite tensors is relatively closed as a subset of the set of positive semidefinite tensors. We use this fact to give a deterministic bound which may be used to guarantee the existence of a best low rank approximation of a noisy low rank positive semidefinite tensor. Furthermore, we show that this condition guarantees uniqueness of the canonical polyadic decomposition for best low rank approximations. In addition, for order three tensors, we prove that our bound is sharp and that it can be computed using semidefinite programming. The main results of this article are illustrated through numerical experiments, which show that our bound is highly predictive of numerical issues when attempting to recover underlying component information from a noisy low rank positive semidefinite tensor.

Seismic wavefield information extraction method based on adaptive local singular value decomposition

The key to seismic data processing is to extract the wavefield information related to the subsurface geological targets. In this paper, we first introduce the low-rank signal priority and the strong energy signal priority characteristics of the Singular Value Decomposition (SVD) method. By analyzing the characteristics of the second-order difference spectrum of typical seismic data, we find two indicative parameters to adaptively select the singular value threshold for extracting reflection and diffraction events. Our proposed Adaptive Local Singular Value Decomposition (ALSVD) method takes advantage of the local analysis, and can effectively extract the weak signals in the seismic data. It is a robust method with good resolution and separation integrity, which is beneficial to seismic migration, seismic imaging, and evaluation of Oil and gas traps. Synthetic examples and field examples demonstrate the effectiveness of this method.

Low-rank signal subspace: parameterization, projection and signal estimation

The paper contains several theoretical results related to the weighted nonlinear least-squares problem for low-rank signal estimation, which can be considered as a Hankel structured low-rank approximation problem. A parameterization of the subspace of low-rank time series connected with generalized linear recurrence relations (GLRRs) is described and its features are investigated. It is shown how the obtained results help to describe the tangent plane, prove optimization problem features and construct stable algorithms for solving low-rank approximation problems. For the latter, a stable algorithm for constructing the projection onto a subspace of time series that satisfy a given GLRR is proposed and justified. This algorithm is used for a new implementation of the known Gauss-Newton method using the variable projection approach. The comparison by stability and computational cost is performed theoretically and with the help of an example.

Low-rank signal subspace: parameterization, projection and signal estimation

Low-rank signal subspace: parameterization, projection and signal estimation

Mixed Alternating Projections with Application to Hankel Low-Rank Approximation

The method of alternating projections for extracting low-rank signals is considered. The problem of decreasing the computational costs while keeping the estimation accuracy is analyzed. The proposed algorithm consists of alternating projections on the set of low-rank matrices and the set of Hankel matrices, where iterations of weighted projections with different weights are mixed. For algorithm justification, theory related to mixed alternating projections to linear subspaces is studied and the limit of mixed projections is obtained. The proposed approach is applied to the problem of Hankel low-rank approximation for constructing a modification of the Cadzow algorithm. Numerical examples compare the accuracy and computational cost of the proposed algorithm and Cadzow iterations.

Sparse recovery by reduced variance stochastic approximation

AbstractIn this paper, we discuss application of iterative Stochastic Optimization routines to the problem of sparse signal recovery from noisy observation. Using Stochastic Mirror Descent algorithm as a building block, we develop a multistage procedure for recovery of sparse solutions to Stochastic Optimization problem under assumption of smoothness and quadratic minoration on the expected objective. An interesting feature of the proposed algorithm is linear convergence of the approximate solution during the preliminary phase of the routine when the component of stochastic error in the gradient observation, which is due to bad initial approximation of the optimal solution, is larger than the ‘ideal’ asymptotic error component owing to observation noise ‘at the optimal solution’. We also show how one can straightforwardly enhance reliability of the corresponding solution using Median-of-Means-like techniques.We illustrate the performance of the proposed algorithms in application to classical problems of recovery of sparse and low-rank signals in the generalized linear regression framework. We show, under rather weak assumption on the regressor and noise distributions, how they lead to parameter estimates which obey (up to factors which are logarithmic in problem dimension and confidence level) the best known accuracy bounds.

On the Best Choice of Lasso Program Given Data Parameters

Compressed sensing (CS) is a paradigm in which a structured high-dimensional signal may be recovered from random, under-determined, and corrupted linear measurements. Lasso programs are effective for solving CS problems due to their proven ability to leverage underlying signal structure. Three popular Lasso programs are equivalent in a sense and sometimes used interchangeably. Tuned by a governing parameter, each admit an optimal parameter choice. For sparse or low-rank signal structures, this choice yields minimax order-optimal error. While CS is well-studied, existing theory for Lasso programs typically concerns this optimally tuned setting. However, the optimal parameter value for a Lasso program depends on properties of the data, and is typically unknown in practical settings. Performance in empirical problems thus hinges on a program’s parameter sensitivity: it is desirable that small variation about the optimal parameter choice begets small variation about the optimal risk. We examine the risk for these three programs and demonstrate that their parameter sensitivity can differ for the same data. We prove a <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">gauge-constrained</i> Lasso program admits asymptotic cusp-like behaviour of its risk in the limiting low-noise regime. We prove that a <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">residual-constrained</i> Lasso program has asymptotically suboptimal risk for very sparse vectors. These results contrast observations about an <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">unconstrained</i> Lasso program, which is relatively less sensitive to its parameter choice. We support the asymptotic theory with numerical simulations, demonstrating that parameter sensitivity of Lasso programs is readily observed for even modest dimensional parameters. Importantly, these simulations demonstrate regimes in which a Lasso program exhibits sensitivity to its parameter choice, though the other two do not. We hope this work aids practitioners in selecting a Lasso program for their problem.

Asynchrony Increases Efficiency: Time Encoding of Videos and Low-Rank Signals

In event-based sensing, many sensors independently and asynchronously emit events when there is a change in their input. Event-based sensing can present significant improvements in power efficiency when compared to traditional sampling, because (1) the output is a stream of events where the important information lies in the timing of the events, and (2) the sensor can easily be controlled to output information only when interesting activity occurs at the input. Moreover, event-based sampling can often provide better resolution than standard uniform sampling. Not only does this occur because individual event-based sensors have higher temporal resolution, it also occurs because the asynchrony of events allows for less redundant and more informative encoding. We would like to explain how such curious results come about. To do so, we use ideal time encoding machines as a proxy for event-based sensors. We explore time encoding of signals with low rank structure, and apply the resulting theory to video. We then see how the asynchronous firing times of the time encoding machines allow for better reconstruction than in the standard sampling case, if we have a high spatial density of time encoding machines that fire less frequently.

Enhanced low-rank matrix estimation for simultaneous denoising and reconstruction of 5D seismic data

Noise and missing traces usually influence the quality of multidimensional seismic data. Therefore, it is necessary to estimate the useful signal from its noisy observation. The damped rank-reduction (DRR) method has emerged as an effective method to reconstruct the useful signal matrix from noisy and incomplete observations. However, the higher the noise level and the larger the ratio of missing traces, the weaker the DRR operator becomes. Consequently, the estimated low-rank (LR) signal matrix includes a significant amount of residual noise that influences the following processing steps. Therefore, we focus on the problem of estimating an LR signal matrix from its noisy observation. To elaborate on the novel algorithm, we formulate an improved proximity function by mixing the moving-average filter and the arctangent penalty function. First, we apply the proximity function to the level-4 block Hankel matrix before singular-value decomposition (SVD) and, then, to singular values, during the damped truncated SVD process. The combination of the novel proximity function and the DRR framework leads to an optimization problem, which results in better recovery performance. Our algorithm aims at producing an enhanced rank-reduction operator to estimate the useful signal matrix with higher quality. Experiments are conducted on synthetic and real 5D seismic data to compare the effectiveness of our approach to the DRR approach. Our approach obtains better performance because the estimated LR signal matrix is cleaner and contains fewer artifacts compared to that reconstructed by the DRR algorithm.

Distributed Signal Processing Algorithms for Wireless Sensor Networks

Wireless Body Area Networks (WBAN), in particular in the field of wearable health monitoring system (WMB), such as electromagnetic cardiograms (ECG) data collecting system via WBANs in e-health applications, is becoming increasingly important for future communication systems. Compressive sensing (CS), on the other hand, has been shown to consume less power compared classic transform-coding-based approaches. We propose a new low-rank sparse deep signal recovery algorithm for recovering ECG data in the context of CS (Compressive sensing) because the spatial and temporal data collected by a WBAN have some closely correlated structures in certain wavelet domains e.g., the discrete wavelet transform (DWT) domain

Improved estimation of myelin water fractions with learned parameter distributions.

To improve estimation of myelin water fraction (MWF) in the brain from multi-echo gradient-echo imaging data. A systematic sensitivity analysis was first conducted to characterize the conventional exponential models used for MWF estimation. A new estimation method was then proposed for improved estimation of MWF from practical gradient-echo imaging data. The proposed method uses an extended signal model that includes a finite impulse response filter to compensate for practical signal variations. This new model also enables the use of prelearned parameter distributions as well as low-rank signal structures to improve parameter estimation. The resulting parameter estimation problem was solved optimally in the Bayesian sense. Our sensitivity analysis results showed that the conventional exponential models were very sensitive to measurement noise and modeling errors. Our simulation and experimental results showed that our proposed method provided a substantial improvement in reliability, reproducibility, and robustness of MWF estimates over the conventional methods. Clinical results obtained from stroke patients indicated that the proposed method, with its improved capability, could reveal the loss of myelin in lesions, demonstrating its translational potentials. This paper addressed the problem of robust MWF estimation from gradient-echo imaging data. A new method was proposed to provide improved MWF estimation in the presence of significant noise and modeling errors. The performance of the proposed method has been evaluated using both simulated and experimental data, showing significantly improved robustness over the existing methods. The proposed method may prove useful for quantitative myelin imaging in clinical applications.

Community Detection in Blockchain Social Networks

In this work, we consider community detection in blockchain networks. We specifically take the Bitcoin network and Ethereum network as two examples, where community detection serves in different ways. For the Bitcoin network, we modify the traditional community detection method and apply it to the transaction social network to cluster users with similar characteristics. For the Ethereum network, on the other hand, we define a bipartite social graph based on the smart contract transactions. A novel community detection algorithm which is designed for low-rank signals on graph can help find users' communities based on user-token subscription. Based on these results, two strategies are devised to deliver on-chain advertisements to those users in the same community. We implement the proposed algorithms on real data. By adopting the modified clustering algorithm, the community results in the Bitcoin network are basically consistent with the ground-truth of the betting site community which has been announced to the public. Meanwhile, we run the proposed strategy on real Ethereum data, visualize the results and implement an advertisement delivery on the Ropsten test net.

On the Detection of Low-Rank Signal in the Presence of Spatially Uncorrelated Noise: A Frequency Domain Approach

This paper analyzes the detection of a <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$M$</tex-math></inline-formula> –dimensional useful signal modeled as the output of a <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$M \times K$</tex-math></inline-formula> MIMO filter driven by a <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$K$</tex-math></inline-formula> –dimensional white Gaussian noise, and corrupted by a <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$M$</tex-math></inline-formula> –dimensional Gaussian noise with mutually uncorrelated components. The study is focused on frequency domain test statistics based on the eigenvalues of an estimate of the spectral coherence matrix (SCM), obtained as a renormalization of the frequency-smoothed periodogram of the observed signal. If <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$N$</tex-math></inline-formula> denotes the sample size and <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$B$</tex-math></inline-formula> the smoothing span, it is proved that in the high-dimensional regime where <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$M,B,N$</tex-math></inline-formula> converge to infinity while <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$K$</tex-math></inline-formula> remains fixed, the SCM behaves as a certain correlated Wishart matrix. Exploiting well-known results on the behaviour of the eigenvalues of such matrices, it is deduced that the standard tests based on linear spectral statistics of the SCM fail to detect the presence of the useful signal in the high-dimensional regime. A new test based on the SCM, which is proved to be consistent, is also proposed, and its statistical performance is evaluated through numerical simulations.

Low-rank and joint-sparse signal recovery using sparse Bayesian learning in a WBAN

Wireless body area networks (WBANs) will become increasingly important in future communication systems, especially in the area of wearable health monitoring systems, such as telemonitoring systems for the collection of electrocardiogram (ECG) data/electroencephalogram (EEG) data via WBANs for e-health applications. However, wearable devices usually require limited power consumption to ensure long battery life. Fortunately, compressed sensing (CS) has been proven to use less energy than traditional transform-coding-based methods. Because the spatial and temporal data collected by a WBAN have some closely correlated structures in certain transform domains (e.g., the discrete cosine transform (DCT) domain), we exploit these structures to propose a new low-rank and joint-sparse (L&S) signal recovery algorithm for recovering ECG/EEG data in the framework of CS. Using a simultaneously L&S signal model, we employ a Bayesian learning treatment. This treatment incorporates an L&S-inducing prior over the data and appropriate hyperpriors over all hyperparameters and thereby yields an effective reconstruction of L&S data. Simulation results with synthetic and real ECG/EEG data demonstrate that the proposed algorithm is superior to other state-of-the-art recovery algorithms in terms of reconstruction performance with comparable computational complexity.

Sparse Aperture ISAR Imaging Method Based on Joint Constraints of Sparsity and Low Rank

A new inverse synthetic aperture radar (ISAR) imaging framework is proposed to obtain high cross-range resolution under sparse aperture conditions, which is a challenge when the signal-to-noise ratio is low. Motivated by the sparsity and low rank of target’s 2-D distribution, the imaging problem is converted to the simultaneously sparse and low-rank signal matrix reconstruction problem under multiple measurement vector (MMV) model, and a novel reconstruction method based on joint constraints of sparsity and low rank is proposed. Due to the over-relax problem, the traditional convex optimization method cannot achieve a better performance using joint structures than exploiting just one of the constraints. As such, a nonconvex penalty function is introduced. To avoid the local minima, the convexity of the cost function should be ensured when constructing the nonconvex penalty function. The adaptive filtering framework, which is a powerful way to recovery the sparse low-rank matrix accurately from its noisy observation, is adopted as a reconstruction algorithm. Furthermore, the optimal step size formula and the idea of smoothed zero norm are used to enhance the convergence and the ability to suppress noise. The newly proposed method is verified by the simulation experiment, which has a better performance in image quality, robustness to noise, and imaging speed.

Permutation methods for factor analysis and PCA

Researchers often have datasets measuring features $x_{ij}$ of samples, such as test scores of students. In factor analysis and PCA, these features are thought to be influenced by unobserved factors, such as skills. Can we determine how many components affect the data? This is an important problem, because decisions made here have a large impact on all downstream data analysis. Consequently, many approaches have been developed. Parallel Analysis is a popular permutation method: it randomly scrambles each feature of the data. It selects components if their singular values are larger than those of the permuted data. Despite widespread use, as well as empirical evidence for its accuracy, it currently has no theoretical justification. In this paper, we show that parallel analysis (or permutation methods) consistently select the large components in certain high-dimensional factor models. However, when the signals are too large, the smaller components are not selected. The intuition is that permutations keep the noise invariant, while “destroying” the low-rank signal. This provides justification for permutation methods. Our work also uncovers drawbacks of permutation methods, and paves the way to improvements.

Image Recovery via Transform Learning and Low-Rank Modeling: The Power of Complementary Regularizers.

Recent works on adaptive sparse and on low-rank signal modeling have demonstrated their usefulness in various image/video processing applications. Patch-based methods exploit local patch sparsity, whereas other works apply low-rankness of grouped patches to exploit image non-local structures. However, using either approach alone usually limits performance in image reconstruction or recovery applications. In this work, we propose a simultaneous sparsity and low-rank model, dubbed STROLLR, to better represent natural images. In order to fully utilize both the local and non-local image properties, we develop an image restoration framework using a transform learning scheme with joint low-rank regularization. The approach owes some of its computational efficiency and good performance to the use of transform learning for adaptive sparse representation rather than the popular synthesis dictionary learning algorithms, which involve approximation of NP-hard sparse coding and expensive learning steps. We demonstrate the proposed framework in various applications to image denoising, inpainting, and compressed sensing based magnetic resonance imaging. Results show promising performance compared to state-of-the-art competing methods.