Structured Low Rank Approximation Research Articles

Approximate GCD of several multivariate sparse polynomials based on SLRA interpolation

To compute the greatest common divisor (GCD) of a set of multivariate polynomials, modular algorithms are typically employed to prevent any growth in the coefficient polynomials in the intermediate expressions. However, when dealing with multivariate polynomials with a priori errors on their coefficients, using such modular algorithms becomes challenging. This is because any resulting approximate GCD computed in one variable may have perturbations depending on the evaluation point and may not be an image of the same desired multivariate approximate GCD. This necessitates computing it as given multivariate polynomials, and operating with large matrices whose size is exponential in the number of variables. In this paper, we present a new modular algorithm, suitable for dense cases and effective for sparse ones, called “SLRA interpolation”. This algorithm uses the multidimensional fast Fourier transform (FFT) and the structured low-rank approximation (SLRA) of non-square block diagonal matrices. The SLRA interpolation technique may reduce the time-complexity for one iteration in the computation of approximate GCD of several multivariate polynomials, especially for the sparse case.

Complex Magnetic Anomaly Detection Using Structured Low-Rank Approximation With Total Variation Regularization

In the field of magnetic anomaly detection (MAD), the anomaly signal is easily submerged by ambient electromagnetic interference. Though the existing noise suppression methods can effectively improve the signal-to-noise ratio (SNR), there are still some intractable problems, such as signal distortion and boundary blur. To solve these problems, a novel MAD method based on structured low rank (SLR) and total variation (TV) regularization constraints is proposed in this letter. The noise suppression performance is improved by leveraging the structured low rankness of the signal. To preserve clean boundaries of the anomalies, an anisotropic TV regularization constraint is employed in the approach. Comparing the SLR-TV method with four state-of-the-art methods with extensive field tests, the results demonstrate that the proposed SLR-TV method achieves the greatest SNR improvement by about 63.24% and the best structural similarity (SSIM) improvement by about 53.02% over other methods in the range from −40 to 0 dB, showing the utility and high fidelity of the proposed framework in low SNR.

Synergistic Hankel Structured Low-Rank Approximation With Total Variation Regularization for Complex Magnetic Anomaly Detection

In the field of magnetic anomaly detection (MAD), the anomaly signal is easy to be submerged by ambient electromagnetic interference. Though the existing noise suppression methods can effectively improve the signal-to-noise ratio (SNR), there are still some intractable problems, such as signal distortion and boundary blur. To solve these problems, a novel MAD method based on structured low-rank (SLR) and total variation (TV) regularization constraints is proposed in this paper. To be specific, a new framework SLR-TV, which mainly contains abnormal signal acquisition, objective function solution, and inverse transform operation is constructed. The noise suppression performance is improved by leveraging the structured low-rankness of the signal. To preserve clean boundaries of the anomalies, an anisotropic TV regularization constraint is employed in the approach. Comparing the SLR-TV with five state-of-the-art methods with extensive synthetic and field tests, the results demonstrate that the proposed SLR-TV method achieves the highest SNR improvement by about 15.62% and the best structural similarity (SSIM) improvement by about 62.95% over other methods in the range from -40 dB to 0 dB, showing the utility and high-fidelity of the proposed framework in low SNR.

Approximate GCD by relaxed NewtonSLRA algorithm

We propose a better algorithm for approximate greatest common divisor (approximate GCD) of univariate polynomials in terms of robustness and distance, based on the NewtonSLRA algorithm that is a solver for the structured low rank approximation (SLRA) problem. Our algorithm mainly enlarges the tangent space in the NewtonSLRA algorithm and adapts it to a certain weighted Frobenius norm. Moreover, we propose some improvement in computing time.

Approximate Simultaneous Diagonalization of Matrices via Structured Low-Rank Approximation

Approximate Simultaneous Diagonalization (ASD) is a problem to find a common similarity transformation which approximately diagonalizes a given square-matrix tuple. Many data science problems have been reduced into ASD through ingenious modelling. For ASD, the so-called Jacobi-like methods have been extensively used. However, the methods have no guarantee to suppress the magnitude of off-diagonal entries of the transformed tuple even if the given tuple has a common exact diagonalizer, i.e., the given tuple is simultaneously diagonalizable. In this paper, to establish an alternative powerful strategy for ASD, we present a novel two-step strategy, called Approximate-Then-Diagonalize-Simultaneously (ATDS) algorithm. The ATDS algorithm decomposes ASD into (Step 1) finding a simultaneously diagonalizable tuple near the given one; and (Step 2) finding a common similarity transformation which diagonalizes exactly the tuple obtained in Step 1. The proposed approach to Step 1 is realized by solving a Structured Low-Rank Approximation (SLRA) with Cadzow's algorithm. In Step 2, by exploiting the idea in the constructive proof regarding the conditions for the exact simultaneous diagonalizability, we obtain a common exact diagonalizer of the obtained tuple in Step 1 as a solution for the original ASD. Unlike the Jacobi-like methods, the ATDS algorithm has a guarantee to find a common exact diagonalizer if the given tuple happens to be simultaneously diagonalizable. Numerical experiments show that the ATDS algorithm achieves better performance than the Jacobi-like methods.

Computation of the nearest non-prime polynomial matrix: Structured low-rank approximation approach

In this paper we consider the problem of computing the nearest non-prime polynomial matrix to a given left prime polynomial matrix. This problem is a generalization of the problem of computing the nearest non-coprime polynomials to several given coprime polynomials. In order to solve the problem, we first prove the equivalence of the left primeness property of a polynomial matrix with rank deficiency of a certain block Toeplitz matrix constructed from the given polynomial matrix. Using this equivalence, we reformulate the problem as the problem of computing the nearest Structured Low-Rank Approximation (SLRA) of the associated block Toeplitz matrix. This reformulation has been carried out when the leading coefficient matrix of the left prime polynomial matrix is of full row rank. We also prove that a non-prime polynomial matrix can be obtained arbitrarily close to a left prime polynomial matrix when the leading coefficient matrix is rank deficient. We implement numerical techniques such as STLN and regularized factorization approach, available in literature, to solve the SLRA problem. Furthermore, we demonstrate our proposed algorithm with several numerical examples along with the comparison with the results in literature wherever available.

Block Basis Factorization for Scalable Kernel Evaluation

Kernel methods are widespread in machine learning; however, they are limited by the quadratic complexity of the construction, application, and storage of kernel matrices. Low-rank matrix approximation algorithms are widely used to address this problem and reduce the arithmetic and storage cost. However, we observed that for some datasets with wide intra-class variability, the optimal kernel parameter for smaller classes yields a matrix that is less well approximated by low-rank methods. In this paper, we propose an efficient structured low-rank approximation method -- the Block Basis Factorization (BBF) -- and its fast construction algorithm to approximate radial basis function (RBF) kernel matrices. Our approach has linear memory cost and floating-point operations for many machine learning kernels. BBF works for a wide range of kernel bandwidth parameters and extends the domain of applicability of low-rank approximation methods significantly. Our empirical results demonstrate the stability and superiority over the state-of-art kernel approximation algorithms.

Ambient vibration responses denoising for operational modal analysis of a jacket-type offshore platform

Operational modal analysis based on the ambient vibration data has drawn great attention. However, measured responses are inevitably contaminated with noise and may not be clean enough for estimating the modal parameters with proper accuracy. This study develops an operational modal analysis procedure named NExT-SLRA-CE, based on noise removal from measured ambient vibration responses, including three steps: (1) Using natural excitation technique (NExT) to get free decay responses (auto- and cross-correlation functions) from measured (noisy) ambient vibration data, (2) Implementing iterative Cadzow's algorithm for the structured low rank approximation (SLRA) to remove noise from the noisy correlation functions, and (3) Applying the complex exponential (CE) method to extract modal parameters from the filtered correlation functions. The validity of the proposed operational modal analysis procedure is verified by using a numerical simulation of a four-story shear building subjected to white noise excitation, with considering the effects of noise, record length and different reference channel. Furthermore, the proposed procedure is applied to a field test of a jacket-type offshore platform. Results indicate that the NExT-SLRA-CE method can remove noise from measured signals and improve the efficiency and accuracy of the modal frequencies and damping ratios estimation.

A comparison between structured low-rank approximation and correlation approach for data-driven output tracking

Data-driven control is an alternative to the classical model-based control paradigm. The main idea is that a model of the plant is not explicitly identified prior to designing the control signal. Two recently proposed methods for data-driven control—a method based on correlation analysis and a method based on structured matrix low-rank approximation and completion—solve identical control problems. The aim of this paper is to compare the methods, both theoretically and via a numerical case study. The main conclusion of the comparison is that there is no universally best method: the two approaches have complementary advantages and disadvantages. Future work will aim to combine the two methods into a more effective unified approach for data-driven output tracking.

Application of Structured Low Rank Approximation in Modal Analysis

The quality of measured vibration response signals is adversely affected by noise in modal testing. This situation often leads to serious difficulties in estimating the modal parameters with proper accuracy. This paper presents a signal de-noising method for measured impulsive response functions (IRFs) based on structured low rank approximation (SLRA) so as to improve the accuracy of the modal parameters identification. The proposed method is verified by simulation study of a cantilever beam. The results show that this method can eliminate noise from measured IRFs efficiently, and significant improvement in modal parameters estimation could be obtained based on the filtered IRFs.

A class of multilevel structured low-rank approximation arising in material processing

ABSTRACTThis work is devoted to studying the problem of approximating a given matrix by the product of two low-rank structured matrix which arises in the material processing. Firstly, we analyse some properties of the original problem and utilize the alternating least squares method to reformulate it into two subproblems. Then by using the Gramian representation and a Vandermonde-like transformation, the feasible sets of the subproblems are characterized, which makes them be respectively transformed into two unconstrained minimization problems. Finally, we derive the expressions of the gradients of the objective functions and apply the gradient descent algorithm to solve them. Numerical examples are tested to show that our method performs well in terms of the residual error of the objective function.

Estimation of directions of arrival of multiple distributed sources for nested array

We consider the problem of estimating the nominal directions of arrival (DOA) of multiple distributed sources using nested array. In practical applications, angular spread of distributed source arises due to the multipath propagation and local scattering around the vicinity of a source. DOA estimation methods based on point-source assumption are known to severely degrade in the presence of the angular spread. To solve this problem, we develop a distributed source model based approach for a nested array. Based on the concept of difference co-array, redundancy averaging technique is applied to the sample covariance matrix. By exploiting the annihilating property of the redundancy averaged terms, an efficient method based on annihilating filter (AF) is proposed. This method can be applied to the case of more sources compared with the actual physical sensors without a priori knowledge of the spreading parameters. Whereas the sample covariance matrix is corrupted by either noise or estimation error due to finite snapshots, so an AF method with structured low rank approximation (SLRA) is further proposed, which results in a better estimate. Finally, numerical simulations demonstrate the effectiveness of the proposed methods.

A Quadratically Convergent Algorithm for Structured Low-Rank Approximation

Structured Low-Rank Approximation is a problem arising in a wide range of applications in Numerical Analysis and Engineering Sciences. Given an input matrix $$M$$ , the goal is to compute a matrix $$M'$$ of given rank $$r$$ in a linear or affine subspace $$E$$ of matrices (usually encoding a specific structure) such that the Frobenius distance $$\left\| M-M' \right\| $$ is small. We propose a Newton-like iteration for solving this problem, whose main feature is that it converges locally quadratically to such a matrix under mild transversality assumptions between the manifold of matrices of rank $$r$$ and the linear/affine subspace $$E$$ . We also show that the distance between the limit of the iteration and the optimal solution of the problem is quadratic in the distance between the input matrix and the manifold of rank $$r$$ matrices in $$E$$ . To illustrate the applicability of this algorithm, we propose a Maple implementation and give experimental results for several applicative problems that can be modeled by Structured Low-Rank Approximation: univariate approximate GCDs (Sylvester matrices), low-rank matrix completion (coordinate spaces) and denoising procedures (Hankel matrices).

Stochastic algorithms for solving structured low-rank matrix approximation problems

In this paper, we investigate the complexity of the numerical construction of the Hankel structured low-rank approximation (HSLRA) problem, and develop a family of algorithms to solve this problem. Briefly, HSLRA is the problem of finding the closest (in some pre-defined norm) rank r approximation of a given Hankel matrix, which is also of Hankel structure. We demonstrate that finding optimal solutions of this problem is very hard. For example, we argue that if HSLRA is considered as a problem of estimating parameters of damped sinusoids, then the associated optimization problem is basically unsolvable. We discuss what is known as the orthogonality condition, which solutions to the HSLRA problem should satisfy, and describe how any approximation may be corrected to achieve this orthogonality. Unlike many other methods described in the literature the family of algorithms we propose has the property of guaranteed convergence.

Exact Solutions in Structured Low-Rank Approximation

Structured low-rank approximation is the problem of minimizing a weighted Frobenius distance to a given matrix among all matrices of fixed rank in a linear space of matrices. We study the critical points of this optimization problem using algebraic geometry. A particular focus lies on Hankel matrices, Sylvester matrices, and generic linear spaces.

Factorization Approach to Structured Low-Rank Approximation with Applications

We consider the problem of approximating an affinely structured matrix, for example, a Hankel matrix, by a low-rank matrix with the same structure. This problem occurs in system identification, signal processing, and computer algebra, among others. We impose the low rank by modeling the approximation as a product of two factors with reduced dimension. The structure of the low-rank model is enforced by introducing a penalty term in the objective function. The proposed local optimization algorithm is able to solve the weighted structured low-rank approximation problem, as well as to deal with the cases of missing or fixed elements. In contrast to approaches based on kernel representations (in the linear algebraic sense), the proposed algorithm is designed to address the case of small targeted rank. We compare it to existing approaches on numerical examples of system identification, approximate greatest common divisor problem, and symmetric tensor decomposition and demonstrate its consistently good performance.

Software for weighted structured low-rank approximation

A software package is presented that computes locally optimal solutions to low-rank approximation problems with the following features: •mosaic Hankel structure constraint on the approximating matrix,•weighted 2-norm approximation criterion,•fixed elements in the approximating matrix,•missing elements in the data matrix, and•linear constraints on an approximating matrix’s left kernel basis. It implements a variable projection type algorithm and allows the user to choose standard local optimization methods for the solution of the parameter optimization problem. For an m×n data matrix, with n>m, the computational complexity of the cost function and derivative evaluation is O(m2n). The package is suitable for applications with n≫m. In statistical estimation and data modeling–the main application areas of the package–n≫m corresponds to modeling of large amount of data by a low-complexity model. Performance results on benchmark system identification problems from the database DAISY and approximate common divisor problems are presented.

Nuclear norm system identification with missing inputs and outputs

We present a system identification method for problems with partially missing inputs and outputs. The method is based on a subspace formulation and uses the nuclear norm heuristic for structured low-rank matrix approximation, with the missing input and output values as the optimization variables. We also present a fast implementation of the alternating direction method of multipliers (ADMM) to solve regularized or non-regularized nuclear norm optimization problems with Hankel structure. This makes it possible to solve quite large system identification problems. Experimental results show that the nuclear norm optimization approach to subspace identification is comparable to the standard subspace methods when no inputs and outputs are missing, and that the performance degrades gracefully as the percentage of missing inputs and outputs increases.

Optimization challenges in the structured low rank approximation problem

In this paper we illustrate some optimization challenges in the structured low rank approximation (SLRA) problem. SLRA can be described as the problem of finding a low rank approximation of an observed matrix which has the same structure as this matrix (such as Hankel). We demonstrate that the optimization problem arising is typically very difficult: in particular, the objective function is multiextremal even for simple cases. The main theme of the paper is to suggest that the difficulties described in approximating a solution of the SLRA problem open huge possibilities for the application of stochastic methods of global optimization.

Computing the radius of controllability for state space systems

In this paper, we discuss the problem of computing the nearest uncontrollable system to a given control system represented by a matrix pair ( A , B ) . In order to do so, we construct a sequence of structured matrices from given system matrices A and B . Controllability of the pair ( A , B ) is equivalent to a condition on the null-space dimension of an appropriate matrix in this sequence. We show that the dimension of the reachability space is also related to the above condition. Further, it is shown that the nearest Structured Low Rank Approximation (SLRA) of this structured matrix corresponds to a nearest uncontrollable system to the pair ( A , B ) .