Structured Total Least Squares Research Articles

Structured Total Least Squares Method Based on Variable Projection

Objectives To solve the problem that the coefficient matrix and the observation vector have structured characteristics for many practical applications in the field of surveying and mapping, that is, the coefficient matrix and the observation vector contain fixed quantities (or even fixed columns) and random quantities, and the random quantities at different positions are linearly related. Methods Starting from the errors-in-variables (EIV) function model, the augmented matrix composed of the coefficient matrix and the observation vector is expressed as an affine function form, and the function model is reconstructed by the variable projection method. Then, a structured total least squares (STLS) estimation algorithm is derived by the Lagrange method. Results The example results show that the proposed algorithm is consistent with the existing weighted or structured total least squares estimation algorithms that can solve structured problem in the coefficient matrix and the observation vector. Compared with the weighted total least squares (WTLS) estimation algorithms, the proposed algorithm only needs to establish a positive definite weight matrix of independent random variables and reduces the number of estimates. Compared with other STLS estimation algorithms, the proposed algorithm takes into account the overall structure of the coefficient matrix and the observation vector. Conclusions This shows the effectiveness of the proposed algorithm. The STLS estimation and the WTLS estimation ensure the statistical optimality from the perspective of the function model and the stochastic model, respectively.

Target localization based on structured total least squares with hybrid TDOA-AOA measurements

In this paper, we focus on the target localization problem which finds broad applications in radar, sonar and wireless sensor networks. A pseudolinear overdetermined system of equations is constructed from the nonlinear hybrid TDOA-AOA measurements about target location. Considering the matrix and vector in the constructed pseudolinear system are both contaminated by the measurement noise, a new weight least squares (WLS) method which is based on the first order Taylor expansions of the noise terms is developed in this paper and it can reduce the estimation bias that arise from the least squares (LS) method. In particular we focus on constructing a localization algorithm to reduce the bias that easily arise from the traditional methods. Thus in addition, a novel structured total least squares (STLS) method is also developed in this paper to further reduce the estimation bias specially when the target is outside the convex hull formed by sensors. Numerical examples show the superiority of the proposed STLS method in estimation accuracy compared with the LS method, total least squares (TLS) method and the proposed WLS method.

A splitting method for total least squares color image restoration problem

Color image restoration is an important problem in image processing. Using the structured total least squares (STLS) for fidelity term of the restoration process gives better results in comparison with the least squares (LS) approach. The main drawback of the STLS approach is its complexity. To overcome this issue, in this paper by an appropriate transformation the color image restoration is substituted with two smaller subproblems corresponding to smooth and oscillatory parts of the image. The first and second subproblems are modeled via STLS and LS approaches, respectively. We show that the proposed method is faster than STLS and gives competitive solutions with it. Also, we demonstrate that Haar wavelet perseveres the structure of the blurring operator, which causes a considerable reduction in computational and storage complexity of the proposed method.

Comparison of Structured and Weighted Total Least-Squares Adjustment Methods for Linearly Structured Errors-in-Variables Models

AbstractThe paper focuses on a specific errors-in-variables (EIV) model named the linearly structured EIV (LSEIV) model in which all the random elements of design matrix are in a linear combination of an input vector with random errors. Two existing structured total least-squares (STLS) algorithms named constrained TLS (CTLS) and structured TLS normalization (STLN) are introduced to solve the LSEIV model by treating the input and output vectors as the noisy structure vectors. For comparison purposes, the weighted TLS (WTLS) method is also performed based on the partial EIV model. Approximated accuracy assessment methods are also presented. The plane fitting and Bursa transformation examples are illustrated to demonstrate the accuracy and computational efficiency performance of the proposed algorithms. It shows that the proposed STLS and WTLS algorithms can achieve the same accuracy if the dispersion matrix of the WTLS method is constructed based on the partial EIV model.

Modifying Cadzow's algorithm to generate the optimal TLS-solution for the structured EIV-Model of a similarity transformation

Modifying Cadzow's algorithm to generate the optimal TLS-solution for the structured EIV-Model of a similarity transformationIn 2005, Felus and Schaffrin discussed the problem of a Structured Errors-in-Variables (EIV) Model in the context of a parameter adjustment for a classical similarity transformation. Their proposal, however, to perform a Total Least-Squares (TLS) adjustment, followed by a Cadzow step to imprint the proper structure, would not always guarantee the identity of this solution with the optimal Structured TLS solution, particularly in view of the residuals. Here, an attempt will be made to modify the Cadzow step in order to generate the optimal solution with the desired structure as it would, for instance, also result from a traditional LS-adjustment within an iteratively linearized Gauss-Helmert Model (GHM). Incidentally, this solution coincides with the (properly) Weighted TLS solution which does not need a Cadzow step.

Structured total least squares approach for efficient frequency estimation

A new structured total least squares (STLS) based frequency estimation algorithm for real sinusoids corrupted by white noise is devised. Numerical results are included to contrast the estimator performance with an existing STLS frequency estimation method as well as the Cramér-Rao lower bound in different signal-to-noise ratio conditions.

Structured Total Maximum Likelihood: An Alternative to Structured Total Least Squares

Linear inverse problems with uncertain measurement matrices appear in many different applications. One of the standard techniques for solving such problems is the total least squares (TLS) method. Recently, an alternative approach has been suggested, based on maximizing an appropriate likelihood function assuming that the measurement matrix consists of random Gaussian variables. We refer to this technique as the total maximum likelihood (TML) method. Here we extend this strategy to the case in which the measurement matrix is structured so that the perturbations are not arbitrary but rather follow a fixed pattern. The resulting estimate is referred to as the structured TML (STML). As we show, the STML can be viewed as a regularized version of the structured TLS (STLS) approach in which the regularization consists of a logarithmic penalty. In contrast to the STLS solution, the STML always exists. Furthermore, its performance in practice tends to be superior to that of the STLS and competitive to other regularized solvers, as we illustrate via several examples. We also consider a few interesting special cases in which the STML can be computed efficiently either by reducing it into a one-dimensional problem regardless of the problem size or by a decomposition via a discrete Fourier transform.

The structured total least squares algorithm research for passive location based on angle information

Based on the constrained total least squares (CTLS) passive location algorithm with bearing-only measurements, in this paper, the same passive location problem is transformed into the structured total least squares (STLS) problem. The solution of the STLS problem for passive location can be obtained using the inverse iteration method. It also expatiates that both the STLS algorithm and the CTLS algorithm have the same location mean squares error under certain condition. Finally, the article presents a kind of location and tracking algorithm for moving target by combining STLS location algorithm with Kalman filter (KF). The efficiency and superiority of the proposed algorithms can be confirmed by computer simulation results.

A fast algorithm for solving the Sylvester structured total least squares problem

In this paper, we develop a fast structured total least squares (STLS) algorithm for computing an approximate greatest common divisor (GCD) of two univariate polynomials. By exploiting the displacement structure of the Sylvester matrix and applying the generalized Schur algorithm, each single iteration of the proposed algorithm has quadratic computational complexity in the degrees of the given polynomials.

Structured Total Least Squares for Color Image Restoration

The problem of 3 x 3 color mixing image restoration is considered. The blurring matrices, as well as the observed image, are contaminated by noise; therefore the total least squares (TLS) method is employed to restore the original image. Since the blurring matrices are also structured, we apply the structured total least squares (STLS) method [J. B. Rosen, H. Park, and J. Glick, SIAM J. Matrix Anal. Appl., 17 (1996), pp. 110-126]. The blurring matrices are generally ill conditioned; thus Tikhonov's regularization is used to stabilize the solution. Since Neumann boundary conditions are used in the restoration process, the discrete cosine transform (DCT) based preconditioner is effective for the linear systems encountered in our STLS algorithm.

A Global Solution for the Structured Total Least Squares Problem with Block Circulant Matrices

We study the structured total least squares (STLS) problem of system of linear equations Ax = b, where A has a block circulant structure with N blocks. We show that by applying the discrete Fourier transform (DFT), the STLS problem decomposes into Nunstructured total least squares (TLS) problems. The N solutions of these problems are then assembled to generate the optimal global solution of the STLS problem. Similar results are obtained for elementary block circulant matrices. Here the optimal solution is obtained by assembling two solutions: one of an unstructured TLS problem and the second of a multidimensional TLS problem.

Implementation of the regularized structured total least squares algorithms for blind image deblurring

The structured total least squares (STLS) problem has been introduced to handle problems involving structured matrices corrupted by noise. Often the problem is ill-posed. Recently, regularization has been proposed in the STLS framework to solve ill-posed blind deconvolution problems encountered in image deblurring when both the image and the blurring function have uncertainty. The kernel of the regularized STLS (RSTLS) problem is a least squares problem involving Block–Toeplitz–Toeplitz–Block matrices. In this paper an algorithm is described to solve this problem, based on a particular implementation of the generalized Schur Algorithm (GSA). It is shown that this new implementation improves the computational efficiency of the straightforward implementation of GSA from O( N 2.5) to O( N 2), where N is the number of pixels in the image.

Multiple delays estimation for chirp signals using structured total least squares

We employ the Structured Total Least Squares (STLS) framework for estimating the delays associated with multiple reflections of a chirp signal. The classical methods of using matched-filter correlation processing, originally aimed at estimating the delay of a single reflection, have poor resolution and accuracy in the presence of multiple, closely spaced reflections with overlapping correlation functions. The STLS framework offers enhanced resolution and accuracy for this problem. We provide an overview of the STLS framework for complex-valued data and parameters, showing validity of the Riemannian Singular Value Decomposition (RiSVD) approach (which is a well-established tool for real-valued STLS problems), and also offer a new STLS algorithm via Alternating Coordinates Minimization (ACM), characterized by guaranteed convergence and by an ability to account for errors in the equations, thus potentially being more robust against model mismatch (e.g., whenever the exact number of reflections is unknown). We then turn to formulate the delays estimation problem in the complex-valued STLS framework, and use simulation results to demonstrate and analyze the accuracy and convergence performance (and associated trade-offs) of the proposed approach.

A regularized structured total least squares algorithm for high-resolution image reconstruction

High-resolution image reconstruction is an important problem in image processing. In general, the blurring matrices are ill-conditioned, and it is necessary to compute a regularized solution. Moreover, error exists not only in the blurred image but also the blurring matrix, thus the total least squares method tends to give better results than the ordinary least squares method. Since the blurring matrices are also structured, it is more appropriate to apply Structured Total Least Squares (STLS). Ng et al. [Int. J. Imaging Systems Technol. 12 (2002) 35] recently proposed a Regularized Constrained Total Least Squares (RCTLS) algorithm for this problem. RCTLS is essentially a different name for Regularized Structured Total Least Squares (RSTLS). However, Ng et al. solved a problem that approximates the RCTLS problem. The algorithm proposed in this paper solves the exact regularization of the STLS problem, and it is a faster algorithm. Also proposed is a preconditioner for the linear systems encountered in our RSTLS algorithm.

Efficient implementation of a structured total least squares based speech compression method

We present a fast implementation of a recently proposed speech compression scheme, based on an all-pole model of the vocal tract. Each frame of the speech signal is analyzed by storing the parameters of the complex damped exponentials deduced from the all-pole model and its initial conditions. In mathematical terms, the analysis stage corresponds to solving a structured total least squares (STLS) problem. It is shown that by exploiting the displacement rank structure of the involved matrices the STLS problem can be solved in a very fast way. Synthesis is computationally very cheap since it consists of adding the complex damped exponentials based on the transmitted parameters.The compression scheme is applied on a speech signal. The speed improvement of the fast vocoder analysis scheme is demonstrated. Furthermore, the quality of the compression scheme is compared with that of a standard coding algorithm, by using the segmental signal-to-noise ratio.

The structured total least‐squares approach for non‐linearly structured matrices

AbstractIn this paper, an extension of the structured total least‐squares (STLS) approach for non‐linearly structured matrices is presented in the so‐called ‘Riemannian singular value decomposition’ (RiSVD) framework. It is shown that this type of STLS problem can be solved by solving a set of Riemannian SVD equations. For small perturbations the problem can be reformulated into finding the smallest singular value and the corresponding right singular vector of this Riemannian SVD. A heuristic algorithm is proposed. Some examples of Vandermonde‐type matrices are used to demonstrate the improved accuracy of the obtained parameter estimator when compared to other methods such as least squares (LS) or total least squares (TLS). Copyright © 2002 John Wiley & Sons, Ltd.

IQML-like algorithms for solving structured total least squares problems: a unified view

The structured total least squares (STLS) problem is an extension of the total least squares (TLS) problem for solving an overdetermined system of equations Ax≈ b. In many cases the extended data matrix [A b] has a special structure (Hankel, Toeplitz,…). In those cases the use of STLS is often required if a maximum likelihood (ML) estimate of x is desired. The main objective of this manuscript is to clarify the difference between several IQML-like algorithms—for solving STLS problems—that have been proposed by different authors and within different frameworks. Some of these algorithms yield suboptimal solutions while others yield optimal solutions. An important result is that the classicial IQML algorithm yields suboptimal solutions to the STLS problem. We illustrate this on a specific STLS problem, namely the estimation of the parameters of superimposed exponentially damped cosines in noise. We also indicate when this suboptimality starts playing an important role.

Analysis of the structured total least squares problem for Hankel/Toeplitz matrices

The Structured Total Least Squares (STLS) problem is a natural extension of the Total Least Squares (TLS) approach when structured matrices are involved and a similarly structured rank deficient approximation of that matrix is desired. In many of those cases the STLS approach yields a Maximum Likelihood (ML) estimate as opposed to, e.g., TLS. In this paper we analyze the STLS problem for Hankel matrices (the theory can be extended in a straightforward way to Toeplitz matrices, block Hankel and block Toeplitz matrices). Using a particular parametrisation of rank-deficient Hankel matrices, we show that this STLS problem suffers from multiple local minima, the properties of which depend on the parameters of the new parametrisation. The latter observation makes initial estimates an important issue in STLS problems and a new initialization method is proposed. The new initialization method is applied to a speech compression example and the results confirm the improved performance compared to other previously proposed initialization methods.

The extended least squares criterion: minimization algorithms and applications

The least squares (LS) estimation criterion on one hand, and the total LS (TLS), constrained TLS (CTLS) and structured TLS (STLS) criteria on the other hand, can be viewed as opposite limiting cases of a more general criterion, which we term "extended LS" (XLS). The XLS criterion distinguishes measurement errors from modeling errors by properly weighting and balancing the two error sources. In the context of certain models (termed "pseudo-linear"), we derive two iterative algorithms for minimizing the XLS criterion: One is a straightforward "alternating coordinates" minimization, and the other is an extension of an existing CTLS algorithm. The algorithms exhibit different tradeoffs between convergence rate, computational load, and accuracy. The XLS criterion can be applied to popular estimation problems, such as identifying an autoregressive (AR) with exogenous noise (ARX) system from noisy input/output measurements or estimating the parameters of an AR process from noisy measurements. We demonstrate the convergence properties and performance of the algorithms with examples of the latter.

Fast algorithm for solving the Hankel/Toeplitz Structured Total Least Squares Problem

The Structured Total Least Squares (STLS) problem is a natural extension of the Total Least Squares (TLS) problem when constraints on the matrix structure need to be imposed. Similar to the ordinary TLS approach, the STLS approach can be used to determine the parameter vector of a linear model, given some noisy measurements. In many signal processing applications, the imposition of this matrix structure constraint is necessary for obtaining Maximum Likelihood (ML) estimates of the parameter vector. In this paper we consider the Toeplitz (Hankel) STLS problem (i.e., an STLS problem in which the Toeplitz (Hankel) structure needs to be preserved). A fast implementation of an algorithm for solving this frequently occurring STLS problem is proposed. The increased efficiency is obtained by exploiting the low displacement rank of the involved matrices and the sparsity of the associated generators.