Total Least Squares Problem Research Articles

Efficient eigenspace-based array signal processing using multiple shift-invariant subarrays

This paper deals with the construction of eigensubspaces for adaptive array signal processing. An efficient technique for extracting the eigensubspaces spanned by the data vector received by an N-element adaptive array is presented. We first decompose the original array into several subarrays with multiple shift invariances and find the eigensubspaces corresponding to each of the subarrays. By solving a least-squares (LS) or total least-squares (TLS) problem, the signal and noise subspaces corresponding to the original array can be found from the eigensubspaces spanned by the subarray data vectors. Hence, there is no need to perform the eigenvalue decomposition of the N/spl times/N correlation matrix of the received data vector. The proposed technique significantly reduces the required computational complexity as compared to the conventional eigenspace-based (ESB) methods. In conjunction with the spatial smoothing scheme or a proposed cross-correlation method, this technique can also deal with the case of coherent signals. The effectiveness of the proposed technique is demonstrated by several computer simulations.

Cross-correlation neural network models for the smallest singular component of general matrix

In this paper, we provide the theoretical foundation for a novel neural model to solve the smallest singular component of general matrix, on the basis of an extension of the Hebbian rule and a modification of cross-coupled Hebbian rule. This model can efficiently extract the singular-value component of the cross-correlation matrix of two stochastic signals. By Lasalle’s invariance principle and Lyapunov’s indirect method, we study the global asymptotic convergence of the networks to the first singular vectors of the cross-correlation matrix or non-squares matrix. A comparative study on the related neural networks shows that this neural network is efficient for computing the smallest singular component of general matrix. The novel model may have useful applications in solving the total least-squares problems in adaptive signal processing and image compressing.

Total least-squares problem for exponential function

Given the data , i = 1,...,m, we consider the existence problem for the optimal parameters for the exponential function approximating these data in the sense of total least squares. We give sufficient conditions which guarantee the existence of such optimal parameters.

TLS Method for Pulse Transfer Function Estimation in the Presence of Input and Output Noise

Total least squares (TLS) is a method of solving sets of linear equations Ax = b, where both the observation vector b and the data matrix A are inaccurate. In this paper, it is shown that the parameter estimation problem in the presence of input and output noise can be formulated as a TLS problem. In addition, an algebraic equivalence between the TLS method and the Koopmans-Levin method is proven. A comparison of the TLS method to the bias compensated least-squares method is also made.

Total Least Squares, Matrix Enhancement, and Signal Processing

The concept of least squares (LS) as applied to a system of generally inconsistent linear equations A x ≍ y plays a central role in algorithms used for solving a variety of important signal processing problems. In the standard LS approach, the smallest unconstrained Euclidean norm perturbation of vector y is sought so that the resultant perturbed system of linear equations A x = y + δ has a solution. Implicit in the LS problem formulation is the assumption that only y vector is subject to error. Depending on the nature of the application, however, system matrix A may also be subject to error. When both the system matrix and right side vector are imprecisely known, the use of a least-squares solution can lead to disappointingly poor results. To mitigate the effects of errors in both A and y the concept of total least squares (TLS) is invoked. In the standard TLS problem, the smallest unconstrained perturbation of the matrix-vector pair ( A, y ) is sought so that the perturbed system of equations [ A + Δ] x + = y + δ has a solution. In this paper, theoretical issues regarding TLS are first addressed in which a singular value decomposition approach is used to find the optimal unconstrained choices for the perturbations Δ and δ. The TLS concept is next extended to the case in which the perturbed matrix-vector pair ( A + Δ, y + δ ) identifying the perturbed linear equations is constrained to satisfy prescribed properties known (or hypothesized) to be possessed in the idealized case. These property constraints take on such disparate forms as requiring that the perturbed auxiliary matrix [ A + Δ − y − δ ] be positive definite Hermitian, block Toeplitz, or of a given nonfull rank. Property requirements of this nature characterize a variety of practical signal processing problems. Applications of these concepts to the problems of exponential data modeling and system identification are then made where improvements over more traditional solution procedures are realized. The notion of constrained data perturbations has also been successfully applied to the problems of synthesizing one-dimensional and higher dimensional linear systems, obtaining positive sequence approximations, and missing data interpolation.

Circle fitting by linear and nonlinear least squares

The problem of determining the circle of best fit to a set of points in the plane (or the obvious generalization ton-dimensions) is easily formulated as a nonlinear total least-squares problem which may be solved using a Gauss-Newton minimization algorithm. This straight-forward approach is shown to be inefficient and extremely sensitive to the presence of outliers. An alternative formulation allows the problem to be reduced to a linear least squares problem which is trivially solved. The recommended approach is shown to have the added advantage of being much less sensitive to outliers than the nonlinear least squares approach.

Algebraic relations between the total least squares and least squares problems with more than one solution

This paper completes our previous discussion on the total least squares (TLS) and the least squares (LS) problems for the linear systemAX=B which may contain more than one solution [12, 13], generalizes the work of Golub and Van Loan [1,2], Van Huffel [8], Van Huffel and Vandewalle [11]. The TLS problem is extended to the more general case. The sets of the solutions and the squared residuals for the TLS and LS problems are compared. The concept of the weighted squares residuals is extended and the difference between the TLS and the LS approaches is derived. The connection between the approximate subspaces and the perturbation theories are studied. It is proved that under moderate conditions, all the corresponding quantities for the solution sets of the TLS and the modified LS problems are close to each other, while the quantities for the solution set of the LS problem are close to the corresponding ones of a subset of that of the TLS problem.

On the Significance of Nongeneric Total Least Squares Problems

Total least squares (TLS) is one method of solving overdetermined sets of linear equations $AX \approx B$ that is appropriate when there are errors in both the observation matrix B and the data matrix A. Golub was the first to introduce this method into the field of numerical analysis and to develop an algorithm based on the singular value decomposition. However, as pointed out by Golub and Van Loan, some TLS problems fail to have a solution altogether. Van Huffel and Vandewalle described the properties of these so-called nongeneric problems and proposed an extension of the generic TLS problem, called nongeneric TLS, in order to make these problems solvable. They proved that the solution of these nongeneric TLS problems is still optimal with respect to the TLS criteria for any number of observation vectors in B if additional constraints are imposed on the TLS solution space. These constraints are further scrutinized in this paper and compared with other approaches in linear regression.

Analysis and Properties of the Generalized Total Least Squares Problem $AX \approx B$ When Some or All Columns in A are Subject to Error

The Total Least Squares (TLS) method has been devised as a more global fitting technique than the ordinary least squares technique for solving overdetermined sets of linear equations $AX \approx B$ when errors occur in all data. This method, introduced into numerical analysis by Golub and Van Loan, is strongly based on the Singular Value Decomposition (SVD). If the errors in the measurements A and B are uncorrelated with zero mean and equal variance, TLS is able to compute a strongly consistent estimate of the true solution of the corresponding unperturbed set $A_0 X = B_0 $. In the statistical literature, these coefficients are called the parameters of a classical errors-in-variables model.In this paper, the TLS problem, as well as the TLS computations, are generalized in order to maintain consistency of the parameter estimates in a general errors-in-variables model; i.e., some of the columns of A may be known exactly and the covariance matrix of the errors in the rows of the remaining data matrix may be...

The extended classical total least squares algorithm

PURPOSE : The subroutine DTLS solves the Total Least Squares (TLS) problem. DTLS is an extension 131 of the classical TLS method described by Golub and Van Loan Cl1 (see description). The TLS problem assumes an overdetermined set of linear equations AX = B where both the data matrix A as well as the observation matrix B are inaccurate . If the perturbations D on the data CA;Bl have zero mean and their covariance matrix E(D'D) with E the expected value operator, equals the identity matrix up to an unknown scaling factor (e.g. when all errors are independent and equally sized), then the TLS method computes a strongly consistent estimate of the true solution of the corresponding unperturbed set C31. TLS Definition : Given matrices A and B, find a matrix X satisfying : is an N by L matrix, B and DB are M by L matrices, and DA and DB satisfy the following relationship : the Frobenius norm of CDA;DBl is a minimum and each column of B+DB is in the range of A+DA. Subroutine DTLS allows multiple right hand sides. It also solves determined (M=N) and underdetermined (M<N) sets of equations by computing the minimum norm solution. It is assumed that all preprocessing measures ( scaling,coordinate transformations,whitening, . . . 1 of the data have been performed in advance.

Iterative speed improvement for solving slowly varying total least squares problems

Many problems in signal processing, modal analysis, system identification, etc., give rise to an overdetermined set of linear equations AX ≈ B. Whenever both the observation matrix B and the data matrix A are inaccurate, the total least squares (TLS) method is appropriate for estimating the unknown parameters X. Since the TLS solution is deduced from only one right singular vector or, in general, a basis of the right singular subspace associated with the smallest singular values of the data [ A; B], the computational speed can be improved by only calculating those desired base vectors. If a priori information is available (e.g. the TLS solution at a previous time step when slowly varying sets of equations must be solved at each time instant), iterative methods are appropriate to compute those base vectors. It is shown in this paper that inverse iteration is the most efficient iterative technique for solving generic TLS problems of known rank. Two algorithms for subspace inverse iteration are presented. Their convergence properties and applicability in solving TLS problems are analysed. Based on the convergence rate and the operation counts in each iteration step, these iterative algorithms are compared in efficiency with the direct computation methods—classical TLS and partial TLS. In particular, the efficiency of these methods is illustrated in a practical real life problem, namely multiple input estimation of frequency response functions.

Analysis and Solution of the Nongeneric Total Least Squares Problem

Total least squares (TLS) is one method of solving overdetermined sets of linear equations $AX \approx B$ that is appropriate when there are errors in both the observation matrix B and the data matrix A. Golub and Van Loan (G. H. Golub and C. F. Van Loan, SIAM J. Numer. Anal., 17 (1980), pp. 883–893) introduced this method into the field of numerical analysis and developed an algorithm based on the singular value decomposition. However in some TLS problems, called nongeneric, their algorithm fails to compute a finite TLS solution. This paper generalizes their TLS computations in order to solve these nongeneric TLS problems. The authors describe the properties of those problems and prove that the proposed generalization remains optimal with respect to the TLS criteria for any number of observation vectors in B if additional constraints are imposed. Finally, the TLS computation is summarized in one algorithm which includes the proposed generalization.

An Analysis of the Total Least Squares Problem

Total Least Squares (TLS) is a method of fitting that is appropriate when there are errors in both the observation vector $b(m \times 1)$ and in the data matrix $A(m \times n)$. The technique has been discussed by several authors, and amounts to fitting a “best” subspace to the points $(a_i^T ,b_i ),i = 1, \cdots ,m$, where $a_i^T $ is the ith row of A. In this paper a singular value decomposition analysis of the TLS problem is presented. The sensitivity of the TLS problem as well as its relationship to ordinary least squares regression is explored. An algorithm for solving the TLS problem is proposed that utilizes the singular value decomposition and which provides a measure of the underlying problem’s sensitivity.