Total Least Squares Problem Research Articles

Forward-partial inverse-half-forward splitting algorithm for solving monotone inclusions

In this paper we provide a splitting algorithm for solving coupled monotone inclusions in a real Hilbert space involving the sum of a normal cone to a vector subspace, a maximally monotone, a monotone-Lipschitzian, and a cocoercive operator. The proposed method takes advantage of the intrinsic properties of each operator and generalizes the method of partial inverses and the forward-backward-half forward splitting, among other methods. At each iteration, our algorithm needs two computations of the Lipschitzian operator while the cocoercive operator is activated only once. By using product space techniques, we derive a method for solving a composite monotone primal-dual inclusions including linear operators and we apply it to solve constrained composite convex optimization problems. Finally, we apply our algorithm to a constrained total variation least-squares problem and we compare its performance with efficient methods in the literature.

Ground Maneuvering Target Focusing via High-Order Phase Correction in High-Squint Synthetic Aperture Radar

Moving target imaging in high-squint synthetic aperture radar (SAR) shows great potential for reconnaissance and surveillance tasks. For the desired resolution, high-squint SAR has a long-time coherent processing interval (CPI). In this case, the maneuvering motion of the moving target usually causes high-order phase terms in the echoed data, which cannot be neglected for precise focusing. Many ground moving target imaging (GMTIm) algorithms have been proposed in the literature, but some high-order phase terms remain uncompensated in high-squint SAR. For this problem, a high-order phase correction-based GMTIm (HPC-GMTIm) method is proposed in this paper. We assumed that the target of interest has a constant velocity in the subaperture CPI, but maneuvering motion parameters for the whole CPI. Within the short subaperture CPI, the target signal can be simplified as a three-order phase expression, and the instantaneous Doppler frequency (DF) was estimated by some time–frequency analysis tools, including the Hough transform and the fractional Fourier transform. For the whole CPI, the subaperture, the instantaneous DF was combined to form a total least-squares problem, outputting the undetermined phase coefficients. Using the proposed local-to-global processing chain, all high-order phase terms can be estimated and corrected, which outperforms existing methods. The effectiveness of the HPC-GMTIm method is demonstrated by real measured high-squint SAR data.

A contribution to the conditioning of the mixed least-squares scaled total least-squares problem

A contribution to the conditioning of the mixed least-squares scaled total least-squares problem

Symmetrical Augmented System of Equations for the Parameter Identification of Discrete Fractional Systems by Generalized Total Least Squares

This paper is devoted to the identification of the parameters of discrete fractional systems with errors in variables. Estimates of the parameters of such systems can be obtained using generalized total least squares (GTLS). A GTLS problem can be reduced to a total least squares (TLS) problem. A total least squares problem is often ill-conditioned. To solve a TLS problem, a classical algorithm based on finding the right singular vector or an algorithm based on an augmented system of equations with complex coefficients can be applied. In this paper, a new augmented system of equations with real coefficients is proposed to solve TLS problems. A symmetrical augmented system of equations was applied to the parameter identification of discrete fractional systems. The simulation results showed that the use of the proposed symmetrical augmented system of equations can shorten the time for solving such problems. It was also shown that the proposed system can have a smaller condition number.

Generalized correntropy induced metric based total least squares for sparse system identification

The total least squares (TLS) method has been successfully applied to system identification in the errors-in-variables (EIV) model, which can efficiently describe systems where input–output pairs are contaminated by noise. In this paper, we propose a new gradient-descent TLS filtering algorithm based on the generalized correntropy induced metric (GCIM), called as GCIM-TLS, for sparse system identification. By introducing GCIM as a penalty term to the TLS problem, we can achieve improved accuracy of sparse system identification. We also characterize the convergence behaviour analytically for GCIM-TLS. To reduce computational complexity, we use the first-order Taylor series expansion and further derive a simplified version of GCIM-TLS. Simulation results verify the effectiveness of our proposed algorithms in sparse system identification.

Condition numbers for the truncated total least squares problem and their estimations

AbstractIn this paper, we present explicit expressions for the mixed and componentwise condition numbers of the truncated total least squares (TTLS) solution of under the genericity condition, where A is a m × n real data matrix and is a real m‐vector. Moreover, we reveal that normwise, componentwise, and mixed condition numbers for the TTLS problem can recover the previous corresponding counterparts for the total least squares (TLS) problem when the truncated level of the TTLS problem is n. When A is a structured matrix, the structured perturbations for the structured truncated TLS (STTLS) problem are investigated and the corresponding explicit expressions for the structured normwise, componentwise, and mixed condition numbers for the STTLS problem are obtained. Furthermore, the relationships between the structured and unstructured normwise, componentwise, and mixed condition numbers for the STTLS problem are studied. We devise reliable condition estimation algorithms for the TTLS problem by utilizing small‐sample statistical condition estimation techniques. The proposed condition estimation algorithms employ the singular value decomposition (SVD) of the augmented matrix to reduce the computational complexity, where both unstructured and structured normwise, mixed, and componentwise condition estimations are considered. The proposed condition estimation algorithms can be integrated into the SVD‐based direct solver for the small and medium size TTLS problem to give the error estimation for the numerical TTLS solution. Numerical experiments are reported to illustrate the reliability of the proposed condition estimation algorithms.

Total Least-Squares Determination of Body Segment Attitude.

To examine segment and joint attitudes when using image-based motion capture, it is necessary to determine the rigid body transformation parameters from an inertial reference frame to a reference frame fixed in a body segment. Determine the rigid body transformation parameters must account for errors in the coordinates measured in both reference frames, a total least-squares problem. This study presents a new derivation that shows that a singular value decomposition-based method provides a total least-squares estimate of rigid body transformation parameters. The total least-squares method was compared with an algebraic method for determining rigid body attitude (TRIAD method). Two cases were examined: case 1 where the positions of a marker cluster contained noise after the transformation, and case 2 where the positions of a marker cluster contained noise both before and after the transformation. The white noise added to position data had a standard deviation from zero to 0.002 m, with 101 noise levels examined. For each noise level, 10 000 criterion attitude matrices were generated. Errors in estimating rigid body attitude were quantified by computing the angle, error angle, required to align the estimated rigid body attitude with the actual rigid body attitude. For both methods and cases, as the noise level increased the error angle increased, with errors larger for case 2 compared with case 1. The singular value decomposition (SVD)-based method was superior to the TRIAD algorithm for all noise levels and both cases, and provided a total least-squares estimate of body attitude.

General Total Least Squares Theory for Geodetic Coordinate Transformations

Datum transformations are a fundamental issue in geodesy, Global Positioning System (GPS) science and technology, geographical information science (GIS), and other research fields. In this study, we establish a general total least squares (TLS) theory which allows the errors-in-variables model with different constraints to formulate all transformation models, including affine, orthogonal, similarity, and rigid transformations. Through the adaptation of the transformation models to the constrained TLS problem, the nonlinear constrained normal equation is analytically derived, and the transformation parameters can be iteratively estimated by fixed-point formulas. We also provide the statistical characteristics of the parameter estimator and the unit of precision of the control points. Two examples are given, as well as an analysis of the results on how the estimated quantities vary when the number of constraints becomes larger.

Condition numbers of the multidimensional total least squares problems having more than one solution

Recently, the condition numbers of the total least squares (TLS) problems having a unique solution have been studied at length in Zheng et al. (SIAM J. Matrix Anal. Appl. 38: 924–948, 2017). However, it is known that the TLS problem may have no solution, and even if an existing solution, it may not be unique. As a continuation of their work, in this paper, we investigate the condition numbers of the minimum Frobenius norm solution of the (multidimensional) TLS problem when having more than one solution. The tight and computable upper bound estimates of the normwise, mixed, and componentwise condition numbers are respectively derived. Some numerical experiments are performed to illustrate the tightness of these upper bounds.

Quantum algorithm for total least squares data fitting

The total least squares (TLS) method is widely used in data-fitting. Compared with the least squares fitting method, the TLS fitting takes into account not only observation errors, but also errors from the measurement matrix of the variables. In this work, the TLS problem is transformed to finding the ground state of a Hamiltonian matrix. We propose quantum algorithms for solving this problem based on quantum simulation of resonant transitions. Our algorithms can achieve at least polynomial speedup over the known classical algorithms.

On TLS formulation and core reduction for data fitting with generalized models

The total least squares (TLS) framework represents a popular data fitting approach for solving matrix approximation problems of the form A(X)≡AX≈B. A general linear mapping on spaces of matrices A:X⟼B can be represented by a fourth-order tensor which is in the AX≈B case highly structured. This has a direct impact on solvability of the corresponding TLS problem, which is known to be complicated. Thus this paper focuses on several generalizations of the model A: the bilinear model, the model of higher Kronecker rank, and the fully tensorized model. It is shown how the corresponding generalization of the TLS formulation induces enrichment of the search space for the data corrections. Solvability of the resulting minimization problem is studied. Furthermore, extension of the so-called core reduction to the bilinear model is presented. For the fully tensor model, its relation to a particular single right-hand side TLS problem is derived. Relationships among individual formulations are discussed.

Towards tensor generalizations of TLS & core problem theory

AbstractDespite the wide attention, there are still several not well understood parts in the theory of total least squares (TLS) formulation of linear algebraic approximation problems. In particular, in the problems with multiple (matrix) right‐hand sides one can ask about the meaning of the nongeneric solution in the context of the original data, the nonexistence of the TLS solution for the so‐called irreducible core problems, etc. In the single (vector) right‐hand side TLS, these problems can be easily explained through the core problem theory or simply do not appear. Here we summarize, how the existing TLS and core problem theory can be generalized to problems with tensor right‐hand sides. Such generalization also gives a natural and wider context for further analysis of the above mentioned questions.

Weighted Total Least Squares with Constraints: Alternative Derivation without Using Lagrange Multipliers

AbstractThis technical note presents alternative derivations for the weighted total least-squares (WTLS) problem subject to weighted and hard constraints. The derivations do not take into account t...

Parameter estimation in 3D affine and similarity transformation: implementation of variance component estimation

Three-dimensional (3D) coordinate transformations, generally consisting of origin shifts, axes rotations, scale changes, and skew parameters, are widely used in many geomatics applications. Although in some geodetic applications simplified transformation models are used based on the assumption of small transformation parameters, in other fields of applications such parameters are indeed large. The algorithms of two recent papers on the weighted total least-squares (WTLS) problem are used for the 3D coordinate transformation. The methodology can be applied to the case when the transformation parameters are generally large of which no approximate values of the parameters are required. Direct linearization of the rotation and scale parameters is thus not required. The WTLS formulation is employed to take into consideration errors in both the start and target systems on the estimation of the transformation parameters. Two of the well-known 3D transformation methods, namely affine (12, 9, and 8 parameters) and similarity (7 and 6 parameters) transformations, can be handled using the WTLS theory subject to hard constraints. Because the method can be formulated by the standard least-squares theory with constraints, the covariance matrix of the transformation parameters can directly be provided. The above characteristics of the 3D coordinate transformation are implemented in the presence of different variance components, which are estimated using the least squares variance component estimation. In particular, the estimability of the variance components is investigated. The efficacy of the proposed formulation is verified on two real data sets.

Mixed and componentwise condition numbers for a linear function of the solution of the total least squares problem

In this paper, we consider the mixed and componentwise condition numbers for a linear function Lx of the solution to the total least squares (TLS) problem. We derive the explicit expressions of the mixed and componentwise condition numbers through the dual techniques under both unstructured and structured componentwise perturbations. The sharp upper bounds for condition numbers are obtained. An efficient condition estimation algorithm is proposed, which can be integrated into the iterative method for solving large scale TLS problems. Moreover, the new derived condition number expressions can recover the previous results on the condition analysis for the TLS problem when L=In. Numerical experiments show the effectiveness of the introduced condition numbers and condition estimation algorithm.

Small sample statistical condition estimation for the total least squares problem

In this paper, under the genericity condition, we study the condition estimation of the total least squares (TLS) problem based on small sample condition estimation (SCE), which can be incorporated into the direct solver for the TLS problem via the singular value decomposition (SVD) of the augmented matrix [A, b]. Our proposed condition estimation algorithms are efficient for the small and medium size TLS problem because they utilize the computed SVD of [A, b] during the numerical solution to the TLS problem. Numerical examples illustrate the reliability of the algorithms. Both normwise and componentwise perturbations are considered. Moreover, structured condition estimations are investigated for the structured TLS problem.

Weighted coordinate transformation formulated by standard least-squares theory

This paper presents a universal model of weighted coordinate transformation, i.e. transformation considering the errors of coordinates in both coordinate systems. It is intrinsically one of the typical examples of ‘error-in-variables’ (EIV) models. The proposed method of LS theory application on weighted coordinate transformation does not impose any constraints on the form of functional relationship among stochastic variables. Since the basic idea is to generalise Gauss–Markov model (GMM) by introduction of so-called ‘total residuals’, the proposed procedure is named ‘Generalised Gauss–Markov model’. Formulation of expressions for estimation of unknown transformation parameters is theoretically confirmed using the Gauss–Helmert model (GHM) and three different modifications of the GMM. The proposed procedure is in its essence a strict solution to total least-squares (unweighted) and weighted total least-squares problem in coordinate transformation. This thesis is experimentally confirmed by comparison of its results with those found in four characteristic examples from the literature.

Reweighted l1 regularized TLS linear neuron for the sparse system identification

This paper presents a neural approach for the parameter estimation of adaptive sparse system identification based on a MCA EXIN (minor component analysis by excitatory and inhibitory learning) linear neuron for TLS (total least squares) problem. We incorporate a log-sum (or Reweighted l1) penalty into the cost function of TLS EXIN (total least squares by excitatory and inhibitory learning) in order to identify the sparse system. The given computer simulations illustrate that the neural approach considerably outperforms the existing TLS EXIN method as well as the LMS-type adaptive methods in the sparse system.

An alternating iterative minimisation algorithm for the double-regularised total least square functional

The total least squares (TLS) method is a successful approach for linear problems if both the right-hand side and the operator are contaminated by some noise. For ill-posed problems, a regularisation strategy has to be considered to stabilise the computed solution. Recently a double regularised TLS method was proposed within an infinite dimensional setup and it reconstructs both function and operator, reflected on the bilinear forms Our main focuses are on the design and the implementation of an algorithm with particular emphasis on alternating minimisation strategy, for solving not only the double regularised TLS problem, but a vast class of optimisation problems: on the minimisation of a bilinear functional of two variables.

Weighted total least-squares with constraints: a universal formula for geodetic symmetrical transformations

Although the analytical solutions for total least-squares with multiple linear and single quadratic constraints were developed quite recently in different geodetic publications, these methods are restricted in number and type of constraints, and currently their computational efficiency and applications are mostly unknown. In this contribution, it is shown how the weighted total least-squares (WTLS) problem with arbitrary applicable constraints can be solved based on a Newton type methodology. This iterative process with quadratic convergence is expanded upon to become a compact solution for the WTLS with or without constraints. This compact solution is then further interpreted as a universal formula for the symmetrical adjustment of the errors-in-variables model which represents affine, similarity and rigid transformations in two- and three-dimensional space. Furthermore, statistical analysis of the constrained WTLS including the first-order approximation of precision and the bias was investigated. In order to substantiate our proposed method’s applicability, it was used to solve the affine, similarity and rigid transformation problem in two- and three-dimensional cases, where the structure of the coefficient matrix and multiple constraints were taken into account simultaneously.