Weighted Total Least Squares Research Articles

Low frequency load identification under high noise level using weighted total least squares

Load identification in structural dynamics is an ill-conditioned inverse problem, and the errors exist in both the transfer function matrix and the response vector, which have a great influence on the accuracy of the identification, especially at the low frequency and high noise. Although the Total Least Squares (TLS) method can consider the errors in the transfer function matrix and the response vector, the problem of unequal precision between them has not been properly solved. In this article, the Weighted Total Least Squares (WTLS) method for load identification is presented. First, the variance–covariance matrices are constructed using stochastic information, which can consider the unequal precision of the data and make the solution less biased and more precise. Then, Tikhonov regularization is used to regularize the WTLS method to reduce the ill-condition of the transfer function matrix. To validate the performance of the WTLS method under different noise levels, a load identification simulation with four excitation loads on a plate and a load identification experiment with three excitation loads on a steel plate were studied. The results show that when the noise levels of the transfer function and the response are different, the accuracy of the TLS solution is poor. Especially when the noise levels are high, the identification deviations are large. The WTLS method considers the unequal noise levels using the weight matrices, which provides more stable load estimates at different noise levels. And the identification deviations are greatly reduced compared to the TLS method.

Dual-quaternion-based iterative algorithm of the three dimensional coordinate transformation

Nowadays a unit quaternion is widely employed to represent the three-dimensional (3D) rotation matrix and then applied to the 3D similarity coordinate transformation. A unit dual quaternion can describe not only the 3D rotation matrix but also the translation vector meanwhile. Thus it is of great potentiality to the 3D coordinate transformation. The paper constructs the 3D similarity coordinate transformation model based on the unit dual quaternion in the sense of errors-in-variables (EIV). By means of linearization by Taylor's formula, Lagrangian extremum principle with constraints, and iterative numerical technique, the Dual Quaternion Algorithm (DQA) of 3D coordinate transformation in weighted total least squares (WTLS) is proposed. The algorithm is capable to not only compute the transformation parameters but also estimate the full precision information of computed parameters. Two numerical experiments involving an actual geodetic datum transformation case and a simulated case from surface fitting are demonstrated. The results indicate that DQA is not sensitive to the initial values of parameters, and obtains the consistent values of transformation parameters with the quaternion algorithm (QA), regardless of the size of the rotation angles and no matter whether the relative errors of coordinates (pseudo-observations) are small or large. Moreover, the DQA is advantageous to the QA. The key advantage is the improvement of estimated precisions of transformation parameters, i.e. the average decrease percent of standard deviations is 18.28%, and biggest decrease percent is 99.36% for the scaled quaternion and translations in the geodetic datum transformation case. Another advantage is the DQA implements the computation and precision estimation of traditional seven transformation parameters (which still are frequent used yet) from dual quaternion, and even could perform the computation and precision estimation of the scaled quaternion.Graphical

Multivariate Weighted Total Least Squares Based on the Standard Least-Squares Theory

The weighted total least squares (WTLS) has been widely used in many geodetic problems to solve the error-in-variable (EIV) models in which both the observation vector and the design matrix contain random errors. This method is widely applied in its univariate form, where the observations and unknown coefficients appear in vector forms. However, in some geodetic problems, data sets appear in more than one dimension, and the vector representation of the univariate model may not be suitable to efficiently solve the problem. The observation and unknown parameter vectors can then be replaced with their counterparts in matrix representations in a multivariate model. In this paper, we propose a simple, fast, and flexible procedure for solving the multivariate WTLS (MWTLS) problem using the standard least squares theory. The method has the capability of applying to large-size and high-dimensional data sets. Our numerical experiments on both simulated and real datasets demonstrate the high performance of the proposed method for solving multivariate WTLS problems. In terms of computational complexity, our method outperforms the existing state-of-the-art methods, both numerically and analytically.

Algorithms and statistical analysis for linear structured weighted total least squares problem

Weighted total least squares (WTLS) have been regarded as the standard tool for the errors-in-variables (EIV) model in which all the elements in the observation vector and the coefficient matrix are contaminated with random errors. However, in many geodetic applications, some elements are error-free and some random observations appear repeatedly in different positions in the augmented coefficient matrix. It is called the linear structured EIV (LSEIV) model. Two kinds of methods are proposed for the LSEIV model from functional and stochastic modifications. On the one hand, the functional part of the LSEIV model is modified into the errors-in-observations (EIO) model. On the other hand, the stochastic model is modified by applying the Moore-Penrose inverse of the cofactor matrix. The algorithms are derived through the Lagrange multipliers method and linear approximation. The estimation principles and iterative formula of the parameters are proven to be consistent. The first-order approximate variance-covariance matrix (VCM) of the parameters is also derived. A numerical example is given to compare the performances of our proposed three algorithms with the STLS approach. Afterwards, the least squares (LS), total least squares (TLS) and linear structured weighted total least squares (LSWTLS) solutions are compared and the accuracy evaluation formula is proven to be feasible and effective. Finally, the LSWTLS is applied to the field of deformation analysis, which yields a better result than the traditional LS and TLS estimations.

Thinned planar antenna array synthesis using a hybrid approach combining almost difference sets and weighted total least squares method

This paper introduces a hybrid approach for designing thinned planar antenna arrays with low peak sidelobe levels (PSLLs). The standard almost difference set (ADS) approach allows the analytical determination of the turned-on element positions within the array configuration, but it provides suboptimal performance in PSLL reduction. To address this limitation, a hybrid method is presented by integrating the ADS approach with the weighted total least squares (WTLS) method. Simulation results from various case studies demonstrate the effectiveness of the hybrid ADS-WTLS method in designing thinned planar antenna arrays with low PSLLs. Additionally, the performance of the synthesized designs obtained through the hybrid method is validated using electromagnetic (EM) simulations.

A New Gain-Phase Error Pre-Calibration Method for Uniform Linear Arrays.

In this paper, we consider the gain-phase error calibration problem for uniform linear arrays (ULAs). Based on the adaptive antenna nulling technique, a new gain-phase error pre-calibration method is proposed, requiring only one calibration source with known direction of arrival (DOA). In the proposed method, a ULA with M array elements is divided into M-1 sub-arrays, and the gain-phase error of each sub-array can be uniquely extracted one by one. Furthermore, in order to obtain the accurate gain-phase error in each sub-array, we formulate an errors-in-variables (EIV) model and present a weighted total least-squares (WTLS) algorithm by exploiting the structure of the received data on sub-arrays. In addition, the solution to the proposed WTLS algorithm is exactly analyzed in the statistical sense, and the spatial location of the calibration source is also discussed. Simulation results demonstrate the efficiency and feasibility of our proposed method in both large-scale and small-scale ULAs and the superiority to some state-of-the-art gain-phase error calibration approaches.

An Efficient Algorithm for Infrared Earth Sensor with a Large Field of View

Infrared Earth sensors with large-field-of-view (FOV) cameras are widely used in low-Earth-orbit satellites. To improve the accuracy and speed of Earth sensors, an algorithm based on modified random sample consensus (RANSAC) and weighted total least squares (WTLS) is proposed. Firstly, the modified RANSAC with a pre-verification step was used to remove the noisy points efficiently. Then, the Earth's oblateness was taken into consideration and the Earth's horizon was projected onto a unit sphere as a three-dimensional (3D) curve. Finally, the TLS and WTLS were used to fit the projection of the Earth horizon. With the help of TLS and WTLS, the accuracy of the Earth sensor was greatly improved. Simulated images and on-orbit infrared images obtained via the satellite Tianping-2B were used to assess the performance of the algorithm. The experimental results demonstrate that the method outperforms RANSAC, M-estimator sample consensus (MLESAC), and Hough transformation in terms of speed. The accuracy of the algorithm for nadir estimation is approximately 0.04° (root-mean-square error) when Earth is fully visible and 0.16° when the off-nadir angle is 120°, which is a significant improvement upon other nadir estimation algorithms.

Total Msplit estimation

Msplit estimation is a method that enables the estimation of mutually competing versions of parameters in functional observation models. In the presented study, the classical functional models found in it are replaced by errors-in-variables (EIV) models. Similar to the weighted total least-squares (WTLS) method, the random components of these models were assigned covariance matrix models. Thus, the proposed method, named Total Msplit (TMsplit) estimation, corresponds to the basic rules of WTLS. TMsplit estimation objective function is constructed using the components of squared Msplit and WTLS estimation objective functions. The TMsplit estimation algorithm is based on the Gauss–Newton method that is applied using a linear approximation of EIV models. The basic properties of the method are presented using examples of the estimation of regression line parameters and the estimation of parameters in a two-dimensional affine transformation.

Trajectory tracking under compound noise environments based on weighted total least squares and forney-style factor graph

In compound noise environments, both additive noises and multiplicative noises influence the Time-of-arrival-based (TOA-based) distance measurements, which challenge the source node localization and trajectory tracking in practical sensor networks. In order to realize trajectory tracking under such noise conditions, in this paper, a compound noise parameter estimation algorithm based on Weighted Total Least Squares (WTLS) and a source node tracking algorithm based on Forney-style Factor Graph (FFG) are presented. Four parameters of the compound noise model are estimated based on ranging observations between position-known anchor nodes, and the trajectory tracking is achieved by applying a noise pre-standardized procedure in the FFG. Monte-Carlo simulations evaluate the performance of the proposed system, and the proposed noise parameter estimation algorithm performs better than the traditional Least Square (LS) method and the proposed trajectory tracking algorithm is realizable and robust.

Profile analysis with reconstruction robustness for measurement data subject to outliers.

In the surface profile analysis, there are often a few observations that contain outliers. Due to the existence of outliers, the application of non-robust reconstruction algorithms for measurement data will become a huge problem because these methods are often sensitive to outliers and the approximation effectiveness will be greatly aggravated. In view of this, this paper presents a novel angle-based moving total least squares reconstruction method, to the best of our knowledge, that applies two-step pre-treatment to handle outliers. The first step is an abnormal point detection process that characterizes the geometric features of discrete points in the support domain through a new angle-based parameter constructed by total least square. Then, the point with the largest anomaly degree is removed, and a relevant weight function is defined to adjust the weights of the remaining points. After pre-treatment, the final estimates are calculated by weighted total least squares (WTLS) based on the compact weight function. The detection and removal of outliers are automatic, and there is no need to set a threshold value artificially, which effectively avoids the adverse impacts of human operation. Numerical simulations and experiments verify the applicability of the proposed algorithm as well as its accuracy and robustness.

A simple iterative algorithm based on weighted least-squares for errors-in-variables models: Examples of coordinate transformations

Abstract Although weighted total least-squares (WTLS) adjustment within the errors-in-variables (EIV) model is a rigorous method developed for parameter estimation, its exact solution is complicated since the matrix operations are extremely time-consuming in the whole repeated iteration process, especially when dealing with large data sets. This paper rewrites the EIV model to a similar Gauss–Markov model by taking the random error of the design matrix and observations into account, and reformulates it as an iterative weighted least-squares (IWLS) method without complicated theoretical derivation. IWLS approximates the “exact solution” of the general WTLS and provides a good balance between computational efficiency and estimation accuracy. Because weighted LS (WLS) method has a natural advantage in solving the EIV model, we also investigate whether WLS can directly replace IWLS and WTLS to implement the EIV model when the parameters in the EIV model are small. The results of numerical experiments confirmed that IWLS can obtain almost the same solution as the general WTLS solution of Jazaeri [21] and WLS can achieve the same accuracy as the general WTLS when the parameters are small.

A New Reconstruction Method for Measurement Data With Multiple Outliers

Measurement data sometimes contains a few outliers that have a large deviation from actual value. For such data, the reconstruction results will be greatly distorted around the outliers. To solve this problem, a robust surface approximation method based on moving total least squares (MTLS) is presented in this paper, in which an improved random sample consensus (RANSAC) and classification and regression tree (CART) are introduced into the support domain to detect outliers. Firstly, the residuals of points in the support domain are calculated by the improved RANSAC with an introduced parameter, and then the abnormal residuals are identified by CART algorithm. The points corresponding to abnormal residuals are considered as abnormal points. After the abnormal points are eliminated, the remaining data is processed by weighted total least squares (WTLS) to obtain local fitting parameters. This method can avoid the adverse impact of human intervention and over elimination. The reconstruction results of the numerical simulation and measurement experiment illustrate the effectiveness of our method.

Weighted total least squares–Bayes filter-based estimation of relative pose for a space non-cooperative unknown target without a priori knowledge

The estimation of relative pose of a space non-cooperative target is an attractive yet challenging task due to the complexity of the target background and illumination, and the lack of a priori knowledge. Unfortunately, these negative factors have a grave impact on the accuracy of the estimation and the robustness of the filter algorithms. In response, this paper proposes a novel filter algorithm for estimating relative pose to improve robustness based on a stereovision system. First, to obtain a coarse relative pose, the weighted total least squares (WTLS) algorithm is adopted to estimate the relative pose based on several feature points. The resulting relative pose is fed into the subsequent filter scheme as an observation quantity. Second, the classic Bayes filter is exploited to estimate the relative state except for moment-of-inertia ratios. Additionally, the one-step prediction results are used as feedback for WTLS initialization. The proposed algorithm successfully eliminates the dependence on continuous tracking of several fixed points. Finally, comparison experiments demonstrate that the proposed algorithm presents a better performance in terms of robustness and convergence time.

Enhancement of Computational Efficiency for Weighted Total Least Squares

Weighted total least-squares (WTLS) adjustment is a rigorous method used for estimating parameters in the errors-in-variables (EIV) model. However, its computational efficiency is limited due to the large number of matrix operations involved, which are extremely time-consuming, particularly when processing large data sets. Based on the structural characteristics of the EIV model, the design matrix is divided into a constant matrix and a random matrix. Then the EIV model is rewritten as a general structured model and reformulate it as an efficient WTLS algorithm, which only attaches a weight matrix to the random matrix to reduce the size of the matrices involved in the iterative process. In addition, the proposed algorithm does not reestimate the random matrix in each iteration. All of this helps to improve computational efficiency. Numerical results confirm that the proposed algorithm can obtain the same accuracy as other existing improved algorithms, but using the same hardware, which requires significantly less time and memory.

Location and estimation of multiple outliers in weighted total least squares

Although the weighted total least squares (WTLS) adjustment is a rigorous method for estimating parameters in errors-in-variables (EIV) models, its solution is unreliable if the design matrix and/or observations contain multiple outliers. Existing methods are not capable of fully eliminating the influence of multiple outliers on the parameters. First, we reformulate an EIV model as a correction model by taking the outliers of the design matrix and observations into account, and we introduce the concept of total outliers to simplify the problem of detecting multiple outliers in EIV models. Then, based on the effect of the outliers on the estimated posteriori variance, we develop a full search algorithm to form a location matrix that can describe the location of the total outliers. Next, we derive a set of formulae to numerically estimate the unknown model parameters and total outliers simultaneously. Finally, the results of three numerical experiments show that the proposed method can effectively eliminate the influence of the outliers and obtain reliable parameters.

Alternative formulae for robust Weighted Total Least-Squares solutions for Errors-In-Variables models

Weighted Total Least-Squares (WTLS) can optimally solve the issue of parameter estimation in the Errors-In-Variables (EIV) model; however, this method is relatively sensitive to outliers that may exist in the observation vector and/or the coefficient matrix. Hence, an attempt to identify/suppress those outliers is in progress and will ultimately lead to a novel robust estimation procedure similar to the one used in the Gauss-Markov model. The method can be considered as a follow-up to the WTLS solution formulated with the standard Least-Squares framework. We utilize the standardized total residuals to construct the equivalent weights, and apply the median method to obtain a robust estimator of the variance to provide good robustness in the observation and structure spaces. Moreover, a preliminary analysis for the robustness of related estimators within the EIV model is conducted, which shows that the redescending M-estimates are more robust than the monotonic ones. Finally, the efficacy of the proposed algorithm is demonstrated through two applications, i.e. 2D affine transformation and linear regression on simulated data and on real data with some assumptions. Unfortunately, the proposed algorithm may not be reliable for detecting multiple outliers. Therefore, MM-estimates within the EIV model need to be investigated in further research.

Structured Total Least Squares Method Based on Variable Projection

Objectives To solve the problem that the coefficient matrix and the observation vector have structured characteristics for many practical applications in the field of surveying and mapping, that is, the coefficient matrix and the observation vector contain fixed quantities (or even fixed columns) and random quantities, and the random quantities at different positions are linearly related. Methods Starting from the errors-in-variables (EIV) function model, the augmented matrix composed of the coefficient matrix and the observation vector is expressed as an affine function form, and the function model is reconstructed by the variable projection method. Then, a structured total least squares (STLS) estimation algorithm is derived by the Lagrange method. Results The example results show that the proposed algorithm is consistent with the existing weighted or structured total least squares estimation algorithms that can solve structured problem in the coefficient matrix and the observation vector. Compared with the weighted total least squares (WTLS) estimation algorithms, the proposed algorithm only needs to establish a positive definite weight matrix of independent random variables and reduces the number of estimates. Compared with other STLS estimation algorithms, the proposed algorithm takes into account the overall structure of the coefficient matrix and the observation vector. Conclusions This shows the effectiveness of the proposed algorithm. The STLS estimation and the WTLS estimation ensure the statistical optimality from the perspective of the function model and the stochastic model, respectively.

A novel inequality-constrained weighted linear mixture model for endmember variability

Spectral unmixing has trigged considerable attention to both multispectral and hyperspectral data processing. Among different kinds of spectral unmixing models, the linear mixture model (LMM) has been widely applied with its simplicity in light scattering mechanisms and flexibility in different scenarios. Recently, a variety of optimized LMMs have been developed to enhance the decomposing ability. However, two major challenges still exist: endmember variability and global optimum solution with the inequality constraint. In this study, aiming to avoid the influence of endmember variability and to extend the application of LMM, a novel inequality-constrained weighted linear mixture model (IWLMM) is proposed. Based on the errors-in-variables (EIV) model and the theory of weighted total least squares (WTLS), the issue of endmember variability and stochastic errors of both endmembers and mixed pixels are eliminated. With the non-negativity constraint, a brief optimal algorithm to solve the IWLMM is generated as well. In order to indicate the advantages of proposed IWLMM, comparative experiments with the conventional fully constrained least squares (FCLS), three currently proposed LMM-based models (perturbed LMM (PLMM), extended LMM (ELMM) and augmented LMM (ALMM)) and three physics-based nonlinear mixture models (bilinear-Fan model (BFM), generalized bilinear model (GBM) and polynomial post-nonlinear model (PPNM)) on four synthetic and real scenarios with hyperspectral and multispectral datasets were performed. The experimental results suggest the IWLMM outperforms the other seven linear and nonlinear unmixing models after adding the correction for endmember variability by modeling the perturbation and scaling factors of spectral features simultaneously. It has an impressive ability for decomposing and reconstructing pixels.

New First-Order Approximate Precision Estimation Method for Parameters in an Errors-in-Variables Model

To evaluate the posterior precision of weighted total least-squares (WTLS) estimates in an errors-in-variables model, first-order approximate precision estimation (FOA) methods are usually used. However, FOAs might not be valid if the underlying assumption is invalid, and this assumption has not been sufficiently proven. Therefore, this paper investigates the validity of the latent assumption and proposes a new first-order approximate (NFOA) precision estimation method to avoid the underlying assumption and design a corresponding algorithm. The difference between NFOA and FOA is formulated and analyzed. The proposed NFOA method is tested by a simulated classic straight-line fitting example with six scenarios and a simulated three-dimensional (3D) affine transformation experiment with four scenarios, and the mean values of the standard deviation of true errors (MSDTE) and FOA are also calculated for comparison. The results numerically indicate that NFOA works better than FOA and is close to the MSDTE, which means that NFOA can evaluate the precision of estimated parameters more reasonably and accurately.

Iteratively reweighted total least squares for PEIV model

The weighted total least-squares (WTLS) estimate for the partial EIV (PEIV) model is very susceptible to outliers. In view of outliers in observations and coefficient matrix, a robust estimate of WTLS which makes using of iteratively reweighted technique (IRTLS) for the PEIV model is developed by combining a new two-step iterated algorithm of the WTLS estimate with the sensitivity-analysis based on the robust M-estimation. The uniformly most powerful test statistics are constructed to determine the down-weighting factor and the variance of unit weight is estimated by least median squares (LMS) method possessing high break-down point. Depended on the PEIV model, two different down-weighting schemes are presented. In the first scheme down-weighting is only implemented for the coefficient matrix and not for observations when the elements of the coefficient matrix are estimated, and the second scheme is contrary. The most attractive aspect is that the suggested computational formulae are the same with the traditional robust least-squares (LS) methods. The two-dimensional affine transformation and liner fitting example are analyzed, and some comparisons are performed with different available weight functions. The proposed approach (Scheme 1) proves to be a powerful tool in detecting outliers for the PEIV model.