Weighted Least-squares Problem Research Articles

Elliptic Localization of a Moving Object by Transmitter at Unknown Position and Velocity: A Semidefinite Relaxation Approach

This paper investigates the elliptic localization for moving object problem from time delay (TD) and Doppler frequency shift (DFS) measurements, where the transmitter position and velocity are unknown. The transmitter is not perfectly time syncronized such that unknown offsets exist in the TD and DFS measurements. We propose to jointly estimate the object and transmitter positions and velocities and the offsets. Using the TD and DFS measurements from both the indirect and direct paths between the transmitter and the receivers, we formulate a non-convex weighted least squares (WLS) problem. Local convergence may occur when solving the non-convex WLS problem, implying that good estimate is not guaranteed. Thus, we relax the non-convex WLS problem into a convex semidefinite program by applying semidefinite relaxation (SDR). Moreover, we theoretically show that the performance can be improved by using multiple transmitters as compared to that using single transmitter, although more unknown parameters are introduced. We then extend the proposed SDR method to handle the multiple transmitters case. Finally, the mean square error analysis is provided to show that the proposed WLS method reaches the Cramer-Rao lower bound accuracy under small Gaussian noise condition. Simulation results validate the theoretical analysis and show the superior performance over the existing methods.

Complete Closed-Form and Accurate Solution to Pose Estimation From 3D Correspondences

Computing the pose from 3D data acquired in two different frames is of high importance for several robotic tasks like odometry, SLAM and place recognition. The pose is generally obtained by solving a least-squares problem given points-to-points, points-to-planes or points to lines correspondences. The non-linear least-squares problem can be solved by iterative optimization or, more efficiently, in closed-form by using solvers of polynomial systems. In this letter, a complete and accurate closed-form solution for a weighted least-squares problem is proposed. Adding weights for each correspondence allow to increase robustness to outliers. Contrary to existing methods, the proposed approach is complete since it is able to solve the problem in any non-degenerate case and it is accurate since it is guaranteed to find the global optimal estimate of the weighted least-squares problem. Simulations and experiments on real data demonstrate the superior accuracy and robustness of the proposed algorithm compared to previous approaches.

Distributed and scalable optimization for robust proton treatment planning.

The importance of robust proton treatment planning to mitigate the impact of uncertainty is well understood. However, its computational cost grows with the number of uncertainty scenarios, prolonging the treatment planning process. We developed a fast and scalable distributed optimization platform that parallelizes the robust proton treatment plan computation over the uncertaintyscenarios. We modeled the robust proton treatment planning problem as a weighted least-squares problem. To solve it, we employed an optimization technique called the alternating direction method of multipliers with Barzilai-Borwein step size (ADMM-BB). We reformulated the problem in such a way as to split the main problem into smaller subproblems, one for each proton therapy uncertainty scenario. The subproblems can be solved in parallel, allowing the computational load to be distributed across multiple processors (e.g., CPU threads/cores). We evaluated ADMM-BB on four head-and-neck proton therapy patients, each with 13 scenarios accounting for 3 mm setup and 3.5% range uncertainties. We then compared the performance of ADMM-BB with projected gradient descent (PGD) applied to the sameproblem. For each patient, ADMM-BB generated a robust proton treatment plan that satisfied all clinical criteria with comparable or better dosimetric quality than the plan generated by PGD. However, ADMM-BB's total runtime averaged about 6 to 7 times faster. This speedup increased with the number ofscenarios. ADMM-BB is a powerful distributed optimization method that leverages parallel processing platforms, such as multicore CPUs, GPUs, and cloud servers, to accelerate the computationally intensive work of robust proton treatment planning. This results in (1) a shorter treatment planning process and (2) the ability to consider more uncertainty scenarios, which improves planquality.

Semidefinite Relaxation for Source Localization by TOA in Unsynchronized Networks

This letter addresses the time-of-arrival based localization problem when the source and sensors are not synchronized, whereby an unknown transmission time is introduced. A non-convex weighted least squares (WLS) minimization problem is first formulated to jointly estimate the source position and the unknown transmission time by transforming the measurement model, and then semidefinite relaxation is applied to relax the WLS problem as a convex semidefinite program (SDP). The relaxed SDP problem is always tight and thus the optimal solution of the WLS problem can always be obtained. The proposed method is then extended to the moving source localization scenario, where the source velocity is assumed to be a constant for a sufficiently small observation period. Simulation results show that the proposed method is able to reach the Cramer-Rao lower bound accuracy when the noise is not very large.

Moving source localization using TOA and FOA measurements with imperfect synchronization

The clock bias and drift caused by imperfect time synchronization is a practical issue in localization problems. In this paper, we address the moving source localization problem using time-of-arrival and frequency-of-arrival measurements, where imperfect time synchronization is present. We transform the measurement models and formulate a nonlinear and non-convex weighted least squares (WLS) problem to jointly estimate the unknown source position, velocity, clock bias and drift. We then apply semidefinite relaxation (SDR) to the non-convex WLS problem to obtain a convex semidefinite program, which can be solved efficiently. We show through theoretical mean square error analysis that the WLS solution is able to attain the Cramer–Rao lower bound (CRLB) under small Gaussian noise condition. Simulation results show that the proposed SDR method has comparable performance with the maximum likelihood estimation method, and thus achieves the CRLB accuracy, when the noise is not too large.

Single-shot fringe-pattern analysis algorithm robust against abrupt phase jumps

Fringe-analysis methods, with fine carrier fringes, allow us to extract phase signals using a single-shot imaging measurement, enabling real-time observation of dynamic phenomena. However, abrupt phase jumps produced by the structural boundaries (or edges) in an object and/or object surfaces occasionally generate significant errors in the resulting images. In this paper, a new algorithm consisting of two steps is proposed to address this problem. In the first step, we computed an approximate phase map by ignoring abrupt phase jumps. In the second step, we solved a weighted least-squares problem to recover the abrupt phase jumps using the approximate phase map. The influence of several factors, such as carrier-fringe frequency and noise, was examined through simulation and experiment, and the effectiveness of the proposed method was demonstrated.

Gradient-Based Multi-Area Distribution System State Estimation

The increasing distributed and renewable energy resources and controllable devices in distribution systems make fast distribution system state estimation (DSSE) crucial in system monitoring and control. We consider a large multi-phase distribution system and formulate DSSE as a weighted least squares (WLS) problem. We divide the large distribution system into smaller areas of subtree structure, and by jointly exploring the linearized power flow model and the network topology, we propose a gradient-based multi-area algorithm to exactly and efficiently solve the WLS problem. The proposed algorithm enables distributed and parallel computation of the state estimation problem without compromising any performance. Numerical results on a 4,521-node test feeder show that the designed algorithm features fast convergence and accurate estimation results. Comparison with traditional Gauss-Newton method shows that the proposed method has much better performance in distribution systems with a limited amount of reliable measurement. The real-time implementation of the algorithm tracks time-varying system states with high accuracy.

Robust Geometric Model Estimation Based on Scaled Welsch q-Norm

Robust estimation, which aims to recover the geometric transformation from outlier contaminated observations, is essential for many remote sensing and photogrammetry applications. This article presents a novel robust geometric model estimation method based on scaled Welsch $q$ -norm ( $l_{q}$ -norm, $0 ). The proposed algorithm integrates a scaled Welsch weight function into the $q$ -norm framework. It, thus, inherits all the advantages of the standard $q$ -norm, i.e., fast and robust. The parameter sensitivity of the standard $q$ -norm is also largely alleviated by integrating such a weight function. These make the proposed algorithm much superior to RANSAC-type methods in real-life applications. We formulate the new cost function as an augmented Lagrangian function (ALF) and divide the ALF into two subproblems [a $q$ -norm penalized least-squares ( $l_{q}$ LS) problem and a weighted least-squares (WLS) problem] by using alternating direction method of multipliers (ADMM) method. For the WLS problem, we introduce a coarse-to-fine strategy into the iterative reweighted least-squares (IRLS) method. We change the weight function by decreasing its scale parameter. This strategy can largely avoid that the solver gets stuck in local minimums. We adapt the proposed cost into classical remote sensing tasks and develop new robust feature matching (RFM), robust exterior orientation (REO), and robust absolute orientation (RAO) algorithms. Both synthetic and real experiments demonstrate that the proposed method significantly outperforms the other compared state-of-the-art methods. Our method is still robust even if the outlier rate is up to 90%.

Source localization using TDOA and FDOA measurements based on semidefinite programming and reformulation linearization

The problem of source localization using time-difference-of-arrival (TDOA) and frequency-difference-of-arrival (FDOA) measurements has been widely studied. It is commonly formulated as a weighted least squares (WLS) problem with quadratic equality constraints. Due to the nonconvex nature of this formulation, it is difficult to produce a global solution. To tackle this issue, semidefinite programming (SDP) is utilized to convert the WLS problem to a convex optimization problem. However, the SDP-based methods will suffer obvious performance degradation when the noise level is high. In this paper, we devise a new localization solution using the SDP together with reformulation-linearization technique (RLT). Specifically, we firstly apply the RLT strategy to convert the WLS problem to a convex problem, and then add the SDP constraint to tighten the feasible region of the resultant formulation. Moreover, this solution is also extended for cases when there are sensor position and velocity errors. Numerical results show that our solution has significant accuracy advantages over the existing localization schemes at high noise levels.

Accurate solutions of weighted least squares problems associated with rank-structured matrices

In this paper, we consider how to accurately solve the weighted least squares (WLS) problem associated with a class of rank-structured matrices admitting bidiagonal representations (BRs) from weighted polynomial regression. We develop a new algorithm to solve the structured WLS problem, provided that BRs are available. A mechanism is exploited to guarantee that all the solution components are computed with a desirable high accuracy. Forward error analysis and numerical experiments are performed to confirm the high accuracy. In particular, our random examples demonstrate the high relative accuracy of each computed solution component, independently of the ill-conditioning of weighted matrices.

RSS-Based Target Localization in Underwater Acoustic Sensor Networks via Convex Relaxation.

The received signal strength (RSS) based target localization problem in underwater acoustic wireless sensor networks (UWSNs) is considered. Two cases with respect to target transmit power are considered. For the first case, under the assumption that the reference of the target transmit power is known, we derive a novel weighted least squares (WLS) estimator by using an approximation to the RSS expressions, and then transform the originally non-convex problem into a mixed semi-definite programming/second-order cone programming (SD/SOCP) problem for reaching an efficient solution. For the second case, there is no knowledge on the target transmit power, and we treat the reference power as an additional unknown parameter. In this case, we formulate a WLS estimator by using a further approximation, and present an iterative ML and mixed SD/SOCP algorithm for solving the derived WLS problem. For both cases, we also derive the closed form expressions of the Cramer–Rao Lower Bounds (CRLBs) on root mean square error (RMSE). Computer simulation results show the superior performance of the proposed methods over the existing ones in the underwater acoustic environment.

Properties of subgradient projection iteration when applying to linear imaging system

In this paper, the subgradient projection iteration is used to find an approximation solution of a weighted least-squares problem with respect to linear imaging system. Instead of an exact or approximate line search in each iteration, the step length in this paper is fixed by the weighted least-square function and the current iteration. Using weighted singular value decomposition, we estimate the bounds of step length. Consequently, we provide the decreasing property and the sufficient condition for convergence of the iterative algorithm. Furthermore, we perform a numerical experiment on a two dimensional image reconstruction problem to confirm the validity of this subgradient projection iteration.

Power System State Estimation via Feasible Point Pursuit: Algorithms and Cramér-Rao Bound

Accurately monitoring the system's operating point is central to the reliable and economic operation of an electric power grid. Power system state estimation (PSSE) aims to obtain complete voltage magnitude and angle information at each bus given a number of system variables at selected buses and lines. Power flow analysis is a special case of PSSE, and amounts to solving a set of noise-free power flow equations. Physical laws dictate quadratic relationships between available quantities and unknown voltages, rendering general instances of power flow and PSSE nonconvex and NP-hard. Past approaches are largely based on gradient-type iterative procedures or semidefinite relaxation (SDR). Due to nonconvexity, the solution obtained via gradient-type schemes depends on initialization, while SDR methods do not perform as desired in challenging scenarios. This paper puts forth novel \emph{feasible point pursuit} (FPP)-based solvers for power flow and PSSE, which iteratively seek feasible solutions for a nonconvex quadratically constrained quadratic programming (QCQP) reformulation of the weighted least-squares (WLS) problem. Relative to the prior art, the developed solvers offer superior performance at the cost of higher complexity. Furthermore, they converge to a stationary point of the WLS problem. As a baseline for comparing different estimators, the Cram{\' e}r-Rao lower bound (CRLB) is derived for the fundamental PSSE problem in this paper. Judicious numerical tests on several IEEE benchmark systems showcase markedly improved performance of our FPP-based solvers for both power flow and PSSE tasks over popular WLS-based Gauss-Newton iterations and SDR approaches.

A Christoffel function weighted least squares algorithm for collocation approximations

We propose, theoretically investigate, and numerically validate an algorithm for the Monte Carlo solution of least-squares polynomial approximation problems in a collocation frame- work. Our method is motivated by generalized Polynomial Chaos approximation in uncertainty quantification where a polynomial approximation is formed from a combination of orthogonal polynomials. A standard Monte Carlo approach would draw samples according to the density of orthogonality. Our proposed algorithm samples with respect to the equilibrium measure of the parametric domain, and subsequently solves a weighted least-squares problem, with weights given by evaluations of the Christoffel function. We present theoretical analysis to motivate the algorithm, and numerical results that show our method is superior to standard Monte Carlo methods in many situations of interest.

Solving Weighted Least Squares (WLS) problems on ARM-based architectures

The Weighted Least Squares algorithm (WLS) is applied to numerous optimization problems, but requires the use of high computational resources, especially when complex arithmetic is involved. This work aims to accelerate the resolution of a WLS problem by reducing the computational cost (relaying on BLAS/LAPACK routines) and the computational precision from double to single. As a test case, we design an IIR filter for a Graphic Equalizer, where the numerical errors due to single precision are easily visualized. In addition, given the importance of low power architectures for this kind of implementations, we evaluate the performance, scalability, and energy efficiency of each method on two different processors implementing the ARMv7 architecture, widely used in current mobile devices with power constraints. Results show that the method that exhibits a high theoretical computational cost overcomes in efficiency other methods with lower theoretical cost in architectures of this type.

A Bias-Reduced Nonlinear WLS Method for TDOA/FDOA-Based Source Localization

We address the source localization problem by using both time-difference-of-arrival (TDOA) and frequency-difference-of-arrival (FDOA) measurements. We solve this problem in two steps, and in each step, we formulate a nonlinear weighted least squares (WLS) problem followed by a bias reduction scheme. In the first step, we formulate a nonlinear WLS problem using TDOA measurements only and derive the bias of the WLS solution, which is then used to develop an unbiased WLS solution by subtracting the bias from the WLS solution. In the second step, we formulate another nonlinear WLS problem by combining the results in the first step and the FDOA measurements. To avoid the potential risk of local convergence, this WLS problem is reduced to an approximate WLS problem, for which the globally optimal solution can be obtained. The bias of the WLS solution is also derived and then subtracted from the WLS solution to reduce the bias. Simulation results show that the bias of the proposed method is reduced and that the Cramér-Rao lower bound accuracy is also achieved.

The Yule–Walker equations as a weighted least-squares problem and the association with tapering

ABSTRACTA common method for estimating the time-domain parameters of an autoregressive process is to use the Yule–Walker equations. Tapering has been shown intuitively and proven theoretically to reduce the bias of the periodogram in the frequency domain, but the intuition for the similar bias reduction in the time-domain estimates has been lacking. We provide insightful reasoning for why tapering reduces the bias in the Yule–Walker estimates by showing them to be equivalent to a weighted least-squares problem. This leads to the derivation of an optimal taper which behaves similarly to commonly used tapers.

Covariance-Assisted Matching Pursuit

We consider the problem of greedy sparse approximation in the presence of noise, given a-priori knowledge of the sparse coefficients’ covariance and mean. The proposed Covariance-Assisted Matching Pursuit (CAMP) combines the a-priori knowledge by leveraging the Gauss-Markov theorem, and provides significantly better performance than the classical Orthogonal Matching Pursuit (OMP). This improvement is achieved by solving in each matching pursuit stage a weighted least-squares problem that provides the best linear unbiased estimator of the sparse representation. The covariance and mean of the sparse representation coefficients can be estimated by a simple procedure from training data, and the advantage of the proposed approach is demonstrated for the tasks of denoising and inpainting 150,000 patches from natural images.

Designing shale-well proxy models for field development and production optimization problems

The surge to maintain a high and sustainable gas supply causes shale-gas fields to be developed with a high number of wells and an extensive infrastructure. This may result in large and complex optimization problems in order to utilize shared infrastructure, optimize the field design, and improve the recovery of the natural-gas resource. Solving large-scale, field-wide shale-gas optimization problems put different requirements on the well and reservoir models than conventional history matching, as it is crucial that the models are low ordered, numerically efficient and adaptable to changing operating conditions. To this end, using appropriately designed proxy models is critical. In this paper, we construct a dynamic shale-well proxy model based on the first principles for gas flow in dry shale-gas reservoirs, rendering a semi-physical modeling approach between purely data-driven black-box models and detailed numerical models. To tune proxy models, we propose a parameter-estimation scheme that incorporates prefiltering of prediction errors between the proxy model and reference data. The prefiltering approach provides a means for emphasizing certain ranges of the dynamics in the model, and thus constitutes a structured approach for defining weights in the accompanying weighted least-squares problem. Further, we show how the proposed approach allows for considering the model tuning and spatial grid design as a joint problem with respect to the purpose of the proxy model.

A Novel Method for Fault Location of Transmission Lines by Wide-Area Voltage Measurements Considering Measurement Errors

Smart transmission grids are intended to utilize various measurements for enhanced operation. In particular, synchronized phasor measurements have found many applications in this context. This paper presents a linear weighted least-squares (WLS) method for fault location estimation of transmission lines by synchronized voltage measurements. The measurements may be taken from the faulted line terminals or buses far from the faulted line. The circuit equations of the network are used to find the transfer function between the fault location and each voltage measurement. Next, two auxiliary variables are defined to transform the nonlinear fault location estimation problem into a linear WLS problem. A closed-form solution is then obtained for fault location. The use of multiple measurements provides the opportunity of identifying erroneous measurements by a statistical test. The linearity of the problem expedites both estimation and identification procedures. Moreover, the method requires neither to classify the fault type nor to estimate the fault resistance. Furthermore, transmission lines are modeled by distributed parameters so that the method can be applicable to long lines. Electromagnetic transient simulations for 9-bus and 22-bus test systems reveal accurate fault location estimation, regardless of fault type and resistance, as well as successful identification of measurement errors.