Iteratively Reweighted Least Squares Method Research Articles

Solving Maxmin Optimization Problems via Population Games

Population games are games with a finite set of available strategies and an infinite number of players, in which the reward for choosing a given strategy is a function of the distribution of players over strategies. The paper shows that, in a certain class of maxmin optimization problems, it is possible to associate a population game to a given maxmin problem in such a way that solutions to the optimization problem are found from Nash equilibria of the associated game. Iterative solution methods for maxmin optimization problems can then be derived from systems of differential equations whose trajectories are known to converge to Nash equilibria. In particular, we use a discrete-time version of the celebrated replicator equation of evolutionary game theory, also known in machine learning as the exponential multiplicative weights algorithm. The resulting algorithm can be viewed as a generalization of the Iteratively Reweighted Least Squares (IRLS) method, which is well known in numerical analysis as a useful technique for solving Chebyshev function approximation problems on a finite grid. Examples are provided to show the use of the generalized IRLS method in collective investment and in decision making under model uncertainty.

A robust lp‐norm localization of moving targets in distributed multiple‐input multiple‐output radar with measurement outliers

AbstractThe Gaussian noise model and estimators based on least squares (LS) are widely used in target localisation with distributed multiple‐input multiple‐output (MIMO) radar because of their computational efficiency. However, the accuracy of existing LS‐based target localisation algorithms deteriorates sharply in the presence of outliers in the measurements. Thus, a robust solution is developed based on the ‐norm minimisation criterion and iteratively reweighted least squares (IRLS) for locating a moving target with impulse noise using the angle of arrival (AOA), time delay (TD), and Doppler shift (DS) measurements. First, the AOA, TD, and DS measurement noise models are developed based on the α‐stable distribution. Then, the localisation problem is transformed into an ‐norm minimisation problem by linearising the AOA, TD, and DS measurement equations. Finally, the ‐norm minimisation problem is solved using an IRLS method to obtain the target position and estimate the velocity. Moreover, the optimum of the norm order (p) and the Cramér–Rao lower bound for the target position and velocity estimation are derived under α‐stable distributed measurement noise. The simulation results demonstrate that the developed algorithm offers higher accurascy and robustness than the existing ones in the presence of measurement outliers.

EHP VALUE OF MINI LNG SHIP WITH FORM FACTOR FROM PROHASKA AND IRLS METHOD USING SHIP RESISTANCE TESTING DATA

The ship model test was believed to be one of the effective methods for figuring out the boundaries and reliability of the ship's horsepower. The ship's form factor determines a full-scale ship's effective horsepower. Determination of the form factor value can be done experimentally through the Prohaska method. The new method proposed in this study is employed the regression Iteratively Reweighted Least Squares (IRLS) method by utilizing the principle dimension of the ship, such as LWL, B, CB, CP, CM, WSA, T, ∆. etc. The Indonesian Hydrodynamics Laboratory has a database of ships with various principle dimensions which have undergone the towing model test. Through the database, the form factor can be predicted with the IRLS method. The method is then verified and validated with the Prohaska method. The result shows a good agreement with the Prohaska method. The obtained results from the IRLS method also show that the EHP & Resistance calculations are identical with old fashion Prohaska methods. The residual bias factor established by the IRLS method was verified in comparison to the value of the form factor generated by the Prohaska method. Comparison between the two methods results in a small error.

Improved iteratively reweighted least squares algorithms for sparse recovery problem

In this paper, some new algorithms based on the iteratively reweighted least squares (IRLS) method are proposed for sparse recovery problem. There are two important parameters in the IRLS method: a weighted parameter and a regularization parameter. On the one hand, in order to improve the performance of IRLS method, a new way is given to update the weight vector. On the other hand, for the regularization parameter, three new update methods are introduced to avoid the phenomenon that the regularization parameter drops too fast. Then, some improved iteratively reweighted least squares (IIRLS) algorithms are proposed, and their convergence and convergence rate are analyzed. The local convergence of our algorithms is superlinear and approaches a quadratic rate in special cases. Finally, a large number of algorithms are compared in solving the sparse recovery problem, including IRLS methods and IIRLS methods with different weighting parameters and regularization parameters, and certain iterative methods. The experimental results demonstrate that the proposed methods are efficient and promising.

Dynamic parameter identification and nonlinear friction compensation method for safety perception of heavy explosion-proof robots

This article focuses on the problem of how to accurately calculate the joint control torques when the explosion-proof robot performs collision detection without sensors and gives a complete solution. Nonlinear joint frictions are incorporated into the dynamic model of a robotic manipulator to improve calculation accuracy. A genetic algorithm is used to optimise the excitation trajectories to fully stimulate the robot dynamic characteristics. Effective and applicable data filtering and smoothing methods are proposed and the Iteratively Reweighted Least-Squares method based on the error term is applied to identify the robot dynamic parameters. Compared with Ordinary Least-Squares method, the proposed algorithm improves the accuracy of joint control torques estimation.

Robust Estimation of Deformation from Observation Differences Using Some Evolutionary Optimisation Algorithms.

In this paper, an original modification of the generalised robust estimation of deformation from observation differences (GREDOD) method is presented with the application of two evolutionary optimisation algorithms, the genetic algorithm (GA) and generalised particle swarm optimisation (GPSO), in the procedure of robust estimation of the displacement vector. The iterative reweighted least-squares (IRLS) method is traditionally used to perform robust estimation of the displacement vector, i.e., to determine the optimal datum solution of the displacement vector. In order to overcome the main flaw of the IRLS method, namely, the inability to determine the global optimal datum solution of the displacement vector if displaced points appear in the set of datum network points, the application of the GA and GPSO algorithms, which are powerful global optimisation techniques, is proposed for the robust estimation of the displacement vector. A thorough and comprehensive experimental analysis of the proposed modification of the GREDOD method was conducted based on Monte Carlo simulations with the application of the mean success rate (MSR). A comparative analysis of the traditional approach using IRLS, the proposed modification based on the GA and GPSO algorithms and one recent modification of the iterative weighted similarity transformation (IWST) method based on evolutionary optimisation techniques is also presented. The obtained results confirmed the quality and practical usefulness of the presented modification of the GREDOD method, since it increased the overall efficiency by about 18% and can provide more reliable results for projects dealing with the deformation analysis of engineering facilities and parts of the Earth’s crust surface.

A Novel Algorithm Based on Compressive Sensing to Mitigate Phase Unwrapping Errors in Multitemporal DInSAR Approaches

In this work, we present a new method based on the compressive sensing (CS) theory to correct phase unwrapping (PhU) errors in the multitemporal sequence of interferograms exploited by advanced differential interferometric synthetic aperture radar (DInSAR) techniques to generate deformation time series. The developed algorithm estimates the PhU errors by using a modified L₁-norm estimator applied to the interferometric network built in the temporal/spatial baseline plane. Indeed, in order to search the minimum L₁-norm sparse solution, we apply the iterative reweighted least-squares method with an improved weight function that takes account of the baseline characteristics of the interferometric pairs. Moreover, we also introduce a quality function to identify those solutions that have no physical meaning. Although the proposed approach can be applied to different multitemporal DInSAR approaches, our analysis is tailored to the full-resolution small baseline subset (SBAS) processing chain that we properly modify to implement the proposed CS-based algorithm. To assess the performance of the developed technique, we carry out an extended experimental analysis based on simulated and real SAR data. In particular, we process two wide SAR datasets acquired by Sentinel-1 and COSMO-SkyMed constellations over central Italy between 2011 and 2019. The achieved experimental results clearly demonstrate the effectiveness of the developed approach in retrieving PhU errors and generating displacement time series related to strongly nonlinear deformation phenomena. Indeed, the developed CS-based technique significantly increases the number of detected coherent points and improves the accuracy of the retrieved deformation time series.

Increasing efficiency of the robust deformation analysis methods using genetic algorithm and generalised particle swarm optimisation

The paper analyses the possibility of increasing efficiency of the Iterative Weighted Similarity Transformation (IWST) method, which is a prototype of classic robust methods, using global optimisation approach instead of classical one, available in the literature. For the purpose of solving the optimisation problem of the IWST method, in addition to the Iterative Reweighted Least Squares (IRLS) method, the Genetic algorithm (GA) and Generalised Particle Swarm Optimisation (GPSO) algorithm were applied, in order to overcome some flaws of IRLS method. Experimental research was performed based on the Monte Carlo simulation using the mean success rate (MSR) on the example of the geodetic control network for monitoring the Šelevrenac dam in the Republic of Serbia. By using the GA and GPSO algorithms, the overall efficiency of the IWST method has been increased by about 18% compared to the IRLS method. Also, it has been determined that the efficiency of the IRLS method significantly reduces with the increase in the number of displaced potential reference points (PRPs), while the GA and GPSO algorithms’ efficiency does not change significantly. The values of overall absolute true errors due to the increased number of displaced PRPs in the GA and GPSO algorithms did not change notably while with the IRLS method their values increased significantly.

Total Variation Regularization Based on Iteratively Reweighted Least-Squares Method for Electrical Resistance Tomography

The regularization method has been utilized to address the ill-posed inverse problem of electrical resistance tomography (ERT). As one of the regularization methods, the total variation (TV) regularization method has the advantage of edge-preserving. A majority of research has been focused on the study of the penalty term in the TV method. In this article, a novel TV regularization based on iteratively reweighted least squares (TV-IRLS) method is proposed, in which, $L_{1}$ -norm is applied to the fidelity term, and a weighting matrix is applied to convert the $L_{1}$ -norm to $L_{2}$ -norm when calculating the objective function, which reduces the difficulty of solving the $L_{1}$ -norm. An improved generalized cross-validation method is adopted to determine the optimal regularization parameter (RP) automatically, and an automatic method is presented for the selection of the threshold value in the weighting matrix to guarantee the accuracy and speed of the solution. Meanwhile, a scaling factor is introduced to constrain the solution and ensure the stability of the solution. For verifying the performance of the proposed TV-IRLS regularization method, both numerical simulation and static experiments are carried out in the image reconstruction of ERT through qualitative and quantitative analysis. The results demonstrate that the proposed TV-IRLS regularization method performs well in improving the quality and resolution of the reconstructed images in ERT. Besides, it is more robust to data noises compared with the Tikhonov regularization method and TV regularization method.

Sparse Solutions in Optimal Control of PDEs with Uncertain Parameters: The Linear Case

We study sparse solutions of optimal control problems governed by PDEs with uncertain coefficients. We propose two formulations, one where the solution is a deterministic control optimizing the mean objective, and a formulation aiming at stochastic controls that share the same sparsity structure. In both formulations, regions where the controls do not vanish can be interpreted as optimal locations for placing control devices. In this paper, we focus on linear PDEs with linearly entering uncertain parameters. Under these assumptions, the deterministic formulation reduces to a problem with known structure, and thus we mainly focus on the stochastic control formulation. Here, shared sparsity is achieved by incorporating the $L^1$-norm of the mean of the pointwise squared controls in the objective. We reformulate the problem using a norm reweighting function that is defined over physical space only and thus helps to avoid approximation of the random space using samples or quadrature. We show that a fixed point algorithm applied to the norm reweighting formulation leads to a variant of the well-studied iterative reweighted least squares (IRLS) algorithm, and we propose a novel preconditioned Newton-conjugate gradient method to speed up the IRLS algorithm. We combine our algorithms with low-rank operator approximations, for which we provide estimates of the truncation error. We carefully examine the computational complexity of the resulting algorithms. The sparsity structure of the optimal controls and the performance of the solution algorithms are studied numerically using control problems governed by the Laplace and Helmholtz equations. In these experiments the Newton variant clearly outperforms the IRLS method.

Amplitude-versus-angle inversion based on theL1-norm-based likelihood function and the total variation regularization constraint

In prestack seismic data, outlier errors occur and can negatively influence the outcome of the amplitude-versus-angle (AVA) inversion process. Hence, their effect needs to be minimized during AVA inversion. AVA inversion based on the [Formula: see text]-norm-based likelihood function is highly sensitive to outlier errors. In comparison, AVA inversion based on the [Formula: see text]-norm-based likelihood function is less affected by outlier errors, and for this reason we have used it with the total variation regularization method used as a constraint to invert discontinuities from geologic bodies. To ensure that the inversion results contain low-frequency components, prior information constraints from model parameters are added to the inverse objective function, which is then solved by the iterative reweighted least-squares method. Results of numerical tests and real-data examples from the application of this method indicate that the algorithm is strongly robust against noise, especially abnormal outlier errors, and that the results of the inversion are reasonable.

Conjugate gradient acceleration of iteratively re-weighted least squares methods

Iteratively re-weighted least squares (IRLS) is a method for solving minimization problems involving non-quadratic cost functions, perhaps non-convex and non-smooth, which however can be described as the infimum over a family of quadratic functions. This transformation suggests an algorithmic scheme that solves a sequence of quadratic problems to be tackled efficiently by tools of numerical linear algebra. Its general scope and its usually simple implementation, transforming the initial non-convex and non-smooth minimization problem into a more familiar and easily solvable quadratic optimization problem, make it a versatile algorithm. However, despite its simplicity, versatility, and elegant analysis, the complexity of IRLS strongly depends on the way the solution of the successive quadratic optimizations is addressed. For the important special case of compressed sensing and sparse recovery problems in signal processing, we investigate theoretically and numerically how accurately one needs to solve the quadratic problems by means of the conjugate gradient (CG) method in each iteration in order to guarantee convergence. The use of the CG method may significantly speed-up the numerical solution of the quadratic subproblems, in particular, when fast matrix-vector multiplication (exploiting for instance the FFT) is available for the matrix involved. In addition, we study convergence rates. Our modified IRLS method outperforms state of the art first order methods such as Iterative Hard Thresholding (IHT) or Fast Iterative Soft-Thresholding Algorithm (FISTA) in many situations, especially in large dimensions. Moreover, IRLS is often able to recover sparse vectors from fewer measurements than required for IHT and FISTA.

M-estimator-based robust Kalman filter for systems with process modeling errors and rank deficient measurement models

The aim of this paper was to develop a robust version of the Kalman filter (KF) to address process modeling errors in linear system with rank deficient measurement models. An important infinitesimal robustness metric, called the influence function, which is widely used in location, scale, and regression models, is introduced into KF. After recasting KF’s update as an artificial linear regression, the influence function of KF is derived in detail. The lack of robustness of KF is clearly demonstrated by the nonboundedness of its influence function. Huber estimator, the most important member of the robust M-estimator family, is introduced into the recast linear regression. The iteration and initialization to iteratively solve the M-estimator are emphasized to address the specific case with process modeling errors and rank deficient measurement models. The iterative reweighted least-squares method together with a covariance construction method is preferred to the standard and simplified Newton’s method and the asymptotic covariance. A new initial value constructed through correcting the a priori estimate to accord with the actual measurement exactly is proposed using the Moore–Penrose pseudoinverse method. A simple but illustrative example is simulated to check the feasibility of the proposed method.

On the Convergence of Alternating Minimization for Convex Programming with Applications to Iteratively Reweighted Least Squares and Decomposition Schemes

This paper is concerned with the alternating minimization (AM) method for solving convex minimization problems where the decision variables vector is split into two blocks. The objective function is a sum of a differentiable convex function and a separable (possibly) nonsmooth extended real-valued convex function, and consequently constraints can be incorporated. We analyze the convergence rate of the method and establish a nonasymptotic sublinear rate of convergence where the multiplicative constant depends on the minimal block Lipschitz constant. We then analyze the iteratively reweighted least squares (IRLS) method for solving convex problems involving sums of norms. Based on the results derived for the AM method, we establish a nonasymptotic sublinear rate of convergence of the IRLS method. In addition, we show an asymptotic rate of convergence whose efficiency estimate does not depend on the data of the problem. Finally, we study the convergence properties of a decomposition-based approach designed t...

Robust inversion using biweight norm and its application to seismic inversion

In spite of some minor drawbacks such as nonuniqueness and higher computational cost, finding the least-absolute (l1 norm) error solution to solve an optimisation problem is mostly known to give a better answer than the classical least-squares (l2 norm) method. This is because the robust property of the median value is affected little by outlier values and the solution of the least l1 norm error corresponds to the solution of minimum median error. Several variants of the l1 norm such as the Huber norm and the Hybrid norm have the same robust properties as the l1 norm. The optimisation methods based on l1 norm obtain their robustness by reducing the influence of outliers significantly, although never ignoring it. Therefore, if the proportion of outliers increases, most of the methods based on l1 norm may begin to be affected by the outliers. In such a case, other types of robust measures such as Tukey’s Biweight (Bisquare weight) norm, which excludes outliers in computing the misfit measure, could perform better. This paper describes the application of the Biweight norm using the IRLS (iteratively reweighted least-squares) method as a robust inversion and shows its possible improvement in robustness when dealing with data having many outliers.

Regularization Parameter Selection for Nonlinear Iterative Image Restoration and MRI Reconstruction Using GCV and SURE-Based Methods

Regularized iterative reconstruction algorithms for imaging inverse problems require selection of appropriate regularization parameter values. We focus on the challenging problem of tuning regularization parameters for nonlinear algorithms for the case of additive (possibly complex) Gaussian noise. Generalized cross-validation (GCV) and (weighted) mean-squared error (MSE) approaches (based on Steinfs Unbiased Risk Estimate. SURE) need the Jacobian matrix of the nonlinear reconstruction operator (representative of the iterative algorithm) with respect to the data. We derive the desired Jacobian matrix for two types of nonlinear iterative algorithms: a fast variant of the standard iterative reweighted least-squares method and the contemporary split-Bregman algorithm, both of which can accommodate a wide variety of analysis- and synthesis-type regularizers. The proposed approach iteratively computes two weighted SURE-type measures: Predicted-SURE and Projected-SURE (that require knowledge of noise variance Ð2), and GCV (that does not need Ð2) for these algorithms. We apply the methods to image restoration and to magnetic resonance image (MRI) reconstruction using total variation (TV) and an analysis-type .1-regularization. We demonstrate through simulations and experiments with real data that minimizing Predicted-SURE and Projected-SURE consistently lead to near-MSE-optimal reconstructions. We also observed that minimizing GCV yields reconstruction results that are near-MSE-optimal for image restoration and slightly suboptimal for MRI. Theoretical derivations in this work related to Jacobian matrix evaluations can be extended, in principle, to other types of regularizers and reconstruction algorithms.

On the Iteratively Reweighted Rank Regression Estimator

The finite sample performance of the rank estimator of regression coefficients obtained using the iteratively reweighted least squares (IRLS) of Sievers and Abebe (2004) is evaluated. Efficiency comparisons show that the IRLS method does quite well in comparison to least squares or the traditional rank estimates in cases of moderate-tailed error distributions; however, the IRLS method does not appear to be suitable for heavy-tailed data. Moreover, our results show that the IRLS estimator will have an unbounded influence function even if we use an initial estimator with a bounded influence function.

Design of IIR digital all-pass filters using least pth phase error criterion

In this brief, the least pth-phase error criterion is adopted to design digital infinite-impulse-response all-pass filters to have an arbitrarily prescribed phase response. First, an iterative reweighted least-squares method is used to design an unconstrained all-pass filter whose optimal filter coefficients are obtained by solving a set of linear simultaneous equations in each iteration. Then, the proposed method is extended to design a constrained all-pass filter. Finally, a design example of the Hilbert transformer is demonstrated to illustrate the effectiveness of the new design method.

A Globally Convergent Method for $l_p $ Problems

The $l_p $-norm discrete estimation problem $\min_{x \in \Re^n } ||b - A^T x||_p^p $ is troublesome when p is close to unity because the objective function approaches a nonsmooth form as p converges to one. This paper presents an efficient algorithm for solving $l_p $-norm problems for all $1 \leq p < 2$. When $p = 1$ it is essentially the method presented by T. F. Coleman and Y. Li [Math. Programming, 56 (1992), pp. 189–222], which is a globally and quadratically convergent algorithm under some nondegeneracy assumptions. The existing iteratively reweighted least-squares (IRLS) method can be obtained from the new approach by updating some dual multipliers in a special fashion. The new method is globally convergent, and it is superlinearly convergent when there is no zero residual at the solution. At each iteration the main computational cost of the new method is the same as that of the IRLS method: solving a reweighted least-squares problem. Numerical experiments indicate that this method is significantly faster than popular iteratively reweighted least-squares methods when p is close or equal to one.

Robust iterative inversion for the one‐dimensional acoustic wave equation

Seismic inversion can be formulated by considering a linearized integral relation which is deduced from the wave equation. This Born inversion approach is equivalent to linear least‐squares inversion for a particular parameterization of the medium. The least‐squares solution is a member of a family of generalized LP norm solutions which are deduced from a maximum‐likelihood formulation. This formulation allows design of various statistical inversion solutions. We present two iterative solutions to the one‐dimensional (1-D) seismic inverse problem: the iterative least‐squares (ILS) and the iterative reweighted least‐squares (IRLS) methods. The ILS method involves solving a distorted background velocity problem after the initial least‐squares solution is obtained. The IRLS method is used as a robust regression technique which is better suited for dealing with certain types of noise and is computationally faster than ILS. Several numerical examples demonstrate that the IRLS method accurately estimates impedance profiles despite the presence of large‐amplitude noise spikes in the seismic traces. Numerical experiments suggest that the IRLS inversion can also be insensitive to noise bursts which are of a lower frequency band than noise spikes.