Iterative Weighted Least Squares Research Articles

3D TDOA Emitter Localization Using Conic Approximation.

This paper develops a new time difference of arrival (TDOA) emitter localization algorithm in the 3D space, employing conic approximations of hyperboloids associated with TDOA measurements. TDOA measurements are first converted to 1D angle of arrival (1D-AOA) measurements that define TDOA cones centred about axes connecting the corresponding TDOA sensor pairs. Then, the emitter location is calculated from the triangulation of 1D-AOAs, which is formulated as a system of nonlinear equations and solved by a low-complexity two-stage estimation algorithm composed of an iterative weighted least squares (IWLS) estimator and a Taylor series estimator aimed at refining the IWLS estimate. Important conclusions are reached about the optimality of sensor-emitter and sensor array geometries. The approximate efficiency of the IWLS estimator is also established under mild conditions. The new two-stage estimator is shown to be capable of outperforming the maximum likelihood estimator while performing very close to the Cramer Rao lower bound in poor sensor-emitter geometries and large noise by way of numerical simulations.

A simple iterative algorithm based on weighted least-squares for errors-in-variables models: Examples of coordinate transformations

Abstract Although weighted total least-squares (WTLS) adjustment within the errors-in-variables (EIV) model is a rigorous method developed for parameter estimation, its exact solution is complicated since the matrix operations are extremely time-consuming in the whole repeated iteration process, especially when dealing with large data sets. This paper rewrites the EIV model to a similar Gauss–Markov model by taking the random error of the design matrix and observations into account, and reformulates it as an iterative weighted least-squares (IWLS) method without complicated theoretical derivation. IWLS approximates the “exact solution” of the general WTLS and provides a good balance between computational efficiency and estimation accuracy. Because weighted LS (WLS) method has a natural advantage in solving the EIV model, we also investigate whether WLS can directly replace IWLS and WTLS to implement the EIV model when the parameters in the EIV model are small. The results of numerical experiments confirmed that IWLS can obtain almost the same solution as the general WTLS solution of Jazaeri [21] and WLS can achieve the same accuracy as the general WTLS when the parameters are small.

Fast Cross-validation for Multi-penalty High-dimensional Ridge Regression

High-dimensional prediction with multiple data types needs to account for potentially strong differences in predictive signal. Ridge regression is a simple model for high-dimensional data that has challenged the predictive performance of many more complex models and learners, and that allows inclusion of data type-specific penalties. The largest challenge for multi-penalty ridge is to optimize these penalties efficiently in a cross-validation (CV) setting, in particular for GLM and Cox ridge regression, which require an additional estimation loop by iterative weighted least squares (IWLS). Our main contribution is a computationally very efficient formula for the multi-penalty, sample-weighted hat-matrix, as used in the IWLS algorithm. As a result, nearly all computations are in low-dimensional space, rendering a speed-up of several orders of magnitude. We developed a flexible framework that facilitates multiple types of response, unpenalized covariates, several performance criteria and repeated CV. Extensions to paired and preferential data types are included and illustrated on several cancer genomics survival prediction problems. Moreover, we present similar computational shortcuts for maximum marginal likelihood and Bayesian probit regression. The corresponding R-package, multiridge, serves as a versatile standalone tool, but also as a fast benchmark for other more complex models and multi-view learners. Supplementary materials for this article are available online.

Hybird Nonlinear Control for Fighter With Center of Gravity Perturbation and Aerodynamic Parameter Uncertainty

In this paper, a hybrid nonlinear control scheme combining nonlinear dynamic inversion (NDI) and adaptive sliding mode (ASM) control is proposed for high angle of attack fighter system with center of gravity perturbation and aerodynamic parameter uncertainty. By introducing the affine nonlinear model of inner loop angular velocity with center of gravity perturbation, an adaptive sliding mode control scheme based on the online estimation of radial basis function (RBF) neural network is designed, which can reduce the dynamic inversion error of NDI control and rapidly compensate the multi-coupled channel oscillation. The aerodynamic force and moment coefficients are estimated via the iterative weighted least squares (IRLS) method. Then an outer loop integral sliding mode (ISM) NDI controller is designed, which can be robust to disturbances and mismatched uncertainties through adjusting adaptive gain, and the rudder yaw chattering caused by the uncertainty of aerodynamic parameters is reduced simultaneously. The Lyapunov stability theory and Barbarat’s lemma prove the stability of the designed control scheme, and simulation based on F-16 model verifies the effectiveness of the control scheme.

Suppression of Coherence Matrix Bias for Phase Linking and Ambiguity Detection in MTInSAR

Phase decorrelation, as one of the main error sources, limits the capability of interferometric synthetic aperture radar (InSAR) for deformation mapping over areas with low coherence. Although several methods have been realized to reduce decorrelation noise, for example, by phase linking and spatial and temporal filters, their performances deteriorate when coherence estimation bias exists. We present an arc-based approach that allows reconstructing unwrapped interval phase time-series based on iterative weighted least squares (WLS) in temporal and spatial domains. The main features of the method are that phase optimization and unwrapping can be jointly conducted by spatial and temporal iterative WLS and coherence matrix bias has negligible effects on the estimation. In addition, the linear formation makes the implementation suitable with small subset of interferograms, providing an efficient solution for future big SAR data. We demonstrate the effectiveness of the proposed method using simulated and real data with different decorrelation mechanisms and compare our approach with the state-of-art phase reconstruction methods. Substantial improvement can be achieved in terms of reduced root-mean-square error (RMSE) in the simulation data and increased density of coherent measurements in the real data.

Negative binomial mixed models for analyzing microbiome count data

BackgroundRecent advances in next-generation sequencing (NGS) technology enable researchers to collect a large volume of metagenomic sequencing data. These data provide valuable resources for investigating interactions between the microbiome and host environmental/clinical factors. In addition to the well-known properties of microbiome count measurements, for example, varied total sequence reads across samples, over-dispersion and zero-inflation, microbiome studies usually collect samples with hierarchical structures, which introduce correlation among the samples and thus further complicate the analysis and interpretation of microbiome count data.ResultsIn this article, we propose negative binomial mixed models (NBMMs) for detecting the association between the microbiome and host environmental/clinical factors for correlated microbiome count data. Although having not dealt with zero-inflation, the proposed mixed-effects models account for correlation among the samples by incorporating random effects into the commonly used fixed-effects negative binomial model, and can efficiently handle over-dispersion and varying total reads. We have developed a flexible and efficient IWLS (Iterative Weighted Least Squares) algorithm to fit the proposed NBMMs by taking advantage of the standard procedure for fitting the linear mixed models.ConclusionsWe evaluate and demonstrate the proposed method via extensive simulation studies and the application to mouse gut microbiome data. The results show that the proposed method has desirable properties and outperform the previously used methods in terms of both empirical power and Type I error. The method has been incorporated into the freely available R package BhGLM (http://www.ssg.uab.edu/bhglm/ and http://github.com/abbyyan3/BhGLM), providing a useful tool for analyzing microbiome data.

A Modified Iterative Weighted Least Squares Method

The Iterative Weighted Least Squares (IWLS) method is one of the estimation procedures in logistic regression modeling. In consideration of the strategic role played by this model, especially in biometrics, the need to source for alternative logistic regression estimators has continued to resurface in the literature. In this paper, a modified IWLS method is developed by exponentiating the response probability. As a consequence, both the weight function and the adjusted dependent variate are modified. The resulting estimator is compared with the existing IWLS estimator using variances of parameter estimates and confidence intervals of the estimates. Four subpopulations were used in three illustrative examples with gender, Fasting Blood Sugar and Body Mass Index as explanatory variables. It is shown in this paper that the new update is superior to the traditional IWLS update in terms of variance reduction of parameter estimates and for the fact that it provides a more strict confidence interval for the test of hypotheses. In the first example, the confidence intervals for the parameter estimates of the existing IWLS scheme are (0.2183, 1.7197), (-1.9993, -0.1687), (1.7993, 0.0313), while those of the proposed method are (-2.6588, -2.2240), (1.1228, 1.6826), (0.8552, 1.4150). The proposed estimator allows for improved goodness-of-fit. By a careful formulation of the model, the proposed estimator is made to behave like a survival function in the sense that it can be used to model the probability of extra survival time in survival analysis.

Minimax Design of Low-Complexity Allpass Variable Fractional-Delay Digital Filters

This paper proposes noniterative and iterative linear programming (LP) procedures for designing low-complexity allpass variable fractional-delay (VFD) digital filters in the minimax sense. Expressing each coefficient of an allpass VFD filter as a polynomial in the VFD parameter p, we show that the frequency response error of an allpass VFD filter can be written as a pure imaginary part divided by its denominator. Thus, the minimax design can be approximately formulated as an LP problem through neglecting the denominator, which leads to a noniterative minimax design. To improve the minimax design accuracy, we propose an iterative LP procedure that utilizes the denominator from the preceding iteration as a known. The iterative LP minimization is repeated until it converges to the minimax solution. Moreover, we also present a two-stage algorithm for optimizing the optimal variable range p ∈ [pMin, pMax] of the VFD parameter p and successively reducing the subfilter orders. Design examples are given to show that both noniterative and iterative LP methods can achieve much better minimax designs (smaller peak errors) than the existing iterative weighted-least-squares (WLS) approaches, which aim to minimize the peak errors of VFD response and variable phase response, respectively. Moreover, the resulting allpass VFD filters have lower complexities than those from the iterative WLS approaches.

Gauss–Seidel Estimation of Generalized Linear Mixed Models With Application to Poisson Modeling of Spatially Varying Disease Rates

Generalized linear mixed models (GLMMs) are often fit by computational procedures such as penalized quasi-likelihood (PQL). Special cases of GLMMs are generalized linear models (GLMs), which are often fit using algorithms like iterative weighted least squares (IWLS). High computational costs and memory space constraints make it difficult to apply these iterative procedures to datasets having a very large number of records.We propose a computationally efficient strategy based on the Gauss–Seidel algorithm that iteratively fits submodels of the GLMM to collapsed versions of the data. The strategy is applied to investigate the relationship between ischemic heart disease, socioeconomic status, and age/gender category in New South Wales, Australia, based on outcome data consisting of approximately 33 million records. For Poisson and binomial regression models, the Gauss–Seidel approach is found to substantially outperform existing methods in terms of maximum analyzable sample size. Remarkably, for both models, the average time per iteration and the total time until convergence of the Gauss–Seidel procedure are less than 0.3% of the corresponding times for the IWLS algorithm. Platform-independent pseudo-code for fitting GLMS, as well as the source code used to generate and analyze the datasets in the simulation studies, are available online as supplemental materials.

Incremental Value Relevance of Unrecognized Deferred Taxes: Evidence from the United Kingdom

ABSTRACT: We examine empirically whether the use of the partial method for deferred taxes provides incremental information of use to investors. Specifically, we test whether U.K. capital markets valued unrecognized deferred tax amounts reported in the footnotes to U.K. annual reports, pursuant to U.K. Statement of Standard Accounting Practice (SSAP) No. 15 (ASB 1985). Our empirical model is based on Feltham and Ohlson (1995). We run iterative weighted least-squares (IWLS) regression of year-end share prices on a decomposition of book value per share for a pooled sample of U.K. firm-years drawn from the years 1993 through 1998, and find positive associations with price for net deferred tax assets—both recognized and unrecognized. Moreover, we are unable to reject the null hypothesis that both parts of deferred taxes have similar multiples in our price regressions. These findings support some theoretical predictions in Sansing (1998), Guenther and Sansing (2000, 2004), and Amir et al. (2001).

COMPARING MULTIPLE‐GROUP MULTINOMIAL LOG‐LINEAR MODELS FOR MULTIDIMENSIONAL SKILL DISTRIBUTIONS IN THE GENERAL DIAGNOSTIC MODEL

ABSTRACTThe general diagnostic model (GDM) utilizes located latent classes for modeling a multidimensional proficiency variable. In this paper, the GDM is extended by employing a log‐linear model for multiple populations that assumes constraints on parameters across multiple groups. This constrained model is compared to log‐linear models that assume separate sets of parameters to fit the distribution of latent variables in each group of a multiple‐group model. Estimation of these constrained log‐linear models using iterative weighted least squares (IWLS) methods is outlined and an application to NAEP data exemplifies the differences between constrained and unconstrained models in the presence of larger numbers of group‐specific proficiency distributions.The use of log‐linear models for the latent skill space distributions using constraints across populations allows for efficient computations in models that include many proficiency distributions.

Gauge capability studies for attribute data

AbstractA gauge repeatability and reproducibility (R&R) method is developed to assess the capability of a measurement system. Although gauge capability studies have received substantial attention, few studies have investigated attribute data despite their wide application in industry. The primary aim of this research is to develop a procedure, based on the generalized linear model, to evaluate the R&R of a measurement system for attribute data. To calculate repeatability of a system, the procedure integrates the iterative weighted least squares (IWLS) method and deviance analysis. The proposed procedure is applied to an inclusion measurement system to verify its adequacy to model the process capability. Copyright © 2007 John Wiley & Sons, Ltd.

Sequential weighted least squares algorithm for PET image reconstruction

The weighted least squares (WLS) algorithm has proven useful for modern positron emission tomography (PET) scanners to approach reconstructions with non-Poisson precorrected measurement data. In this paper, we propose a new time recursive sequential WLS algorithm whose derivation uses the time-varying property of data acquisition of PET scanning. It ties close relationship with the time-varying Kalman filtering and can be extended appropriately to an iteration fashion as the absence of proper a priori initializations. The performance of sequential WLS is evaluated experimentally. The results show its fast convergence over both the multiplicative and coordinate-based iterative WLS methods. It also produces relative uniform estimate variances that makes it more suitable for routine applications.

Incremental Value Relevance of Unrecognized Deferred Taxes: Evidence from the United Kingdom

We examine empirically whether the use of the partial method for deferred taxes provides incremental information of use to investors. Specifically, we test whether U.K. capital markets valued unrecognized deferred tax amounts reported in the footnotes to U.K. annual reports, pursuant to U.K. Statement of Standard Accounting Practice (SSAP) 15. Our empirical model is based on Feltham and Ohlson (1995). We run iterative weighted least squares (IWLS) regression of year-end share prices on a decomposition of book value per share for a pooled sample of U.K. firm-years drawn from the years 1993 through 1998, and find positive associations with price for net deferred tax assets - both recognized and unrecognized. Moreover, we are unable to reject the null hypothesis that both parts of deferred taxes have similar multiples in our price regressions. These findings support some theoretical predictions in Sansing (1998), Guenther and Sansing (2000, 2004), and Amir et al. (2001).

Noniterative WLS design of allpass variable fractional-delay digital filters

This paper presents a noniterative weighted-least-squares (WLS) method for designing allpass variable fractional-delay (VFD) digital filters. After expressing each coefficient of an allpass VFD filter as a polynomial of the VFD parameter p, we develop a noniterative technique for finding the optimal polynomial coefficients, and show that the allpass VFD filter design problem can be efficiently solved without using any iterative procedure while a closed-form solution can be easily obtained through solving a matrix equation. Compared with the existing iterative WLS method that solves a series of approximately linearized WLS minimization problems, the proposed noniterative one can yield much better design results with significantly reduced computational complexity. Moreover, the new WLS method does not involve any convergence issue.

Estimating the parameters of a nonhomogeneous Poisson process with linear rate

Motivated by telecommunication applications, we investigate ways to estimate the parameters of a nonhomogeneous Poisson process with linear rate over a finite interval, based on the number of counts in measurement subintervals. Such a linear arrival-rate function can serve as a component of a piecewise-linear approximation to a general arrival-rate function. We consider ordinary least squares (OLS), iterative weighted least squares (IWLS) and maximum likelihood (ML), all constrained to yield a nonnegative rate function. We prove that ML coincides with IWLS. As a reference point, we also consider the theoretically optimal weighted least squares (TWLS), which is least squares with weights inversely proportional to the variances (which would not be known with data). Overall, ML performs almost as well as TWLS. We describe computer simulations conducted to evaluate these estimation procedures. None of the procedures differ greatly when the rate function is not near 0 at either end, but when the rate function is near 0 at one end, TWLS and ML are significantly more effective than OLS. The number of measurement subintervals (with fixed total interval) makes surprisingly little difference when the rate function is not near 0 at either end. The variances are higher with only two or three subintervals, but there usually is little benefit from going above ten. In contrast, more measurement intervals help TWLS and ML when the rate function is near 0 at one end. We derive explicit formulas for the OLS variances and the asymptotic TWLS variances (as the number of measurement intervals increases), assuming the nonnegativity constraints are not violated. These formulas reveal the statistical precision of the estimators and the influence of the parameters and the method. Knowing how the variance depends on the interval length can help determine how to approximate general arrival-rate functions by piecewise-linear ones. We also develop statistical tests to determine whether the linear Poisson model is appropriate.

Regression Models with Incidental Parameters

SUMMARY It is known that iterative weighted least squares (IWLS) does not merely yield a convenient way of fitting the general class of regression models considered by Green but is also a useful means of unifying techniques of model checking. However, an extension of these results to models containing nuisance or incidental parameters has made little progress beyond some special cases. Unfortunately, regression models with nuisance parameters that are not dispersion parameters arise very commonly in applications. The present paper presents a unified treatment of regression models containing incidental parameters that are permitted to contain covariates as well as being functionally dependent on the predictor or regression parameters. For the general regression model considered it is shown that Fisher's scoring algorithm can also be implemented as IWLS. Unlike the commonly suggested fitting strategy which updates the regression parameters with the nuisance parameters assumed known and set at the current values, the present approach yields the correct asymptotic covariance matrix for the regression parameter estimates. By considering simultaneous inference, residuals and influence measures which accommodate the effect of estimating the incidental parameters are outlined.

A new WLS Chebyshev approximation method for the design of FIR digital filters with arbitrary complex frequency response

In this paper, motivated by Chi-Kou's weighted least-squares (WLS) Chebyshev approximation method for the design of FIR filters with linear phase, we propose a new self-initiated iterative WLS Chebyshev approximation method for the design of FIR digital filters with arbitrary complex frequency response and real filter coefficients. The proposed method not only inherits all the advantages of Chi-Kou's method but also is computationally more efficient than Chi-Kou's method for the case of linear phase FIR filter design. On the other hand, contrast to Chit-Mason's time-domain least mean square (LMS) approximation methods, the proposed method is a frequency-domain WLS method based on similar philosophies. Several design examples are provided to demonstrate the good performance of the proposed method.

Estimation of Parameters in Heteroscedastic Linear Models

SUMMARY This paper deals with linear regression models with non-homogeneous error variances. The common situation where the error variance is a smooth function of the values of the regressor variables is considered. The error variance function is represented by a function of known form, but its expression involves the vector of unknown regression coefficients β and an additional vector parameter θ. Estimation of β by iterative weighted least squares (IWLS) therefore also requires updating the estimate of θ at each iteration. When the error variance function is misspecified, different methods of updating θ would yield IWLS estimators of varying degrees of efficiency. Under such situations, a method of updating θ is proposed, resulting in an iterative procedure for estimating β that is asymptotically optimal in the class of IWLS estimators. When the form of the error variance function is correctly specified, the estimation procedure proposed is asymptotically equivalent to weighted least squares with known optimal weights. A real example of its application is given and the extension to general non-linear models is briefly indicated.