Iteratively Reweighted Least Squares Research Articles

An Iteratively Reweighted Importance Kernel Bayesian Filtering Approach for High-Dimensional Data Processing

This paper proposes an iteratively re-weighted importance kernel Bayes filter (IRe-KBF) method for handling high-dimensional or complex data in Bayesian filtering problems. This innovative approach incorporates importance weights and an iterative re-weighting scheme inspired by iteratively re-weighted Least Squares (IRLS) to enhance the robustness and accuracy of Bayesian inference. The proposed method does not require explicit specification of prior and likelihood distributions; instead, it learns the kernel mean representations from training data. Experimental results demonstrate the superior performance of this method over traditional KBF methods on high-dimensional datasets.

Robust algorithms for spherical angle-of-arrival source localization

The performance of traditional algorithms for spherical angle-of-arrival (AOA) source localization will be significantly degraded when there are outliers in the angle measurements. By using the symmetric α-stable (SαS) distribution to describe the measurement noise containing outliers and constructing the cost function using the lp-norm, we propose a robust algorithm for spherical AOA source localization: the spherical iteratively reweighted pseudolinear estimator (SIRPLE). The SIRPLE is similar to the iteratively reweighted least squares (IRLS), with the difference that a homogeneous least squares (HLS) problem is solved in each iteration. The SIRPLE suffers from bias problems owing to the nature of the pseudolinear estimators. To overcome this problem, the instrumental variable (IV) method is introduced and the spherical iteratively reweighted instrumental variable estimator (SIRIVE) is proposed. Theoretical analysis shows that the SIRIVE is asymptotically unbiased and it can achieve the theoretical error covariance of the constrained least lp-norm estimation. Extensive simulation analyses demonstrate the better performance of the SIRIVE compared to the conventional spherical AOA source localization methods and the SIRPLE under SαS noise environment. The performance of the SIRIVE is similar to that of the Nelder–Mead algorithm (NM), but the SIRIVE are computationally more efficient. In addition, the SIRIVE is nearly unbiased and the root mean square error (RMSE) performance is close to the Cramér–Rao lower bound (CRLB).

Robust Direction-of-Arrival Estimation in the Presence of Outliers and Noise Nonuniformity

In direction-of-arrival (DOA) estimation with sensor arrays, the background noise is usually modeled to be uncorrelated uniform white noise, such that the related algorithms can be greatly simplified by making use of the property of the noise covariance matrix being a diagonal matrix with identical diagonal entries. However, this model can be easily violated by the nonuniformity of sensor noise and the presence of outliers that may arise from unexpected impulsive noise. To tackle this problem, we first introduce an exploratory factor analysis (EFA) model for DOA estimation in nonuniform noise. Then, to deal with the outliers, a generalized extreme Studentized deviate (ESD) test is applied for outlier detection and trimming. Based on the trimmed data matrix, a modified EFA model, which belongs to weighted least-squares (WLS) fitting problems, is presented. Furthermore, a monotonic convergent iterative reweighted least-squares (IRLS) algorithm, called the iterative majorization approach, is introduced to solve the WLS problem. Simulation results show that the proposed algorithm offers improved robustness against nonuniform noise and observation outliers over traditional algorithms.

Fast Algorithms for \(\ell_p\)-Regression

The ℓ p -norm regression problem is a classic problem in optimization with wide ranging applications in machine learning and theoretical computer science. The goal is to compute \(\boldsymbol {\mathit {x}}^{\star } =\arg \min _{\boldsymbol {\mathit {A}}\boldsymbol {\mathit {x}}=\boldsymbol {\mathit {b}}}\Vert \boldsymbol {\mathit {x}}\Vert _p^p \) , where \(\boldsymbol {\mathit {x}}^{\star }\in \mathbb {R}^n,\boldsymbol {\mathit {A}}\in \mathbb {R}^{d\times n},\boldsymbol {\mathit {b}} \in \mathbb {R}^d \) and d ≤ n . Efficient high-accuracy algorithms for the problem have been challenging both in theory and practice and the state-of-the-art algorithms require \(poly(p)\cdot n^{\frac{1}{2}-\frac{1}{p}} \) linear system solves for p ≥ 2. In this paper, we provide new algorithms for ℓ p -regression (and a more general formulation of the problem) that obtain a high-accuracy solution in O ( pn ( p − 2)/(3 p − 2) ) linear system solves. We further propose a new inverse maintenance procedure that speeds-up our algorithm to \(\widetilde{O}(n^{\omega }) \) total runtime, where O ( n ω ) denotes the running time for multiplying n × n matrices. Additionally, we give the first Iteratively Reweighted Least Squares (IRLS) algorithm that is guaranteed to converge to an optimum in a few iterations. Our IRLS algorithm has shown exceptional practical performance, beating the currently available implementations in MATLAB/CVX by 10-50x.

Multiple receiver specific emitter identification

AbstractSpecific emitter identification (SEI) is a technique for identifying emitters based on the principle that the hardware chain is not ideal, causing the emitted signal to contain emitter‐specific information. However, the receiver is also non‐ideal, which affects recognition accuracy and introduces receiver‐specific information that makes SEI difficult to generalise across receiving systems. In this work, a new multi‐receiver receiving and processing system (MR‐SEI) scheme is proposed to mitigate the influence of receivers based on the analysis of receiver distortion models. After receiving and processing in a specific manner, recognition performance can be enhanced. Therefore, extracted features can be shared among different receivers and platforms, and can even be applied to newly added receivers. The concept of common waveform (CW) is first defined, referring to the received signal without receiver distortions. Different receiving devices are working synchronously, and the CW is estimated using multiple copies of the signal obtained from multiple receivers through the iterative reweighted least squares (IRLS) method. For each receiver, a maximum linear correlation algorithm is proposed to calculate the received signal without being affected by distortions. Experimental results show that the proposed scheme can enhance identification performance. With the increase in the number of receivers, the improvement is more noticeable. Using 10 distorted receivers operating under an SNR of 25 dB, the proposed algorithm can significantly improve the identification performance, achieving over 95% and approaching the ideal scenario of no receiver distortion. Meanwhile, influences caused by receiver distortions can be effectively eliminated, and the database can be shared with new receivers, overperforming other SEI methods that eliminate the receiver.

FeatSync: 3D point cloud multiview registration with attention feature-based refinement

Current studies often decompose multiview registration into several individual tasks, ignoring the correlation between each stage and making certain assumptions on noise distribution without knowledge from previous stages. These issues bring difficulties in generalization to real cases. In this paper, we propose an end-to-end feature-based multiview registration model that takes a set of raw 3D point cloud fragments as input and outputs the global transformation. Unlike previous works, our method allows the exchange of information between stages. We firstly estimate pairwise registration by a attention-based model to assist feature learning. In the next stage, we utilize iteratively reweighted least squares (IRLS) algorithm to refine and obtain the global transformation. In each iteration, instead of making assumptions on noises, we directly construct a model to infer the outliers from pairwise registration so that such an inference can help synchronization produce more reliable results. To follow the process in IRLS algorithm, we propose a simple yet effective refinement module to boost feature-based pairwise estimations in an iterative manner, which can be seamlessly integrated into the IRLS procedure. Extensive experiments conducted on benchmark datasets show that the results of our proposed method outperformed existing methods.

Chebyshev polynomial-based Fourier transformation and its use in low pass filter of gravity data

In this paper, we introduce the novel Chebyshev Polynomials Least-Squares Fourier Transformation (C-LSQ-FT) and its robust variant with the Iteratively Reweighted Least-Squares technique (C-IRLS-FT). These innovative techniques for Fourier transformation are predicated on the concept of inversion, and the C-LSQ-FT method establishes an overdetermined inverse problem within the realm of Fourier transformation. However, given the LSQ approach’s vulnerability to data outliers, we note the potential for considerable errors and potentially unrepresentative model estimations. To circumvent these shortcomings, we incorporate Steiner’s Most Frequent Value method into our framework, thereby providing a more reliable alternative. The fusion of the Iteratively Reweighted Least-Squares (IRLS) algorithm with Cauchy-Steiner weights enhances the robustness of our Fourier transformation process, culminating in the C-IRLS-FT method. We use Chebyshev polynomials as the basis functions in both methods, leading to the approximation of continuous Fourier spectra through a finite series of Chebyshev polynomials and their corresponding coefficients. The coefficients were obtained by solving an overdetermined non-linear inverse problem. We validated the performance of both the traditional Discrete Fourier Transform (DFT) and the newly developed C-IRLS-FT through numerical tests on synthetic datasets. The results distinctly exhibited the reduced sensitivity of the C-IRLS-FT method to outliers and dispersed noise, in comparison with the traditional DFT. We leveraged the newly proposed (C-IRLS-FT) technique in the application of low-pass filtering in the context of gravity data. The results corroborate the technique’s robustness and adaptability, making it a promising method for future applications in geophysical data processing.

An evaluation of computational methods for aggregate data meta-analyses of diagnostic test accuracy studies

BackgroundA Generalized Linear Mixed Model (GLMM) is recommended to meta-analyze diagnostic test accuracy studies (DTAs) based on aggregate or individual participant data. Since a GLMM does not have a closed-form likelihood function or parameter solutions, computational methods are conventionally used to approximate the likelihoods and obtain parameter estimates. The most commonly used computational methods are the Iteratively Reweighted Least Squares (IRLS), the Laplace approximation (LA), and the Adaptive Gauss-Hermite quadrature (AGHQ). Despite being widely used, it has not been clear how these computational methods compare and perform in the context of an aggregate data meta-analysis (ADMA) of DTAs.MethodsWe compared and evaluated the performance of three commonly used computational methods for GLMM - the IRLS, the LA, and the AGHQ, via a comprehensive simulation study and real-life data examples, in the context of an ADMA of DTAs. By varying several parameters in our simulations, we assessed the performance of the three methods in terms of bias, root mean squared error, confidence interval (CI) width, coverage of the 95% CI, convergence rate, and computational speed.ResultsFor most of the scenarios, especially when the meta-analytic data were not sparse (i.e., there were no or negligible studies with perfect diagnosis), the three computational methods were comparable for the estimation of sensitivity and specificity. However, the LA had the largest bias and root mean squared error for pooled sensitivity and specificity when the meta-analytic data were sparse. Moreover, the AGHQ took a longer computational time to converge relative to the other two methods, although it had the best convergence rate.ConclusionsWe recommend practitioners and researchers carefully choose an appropriate computational algorithm when fitting a GLMM to an ADMA of DTAs. We do not recommend the LA for sparse meta-analytic data sets. However, either the AGHQ or the IRLS can be used regardless of the characteristics of the meta-analytic data.

3D high-resolution Radon transform based on strong sparse LP‒1 norm and its applications

Abstract Multiple reflections are among the most challenging noises to suppress in seismic data, as they differ from effective waves only in terms of apparent velocity. Besides, the Radon transform, an essential technique for attenuating multiple reflections, has been widely incorporated into various commercial software packages. Thus, this study introduces a 3D Radon transform method based on the LP‒1 norm to enhance sparsity-constraining capability in the transform domain, leveraging high-resolution Radon transform techniques. Specifically, an iteratively reweighted least squares (IRLS) algorithm is employed to obtain the transformed data in the Radon domain. Given that the LP‒1 norm is applied to seismic data processing for the first time, this paper theoretically demonstrates its powerful sparsity-constraining capability. Indeed, the proposed strategy enhances energy concentration in the Radon transform domain, better-separating primaries from multiples and ultimately suppressing the multiples. Both model tests and real data indicate that the 3D Radon transform constrained by the LP‒1 norm outperforms existing high-resolution Radon transform methods with sparsity constraints regarding energy concentration and effectiveness in multiple reflection attenuation.

The global effects of digestive system cancers worldwide on the COVID-19 pandemic by negative binomial (mixed) regression models

In this study, the main goal is to determine statistically significant relationships between the biggest epidemic problem of last years as the COVID-19 pandemic and digestive system cancers belonging to 168 countries worldwide using generalized linear mixed model (GLMM) and its special case as generalized linear model (GLM) approaches, and to obtain global inferences that will shed light on this pandemic. For this goal, the response variable is “total cases of the COVID-19 pandemic per 100,000 people” until January 14, 2022. The explanatory variables are total number of people suffering from colon and rectum, stomach, lip and oral cavity, esophageal, and nasopharynx cancers belonging to 2019, respectively. In this study, the negative binomial (NB) regression model in GLM using iteratively reweighted least squares (IRLS) algorithm and the NB mixed regression model in GLMM with “countries” taken as “random effects” using Adaptive Gauss-Hermite Quadrature (AGHQ) approximation method at 1, 2, 10, and 20 quadrature points are used for modelling the COVID-19 pandemic and digestive systems cancer data. In this study, by using information criteria, the NB mixed regression model in GLMM under log-link function with 168 countries taken as random effects using AGHQ method at 20 quadrature point is detected as the most appropriate model for the global COVID-19 pandemic and digestive system cancers data.

Solving Maxmin Optimization Problems via Population Games

Population games are games with a finite set of available strategies and an infinite number of players, in which the reward for choosing a given strategy is a function of the distribution of players over strategies. The paper shows that, in a certain class of maxmin optimization problems, it is possible to associate a population game to a given maxmin problem in such a way that solutions to the optimization problem are found from Nash equilibria of the associated game. Iterative solution methods for maxmin optimization problems can then be derived from systems of differential equations whose trajectories are known to converge to Nash equilibria. In particular, we use a discrete-time version of the celebrated replicator equation of evolutionary game theory, also known in machine learning as the exponential multiplicative weights algorithm. The resulting algorithm can be viewed as a generalization of the Iteratively Reweighted Least Squares (IRLS) method, which is well known in numerical analysis as a useful technique for solving Chebyshev function approximation problems on a finite grid. Examples are provided to show the use of the generalized IRLS method in collective investment and in decision making under model uncertainty.

L1-norm optimisation of rank deficient GNSS networks by an improved Grey Wolf method

An improved grey wolf optimization (GWO) is proposed for direct L1-norm optimization in rank deficient GNSS networks to detect outliers since the standard GWO and the global optimization (GO) algorithms cannot deal with such problems. In contrast to the traditional method for L1-norm minimization which is solved by a cumbersome search of linear programming (LP) problem, in this algorithm, one only requires to input the target function. Furthermore, it has a greater ability to detect gross errors of observables with low reliability compared to traditional methods such as LP. Moreover, it does not consider the correct observables as blunders in contrast to some algorithms such as the iteratively reweighted least-squares (IRLS) method, which may consider them as outliers. The numerical results of real and simulated GNSS networks approve the efficiency of the new algorithm.

Fully automated structured light scanning for high-fidelity 3D reconstruction via graph optimization.

Convenient and high-fidelity 3D model reconstruction is crucial for industries like manufacturing, medicine and archaeology. Current scanning approaches struggle with high manual costs and the accumulation of errors in large-scale modeling. This paper is dedicated to achieving industrial-grade seamless and high-fidelity 3D reconstruction with minimal manual intervention. The innovative method proposed transforms the multi-frame registration into a graph optimization problem, addressing the issue of error accumulation encountered in frame-by-frame registration. Initially, a global consistency cost is established based on point cloud cross-multipath registration, followed by using the geometric and color differences of corresponding points as dynamic nonlinear weights. Finally, the iteratively reweighted least squares (IRLS) method is adopted to perform the bundle adjustment (BA) optimization of all poses. Significantly enhances registration accuracy and robustness under the premise of maintaining near real-time efficiency. Additionally, for generating watertight, seamless surface models, a local-to-global transitioning strategy for multiframe fusion is introduced. This method facilitates efficient correction of normal vector consistency, addressing mesh discontinuities in surface reconstruction resulting from normal flips. To validate our algorithm, we designed a 3D reconstruction platform enabling spatial viewpoint transformations. We collected extensive real and simulated model data. These datasets were rigorously evaluated against advanced methods, roving the effectiveness of our approach. Our data and implementation is made available on GitHub for community development.

Analyzing Motorcycle Accident Frequency Using Generalized Poisson Distributions

Motorcycle accidents in East Java are more common than accidents in other modes of transportation. In addition to the many motorcycle users today, human, environmental, and road factors are considered the highest causes of these accidents. The study's goal is to find the best model of the Generalized Poisson Family Distribution (GPR), namely Lagrangian Poisson Regression (LPR) and to construct a model that will quantify the frequency of motorcycle accidents in East Java. Akaike Information Criterion (AIC) and Schwarz Bayesian Criterion (SBC) criteria are the model comparison methods used in this research. The selection was also made based on the model's exponential coefficient with a 95% CI to further deepen the selection results obtained. In addition, the paired samples test was performed to determine the degree of dissimilarity between the outcomes produced by the developed model and the actual data. The best performance model is applied to identify the characteristics or factors highly involved in motorcycle accidents. The research uses secondary data from related agencies, namely the East Java Regional Police, especially the traffic accident unit, and East Java BPS, for 38 cities and districts in 2021. The numerical optimization method used is the iteratively reweighted least squares (IRLS) algorithm, assisted by R Studio software. The study findings show that LPR is the most efficient and exact approach for modeling the frequency of motorcycle accidents. Meanwhile, the percentage of teenagers (X1), the frequency of motorized vehicles (X3), and the average annual rainfall (X5) have a considerable impact on accident occurrence. This research has an important contribution, especially in the field of transportation modeling and designing appropriate strategies to reduce the frequency of motorcycle accidents.

An errors-unidentified hand-eye calibration method via robot relocalization and iterative combinatorial refinement

The accuracy of hand-eye calibration is easily affected by robot kinematic error, measurement error, random error, etc., and can be to a certain extent enhanced by the complicated error identification. To address this problem, we propose a novel errors-unidentified hand-eye calibration method. A relocalization-based hand-eye calibration is implemented to overcome the challenge from robot kinematics at first owing to the high robot relocalization accuracy. The tool center point (TCP) coordinates are then obtained based on an iterative reweighted least squares (IRLS) spherical fitting algorithm. An iterative combinatorial refinement algorithm is finally presented to search the solutions in the form of permutation and combination when solving the rigid transformation matrix. Experiments on criterion sphere and blade demonstrate that by converting the hand-eye matrix solving problem into a point set matching problem, the accuracy of the proposed method is enhanced by 60% on average compared with three state-of-the-art hand-eye calibration methods.

Relationship Between World Happiness Index, Hoefstede Cultural Dimensions And Government Education Spending

The author analyzes the interplay between the World Happiness Index and a nuanced array of seven variables in this research. Specifically, the focus extends to the Hofstede Cultural Dimensions, encompassing six distinct facets and the financial allocations dedicated to government education spending. The author starts with a literature survey on the profound domains of happiness and wellbeing. Building upon this foundation, the author posits the Hofstede Cultural Dimensions and Government Education Expenditures as prospective variables that may yield a discernible influence on the World Happiness Index. A dataset spanning 58 countries and analytical methodologies of Ordinary Least Squares (OLS) regression and Iteratively Reweighted Least Squares (IRLS) regression are used to test this conjecture. The outcome of this study reveals compelling insights into the association between the World Happiness Index and the identified variables. The results underscore a discernible relationship between the World Happiness Index and specific dimensions of cultural orientation, such as individualism, long-term orientation, and indulgence. Furthermore, the study reveals that government investment in education is another factor influencing the overarching landscape of happiness. These findings transcend mere statistical findings; they serve as beacons of knowledge with far-reaching implications for education policymakers, educators, and individuals interested in understanding the complexities of cultural variations, educational dynamics, and overall wellbeing. The implications of this research can offer relevant information to those who navigate the intricate intersections of cultural diversity, education policy, and the pursuit of societal happiness, such as education policymakers, educators and anyone interested in the issues of cultural differences, education, and wellbeing.

An improved parameter estimation of HFM signals based on IRLS linear fitting of extracted group delay

To address the issue of robust parameter estimation of hyperbolic frequency modulated (HFM) signals under low signal-to-noise ratios (SNRs), an improved algorithm based on iteratively reweighted least squares (IRLS) linear fitting of extracted group delay is proposed. The algorithm includes three main steps. First, a novel method for estimating the frequency distribution range (FDR) of an HFM signal is presented, which involves analyzing the transient features of the amplitude spectrum (AS). Second, the spectrum phase (SP) is unwrapped in the estimated FDR to extract the group delay (GD) of the HFM signal. Last, parameter estimation of the HFM signal is achieved by fitting the GD, which is a linear function of the reciprocal of the frequency, using the IRLS linear fitting method. The simulation results have shown that for SNR ≥ -14 dB, the normalized root mean square errors (NRMSEs) estimated by the proposed estimator are 1.5 dB lower than those estimated by the spectrum phase unwrapping (SPU) estimator, including the starting frequency and period slope. Furthermore, experimental results based on sea trial data have also verified the validity and feasibility of the proposed estimator.

Formula omitted]-norm minimization for outlier-resistant elliptic positioning in [formula omitted]-stable impulsive interference

This short communication considers the problem of robust elliptic positioning (EP) in α-stable impulsive interference based on the ℓp-norm minimization criterion. We start with converting the associated nonconvex optimization task into the well-established ℓp-minimization framework of iteratively reweighted least squares (IRLS). While the effect of p is taken into account by a weight update step in each IRLS iteration, we employ an assortment of surrogate functions, in order to effectively handle the location update subproblem using the majorization–minimization technique. The superiority of our proposal over a number of existing EP methods in terms of localization accuracy in α-stable impulsive noise is demonstrated via numerical simulations.

Iteratively Reweighted Least Squares Method for Estimating Polyserial and Polychoric Correlation Coefficients

An iteratively reweighted least squares (IRLS) method is proposed for the estimation of polyserial and polychoric correlation coefficients in this article. It calculates the slopes in a series of weighted linear regression models fitting on conditional expected values. For polyserial correlation, conditional expectations of the latent predictor is derived from the observed ordinal categorical variable, and the regression coefficient is obtained with the weighted least squares method. In estimating polychoric correlation coefficient, conditional expectations of the response variable and the predictor are updated in turns. Standard errors of the estimators are obtained using the delta method based on data summaries instead of the whole data. Conditional univariate normal distribution is exploited and a single integral is numerically evaluated in the proposed algorithm, comparing to the double integral computed numerically based on the bivariate normal distribution in the traditional maximum likelihood (ML) approaches. This renders the new algorithm very fast in the estimation of both polyserial and polychoric correlation coefficients. Thorough simulation studies are conducted to compare the performances of the proposed method with the classical ML methods. Real data analyses illustrate the advantage of the new method in computing speed. Supplementary materials for this article are available online.

Mixture extreme learning machine algorithm for robust regression

The extreme learning machine (ELM) is a well-known approach for training single hidden layer feedforward neural networks (SLFNs) in machine learning. However, ELM is most effective when used for regression on datasets with simple Gaussian distributed error because it often employs a squared loss in its objective function. In contrast, real-world data is often collected from unpredictable and diverse contexts, which may contain complex noise that cannot be characterized by a single distribution. To address this challenge, we propose a robust mixture ELM algorithm, called Mixture-ELM, that enhances modeling capability and resilience to both Gaussian and non-Gaussian noise. The Mixture-ELM algorithm uses an adjusted objective function that blends Gaussian and Laplacian distributions to approximate any continuous distribution and match the noise. The Gaussian mixture accurately models the residual distribution, while the inclusion of the Laplacian distribution addresses the limitations of the Gaussian distribution in identifying outliers. We derive a solution to the novel objective function using the expectation maximization (EM) and iteratively reweighted least squares (IRLS) algorithms. We evaluate the effectiveness of the algorithm through numerical simulation and experiments on benchmark datasets, thereby demonstrating its superiority over other state-of-the-art machine learning methods in terms of robustness and generalization.