Iteratively Reweighted Least Squares Research Articles

Mixture extreme learning machine algorithm for robust regression

The extreme learning machine (ELM) is a well-known approach for training single hidden layer feedforward neural networks (SLFNs) in machine learning. However, ELM is most effective when used for regression on datasets with simple Gaussian distributed error because it often employs a squared loss in its objective function. In contrast, real-world data is often collected from unpredictable and diverse contexts, which may contain complex noise that cannot be characterized by a single distribution. To address this challenge, we propose a robust mixture ELM algorithm, called Mixture-ELM, that enhances modeling capability and resilience to both Gaussian and non-Gaussian noise. The Mixture-ELM algorithm uses an adjusted objective function that blends Gaussian and Laplacian distributions to approximate any continuous distribution and match the noise. The Gaussian mixture accurately models the residual distribution, while the inclusion of the Laplacian distribution addresses the limitations of the Gaussian distribution in identifying outliers. We derive a solution to the novel objective function using the expectation maximization (EM) and iteratively reweighted least squares (IRLS) algorithms. We evaluate the effectiveness of the algorithm through numerical simulation and experiments on benchmark datasets, thereby demonstrating its superiority over other state-of-the-art machine learning methods in terms of robustness and generalization.

Multispectral and Hyperspectral Image Fusion Based on Joint-Structured Sparse Block-Term Tensor Decomposition

Multispectral and hyperspectral image fusion (MHF) aims to reconstruct high-resolution hyperspectral images by fusing spatial and spectral information. Unlike the traditional canonical polyadic decomposition and Tucker decomposition models, the block-term tensor decomposition model is able to improve the quality of fused images using known endmember and abundance information. This paper presents an improved hyperspectral image fusion algorithm. Firstly, the two abundance matrices are combined into a single bulk matrix to promote structural sparsity by introducing the L2,1-norm to eliminate the scaling effects present in the model. Secondly, the counter-scaling effect is eliminated by adding the L2-norm to the endmember matrix. Finally, the chunk matrix and the endmember matrix are coupled together, and the matrix is reorganized by adding the L2,1-norm to the matrix to facilitate chunk elimination and solved using an extended iterative reweighted least squares (IRLS) method, focusing on the problem of the inability to accurately estimate the tensor rank in the chunk-term tensor decomposition model and the noise/artifact problem arising from overestimation of rank. Experiments are conducted on standard and local datasets, and the fusion results are compared and analyzed in four ways: visual result analysis, metric evaluation, time of the algorithm, and classification results, and the experimental results show that the performance of the proposed method is better than the existing methods. An extensive performance evaluation of the algorithms is performed by conducting experiments on different datasets. The experimental results show that the proposed algorithm achieves significant improvements in terms of reconstruction error, signal-to-noise ratio, and image quality compared with the existing methods. Especially in the case of a low signal-to-noise ratio, the proposed algorithm shows stronger robustness and accuracy. These results show that the proposed algorithm has significant advantages in dealing with multispectral high-resolution hyperspectral data.

A robust lp‐norm localization of moving targets in distributed multiple‐input multiple‐output radar with measurement outliers

AbstractThe Gaussian noise model and estimators based on least squares (LS) are widely used in target localisation with distributed multiple‐input multiple‐output (MIMO) radar because of their computational efficiency. However, the accuracy of existing LS‐based target localisation algorithms deteriorates sharply in the presence of outliers in the measurements. Thus, a robust solution is developed based on the ‐norm minimisation criterion and iteratively reweighted least squares (IRLS) for locating a moving target with impulse noise using the angle of arrival (AOA), time delay (TD), and Doppler shift (DS) measurements. First, the AOA, TD, and DS measurement noise models are developed based on the α‐stable distribution. Then, the localisation problem is transformed into an ‐norm minimisation problem by linearising the AOA, TD, and DS measurement equations. Finally, the ‐norm minimisation problem is solved using an IRLS method to obtain the target position and estimate the velocity. Moreover, the optimum of the norm order (p) and the Cramér–Rao lower bound for the target position and velocity estimation are derived under α‐stable distributed measurement noise. The simulation results demonstrate that the developed algorithm offers higher accurascy and robustness than the existing ones in the presence of measurement outliers.

Low-Rank and Row-Sparse Decomposition for Joint DOA Estimation and Distorted Sensor Detection

Distorted sensors could occur randomly and may lead to the breakdown of a sensor array system. We consider an array model within which a small number of sensors are distorted by unknown sensor gain and phase errors. With such an array model, the problem of joint direction-of-arrival (DOA) estimation and distorted sensor detection is formulated under the framework of low-rank and row-sparse decomposition. We derive an iteratively reweighted least squares (IRLS) algorithm to solve the resulting problem. The convergence property of the IRLS algorithm is analyzed by means of the monotonicity and boundedness of the objective function. Extensive simulations are conducted regarding parameter selection, convergence speed, computational complexity, and performances of DOA estimation as well as distorted sensor detection. Even though the IRLS algorithm is slightly worse than the alternating direction method of multipliers in detecting the distorted sensors, the results show that our approach outperforms several state-of-the-art techniques in terms of convergence speed, computational cost, and DOA estimation performance.

EHP VALUE OF MINI LNG SHIP WITH FORM FACTOR FROM PROHASKA AND IRLS METHOD USING SHIP RESISTANCE TESTING DATA

The ship model test was believed to be one of the effective methods for figuring out the boundaries and reliability of the ship's horsepower. The ship's form factor determines a full-scale ship's effective horsepower. Determination of the form factor value can be done experimentally through the Prohaska method. The new method proposed in this study is employed the regression Iteratively Reweighted Least Squares (IRLS) method by utilizing the principle dimension of the ship, such as LWL, B, CB, CP, CM, WSA, T, ∆. etc. The Indonesian Hydrodynamics Laboratory has a database of ships with various principle dimensions which have undergone the towing model test. Through the database, the form factor can be predicted with the IRLS method. The method is then verified and validated with the Prohaska method. The result shows a good agreement with the Prohaska method. The obtained results from the IRLS method also show that the EHP & Resistance calculations are identical with old fashion Prohaska methods. The residual bias factor established by the IRLS method was verified in comparison to the value of the form factor generated by the Prohaska method. Comparison between the two methods results in a small error.

3-D Crosswell electromagnetic inversion based on IRLS norm sparse optimization algorithms

Due to the complexity of underground reservoir distribution and the non-uniqueness of geophysical inverse problems, there is still a lack of practical and effective cross-well electromagnetic inversion methods. Our goal is to develop an efficient method to reduce the non-uniqueness of the physical property model recovered in the inversion. Based on a defined framework, we propose a 3-D crosswell electromagnetic sparse inversion based on iterative reweighted least squares (IRLS) method. We perform the inversion using a hybrid norm formulation, where the lp-norm (0 ≤ p ≤ 1) sparse optimization problem can be transformed into a series of weighted l2-norm optimization problems. Compared with the traditional norm inversion method, the sparse inversion method can make more effective use of the known physical information and obtain more accurate abnormal body position and shape. Finally, the effectiveness of sparse inversion method is verified by model test and inversion of measured data in mining area.

Iteratively reweighted least square for kernel expectile regression with random features

To overcome the computational burden of quadratic programming in kernel expectile regression (KER), iteratively reweighted least square (IRLS) technique was introduced in literature, resulting in IRLS-KER. However, for nonlinear models, IRLS-KER involves operations with matrices and vectors of the same size as the training set. Thus, as the training set becomes large, nonlinear IRLS-KER needs a long training time and large memory. To further alleviate the training cost, this paper projects the original data into a low-dimensional space via random Fourier feature. The inner product of the random Fourier features of two data points is approximately the same as the kernel function evaluated at these two data points. Hence, it is possible to use a linear model in the new low-dimensional space to approximate the original nonlinear model, and consequently, the time/memory efficient linear training algorithms could be applied. This paper applies the idea of random Fourier features to IRLS-KER, and our testing results on simulated and real-world datasets show that, the introduction of random Fourier features makes IRLS-KER achieve similar prediction accuracy as the original nonlinear version with substantially higher time efficiency.

Distributed iterative reweighted least squares algorithm for high-resolution finite-frequency traveltime velocity inversion using VSP data

Seismic traveltime between a source-receiver pair is a finite-frequency phenomenon. The velocity information at all points inside the first Fresnel zone is contained inside the traveltime. Therefore, finite-frequency traveltime velocity inversion (FFTVI) is able to invert velocity with finer structures than that can be obtained by ray-based tomography. Since FFTVI is a global inversion, its cost is lower than full waveform inversion. The implementation in this paper mainly focuses on the numerical aspects of FFTVI. We analyze the characteristics of FFTVI from the computational point of view. A parallel version, Iterative Reweighted Least Squares (IRLS) is designed for solving a large linear system in the memories of computer clusters. This implementation not only yields the accurate inversion of velocity at arbitrary grids and with an arbitrary acquisition geometry, but also is able to mitigate the impacts of traveltime picks with large uncertainties. The proposed workflow and parallel algorithm are demonstrated on synthetic VSP examples designed from a land project.

Two parameter estimators for the Conway–Maxwell–Poisson regression model

The two-parameter estimator (TPE) was proposed for the Poisson regression model. It has the limitation of a single parameter. Contrary to this, count data models often exhibit the problems of dispersion and multicollinearity. The Conway–Maxwell–Poisson regression model (COMPRM) is suitable to handle both the dispersion and the multicollinearity issues simultaneously. The TPE for COMPRM is proposed to overcome these issues. In order to estimate the COMPRM co-efficient, the method of iterative reweighted least square (IRLS) is used. The efficiency of the estimator is evaluated in terms of mean square error (MSE) through a Monte Carlo simulation study. In the presence of multicollinearity, mostly the Asar and Genc’s two-parameter estimator (AGTPE) gives more efficient results for COMPRM as compared to the maximum likelihood (MLE), the Ridge estimator, the Liu estimator and the TPE by Huang and Yang (HYTPE). The proposed estimator is also being studied for real-life application.

ASSESSMENT OF SEA LEVEL RISE IMPACT ON PENINSULAR MALAYSIA GEODETIC VERTICAL DATUM

Abstract. Sea level is the height of the ocean surface in relation to a benchmark or vertical control point. The rising of sea level has become a crucial topic, which frequently discussed among professionals, government, non-government, and researchers upon their devastating impacts on daily human life and human survival. Nevertheless, there is a great deal of ambiguity around the rates, ranges, and time frame of sea levels rise. It is importance to study the variation of present-day sea level rise to understand its impact, particularly on our National Geodetic Vertical Datum (NGVD). This study focuses on quantifying the rate and magnitude of sea level rise over Peninsular Malaysia using in-situ tide gauge and multi-mission satellite altimeter. The time period for tide gauge data used is 35 years (1984–2018) consisting 12 tide gauges in Peninsular Malaysia. Meanwhile, the period of satellite altimetry data is 26 years (1993–2018). This study uses robust fit regression (Iterative Re-weighted Least-Squares) for both tide gauge and satellite altimetry data to calculate the rate of sea level rise. The outcomes of this study show that the rate of sea level rise at Pelabuhan Kelang tide gauge station is the lowest among 12 other tide gauges with a rate of 2.36 ± 0.35 mm/year and slightly affected by sea level rise. Besides, the rate of sea level rise in Peninsular Malaysia is on an upward trend with an average of 3.20 ± 0.27 mm/year from tide gauge data and 4.14 ± 0.32 mm/year from satellite altimetry data. The sea level rise study provides insights to the rate and magnitude of sea level rise at tide gauge stations, particularly at Pelabuhan Kelang station for further study. Last but not least, this study would also benefit authorities in preparing mitigation plans to secure and improve our Peninsular Malaysia Geodetic Vertical Datum (PMGVD) for various applications.

Angular Super-Resolution of Multi-Channel APAR in Interference Environments

Aiming to resolve azimuth-dense targets in interference environments, the radar needs to have the ability of single snap echo angular super-resolution with anti-interference. To solve the problem, the angular super-resolution algorithm based on single snap echo while anti-interference with blocking matrix method is studied for active phased array radar (APAR) in this paper. Since the super-resolution ability of the conventional MUSIC algorithm and iterative adaptive algorithm (IAA) algorithm are limited in single snap echo, the iterative re-weighted least squares (IRLS) algorithm under p-norm constraint is proposed. Further, the near main lobe interference suppression ability is enhanced by the adaptive diagonal loading method. The performance differences of IAA and IRLS algorithms for single-target detection and double-target angular super-resolution are analyzed in detail by numerical simulation on three scenes of no interference, side-lobe interference, and near-main-lobe interference. The simulation results show that the proposed algorithm can effectively solve the problem of target angle estimation and super-resolution based on single sample echo in an interference environment, including near main-lobe interference.

On the exact and efficient solution of the Huber function for measurement applications

The findings of this paper are summarized as follows: (1) We point out that the recently published conclusion that Iteratively Reweighted Least-Squares (IRLS) cannot guarantee the global solution of the well-known Huber function is incorrect. We present that the Huber function has only one minimum and IRLS will obtain the exact minimum unless it is incorrectly implemented. (2) We revisit the variants of the IRLS algorithm and compare them with the exact IRLS algorithm in the geodetic free trilateration networks adjustment. Notably, the pointwise differences between the variants of IRLS and the exact IRLS are even beyond the differences between the Least-Squares solution and the exact IRLS for over 75% points. (3) For rigorously and efficiently solving the Huber M−estimation problem, we propose the Quasi-Newton approach and its limited memory version. The time expense of our proposed algorithm is quite low compared with the time expense of the existing exact IRLS in the cases of a large number of unknown parameters and observations. Our results show that the computation efficiency is improved by one or two orders of magnitude.

Load-flow dependent twofold technique of tolerance selection for robust estimation of power system states

Accurate estimation of the voltage magnitudes, angles, and other states of the modern power system is essential for efficient monitoring. The precision of the meter readings affects the estimating procedure. However, the majority of traditional meter readings are incorrect, and some of them, known as bad-data, are far from the true levels. The estimation performance significantly declines when such poor measurements are present, either singularly or in multiple numbers. Some estimators have post-processing capabilities but are unable to deal with flawed data directly. One of the well-known robust estimators that can weed out faulty data during the estimating process is Least Measurement Rejected (LMR). For each measurement in the LMR formulation, a tolerance value must be carefully chosen. The performance of the estimation can be greatly enhanced by the careful selection of these numbers. The use of load-flow to choose the LMR tolerance values is recommended in this paper to prepare a twofold estimation process of robust nature. Several tests and analysis were done to determine how well the suggested load flow dependent LMR (LFLMR) estimator performed. It is contrasted with well-known estimators from the literature, including Weighed Least Square (WLS), its robust counterpart (RWLS), Weighted Least Absolute Value (WLAV), Iteratively Reweighted Least Square (IRLS), and seven distinct versions of LMR. Additionally, several load profile snapshots are simulated to test the LFLMR's robustness. In order to illustrate the effectiveness of the suggested estimator and to identify and estimate the wrongly measured values of the measurement series, IEEE standard 14, 30, and 118-bus power systems with various outlier scenarios are simulated.

Performance analysis of compressive sensing recovery algorithms for image processing using block processing

<p>The modern digital world comprises of transmitting media files like image, audio, and video which leads to usage of large memory storage, high data transmission rate, and a lot of sensory devices. Compressive sensing (CS) is a sampling theory that compresses the signal at the time of acquiring it. Compressive sensing samples the signal efficiently below the Nyquist rate to minimize storage and recoveries back the signal significantly minimizing the data rate and few sensors. The proposed paper proceeds with three phases. The first phase describes various measurement matrices like Gaussian matrix, circulant matrix, and special random matrices which are the basic foundation of compressive sensing technique that finds its application in various fields like wireless sensors networks (WSN), internet of things (IoT), video processing, biomedical applications, and many. Finally, the paper analyses the performance of the various reconstruction algorithms of compressive sensing like basis pursuit (BP), compressive sampling matching pursuit (CoSaMP), iteratively reweighted least square (IRLS), iterative hard thresholding (IHT), block processing-based basis pursuit (BP-BP) based onmean square error (MSE), and peak signal to noise ratio (PSNR) and then concludes with future works.</p>

Analytical and Residual Bootstrap Methods for Parameter Uncertainty Assessment in Tidal Analysis with Temporally Correlated Noise

Abstract Reconstructing tidal signals is indispensable for verifying altimetry products, forecasting water levels, and evaluating long-term trends. Uncertainties in the estimated tidal parameters must be carefully assessed to adequately select the relevant tidal constituents and evaluate the accuracy of the reconstructed water levels. Customary harmonic analysis uses ordinary least squares (OLS) regressions for their simplicity. However, the OLS may lead to incorrect estimations of the regression coefficient uncertainty due to the neglect of the residual autocorrelation. This study introduces two residual resamplings (moving-block and semiparametric bootstraps) for estimating the variability of tidal regression parameters and shows that they are powerful methods to assess the effects of regression errors with nontrivial autocorrelation structures. A Monte Carlo experiment compares their performance to four analytical procedures selected from those provided by the RT_Tide, UTide, and NS_Tide packages and the robustfit.m MATLAB function. In the Monte Carlo experiment, an iteratively reweighted least squares (IRLS) regression is used to estimate the tidal parameters for hourly simulations of one-dimensional water levels. Generally, robustfit.m and the considered RT_Tide method overestimate the tidal amplitude variability, while the selected UTide and NS_Tide approaches underestimate it. After some substantial methodological corrections the selected NS_Tide method shows adequate performance. As a result, estimating the regression variance–covariance with the considered RT_Tide, UTide, and NS_Tide methods may lead to the erroneous selection of constituents and underestimation of water level uncertainty, compromising the validity of their results in some applications. Significance Statement At many locations, the production of reliable water level predictions for marine navigation, emergency response, and adaptation to extreme weather relies on the precise modeling of tides. However, the complicated interaction between tides, weather, and other climatological processes may generate large uncertainties in tidal predictions. In this study, we investigate how different statistical methods may lead to different quantification of tidal model uncertainty when using data with completely known properties (e.g., knowing the tidal signal, as well as the amount and structure of noise). The main finding is that most commonly used statistical methods may estimate incorrectly the uncertainty in tidal parameters and predictions. This inconsistency is due to some specific simplifying assumptions underlying the analysis and may be reduced using statistical techniques based on data resampling.

A Robust Unscented M-Estimation-Based Filter for Vehicle State Estimation With Unknown Input

Longitudinal velocity, lateral velocity, and front wheel steering angle are crucial states for vehicle active safety features. However, direct measurement of these variables requires expensive measurement instruments and can be easily affected by vehicle nonlinear dynamics, outliers and noise pollution. Moreover, unknown inputs bring great challenges to their accurate estimation. Therefore, this paper develops a novel robust unscented M-estimation-based filter (RUMF) for vehicle state estimation with unknown driver steering torque. The nonlinear system model is constructed based on the vehicle dynamics model. Unscented transformation (UT) and statistical linearization are implemented to transform the nonlinear process and measurement function into a linear-like regression form with data redundancy to achieve outlier suppression. Then, an M-estimation-based iterated algorithm is designed to address process uncertainty and innovation and observation outliers for robust vehicle state estimation. The iteratively reweighted least-squares (IRLS) method based on the M-estimation methodology is utilized for unknown input estimation. Simulations and experimental results compared with extended Kalman filter (EKF) and particle filter (PF) have verified the effectiveness and robustness of the proposed algorithm.

Evaluation of regularization methods for acoustic pyrometry

Measurement of temperature distribution is vital in boilers, heat exchangers, and other industrial applications. Acoustic pyrometry offers the advantage of measuring the temperature in the entire domain in a non-intrusive manner. Acoustic pyrometry involves estimating the temperature of the domain using the time of flight information between the transceivers. Since acoustic pyrometry is an inverse problem, it is sensitive to noise and the number of cells in the domain. Regularization methods help in obtaining feasible solutions. Therefore, the performance of four regularization methods, namely, Tikhonov, modified Tikhonov, Total variation (TV), and iterative reweighted least-squares (IRLS) in reconstructing three different temperature profiles, is studied and compared against the pseudo-inverse method. The effect of noise and the cell ratio (CR) on temperature reconstruction is also studied. Overall, the best results are observed for the coarsest cell ratio and modified Tikhonov with sharp filter regularization.

Image Deblurring Algorithm Based on the Gaussian-Scale Mixture Expert Field Model

With the proliferation of portable digital products, image quality degradation has received a lot of attention. As the most common phenomenon in image degradation, the issue of image deblurring is the focus of much attention. Blind motion blur removal is the main target of this paper. The heavy-tailed distribution is the most dominant statistical feature of natural images. However, most image deblurring methods use a gradient prior with fixed parameters to recover a clear image, which leads to loss of details in the recovered clear image and does not consider the higher order prior of the natural image. Therefore, this paper proposes a new regularized image recovery model based on the Gaussian-scale mixture expert field(GSM-FOE) model. First, the GSM-FoE model learns filters and corresponding parameters with higher order prior information of images by training images in a natural image library; second, these learning results are used to guide the image recovery process. The GSM-FoE model and gradient-fidelity based image recovery model is proposed, which can be used with an iterative re-weighted least squares (IRLS) method. Experiments demonstrate that the suggested recovery approach is simple to use and successful at reducing blur and noise, as well as suppressing ringing effects while preserving image information. Moreover, the image restoration method performs well for large blurring kernels. The results fully reflect the effectiveness and robustness of the proposed method for complex noise scenarios. The quality of the generated images is significantly better than that of several classical methods.

Adaptive loss function based least squares one-class support vector machine

• Adaptive loss function based least squares one-class support vector machine is proposed. • Fisher consistency of the adaptive loss function is proved. • Iteratively reweighted least squares (IRLS) method is used to solve the optimization problem. • Efficiency of the proposed method is verified in comparison with its related methods. Least squares one-class support vector machine (LS-OCSVM) can accurately describe the similarity between new sample and training set. However, LS-OCSVM is very sensitive to the outliers among training samples, which means that the separating hyperplane of LS-OCSVM may deviate from the normal data even with a few outliers. To enhance the anti-outlier performance of LS-OCSVM, a novel adaptive loss function based LS-OCSVM is proposed. In the proposed method, an adaptive loss function is utilized to substitute the square loss function in the objective function of LS-OCSVM. The property of Fisher consistency for the adaptive loss function is validated from the theoretical viewpoint. The optimization problem of the proposed method is solved by the iteratively reweighted least squares (IRLS) method. In comparison with its nine related methods, the proposed method demonstrates better anti-outlier and generalization abilities on synthetic and benchmark data sets.

Electroencephalography robust statistical linear modelling using a single weight per trial

Being able to remove or weigh down the influence of outlier data is desirable for any statistical model. While magnetic and electroencephalographic (MEEG) data are often averaged across trials per condition, it is becoming common practice to use information from all trials to build statistical linear models. Individual trials can, however, have considerable weight and thus bias inferential results (effect sizes as well as thresholded t/F/p maps). Here, rather than looking for univariate outliers, defined independently at each measurement point, we apply the principal component projection (PCP) method at each channel, deriving a single weight per trial at each channel independently. Using both synthetic data and open electroencephalographic (EEG) data, we show (1) that PCP is efficient at detecting a large variety of outlying trials; (2) how PCP-based weights can be implemented in the context of the general linear model (GLM) with accurate control of type 1 family-wise error rate; and (3) that our PCP-based weighted least squares (WLS) approach increases the statistical power of group analyses as well as a much slower iterative reweighted least squares (IRLS) approach, although the weighting scheme is markedly different. Together, our results show that WLS based on PCP weights derived from whole trial profiles is an efficient method to weigh down the influence of outlier EEG data in linear models.