Weighted Least-squares Estimation Research Articles

BEP Estimation by Weighted Least Squares From the Ratios of Frame Syncword Error Rates

For telemetry data processing systems, estimating the bit error probability of received data is important in terms of data quality. For systems in which the number of allowable errors in an <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$M$</tex-math></inline-formula> -bit frame syncword (FS) of a telemetry data frame is <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$K$</tex-math></inline-formula> and the numbers of FSs with no error, one error, <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\dots$</tex-math></inline-formula> , <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$K$</tex-math></inline-formula> errors, and more than <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$K$</tex-math></inline-formula> errors are given, respectively, it is known from the literature that conventional methods, including the maximum-likelihood estimator (MLE), are not generally presented as a closed-form expression excluding specific cases. This article proposes a weighted least squares estimator (WLSE) by taking the ratios of the observed FS error rates to minimize the squared discrepancies between the observed and the predicted values, and the WLSE is obtained by straightforward calculation. We analyze the characteristics of the bias and the variance and derive optimal weights that minimize the variance considering that the mean squared error (MSE) of the proposed estimator depends on the variance rather than the bias. Based on the derived optimal weights, a method is proposed to sequentially obtain the weights close to optimum. The analytical and simulation results verify that the MSE of the proposed estimator is only slightly larger or even less than those of the existing methods, while the proposed estimator has a significantly lower computational complexity than those of the conventional schemes.

A new modified Kies Fréchet distribution: Applications of mortality rate of Covid-19

The purpose of this paper is to identify an effective statistical distribution for examining COVID-19 mortality rates in Canada and Netherlands in order to model the distribution of COVID-19. The modified Kies Frechet (MKIF) model is an advanced three parameter lifetime distribution that was developed by incorporating the Frechet and modified Kies families. In particular with respect to current distributions, the latest one has very versatile probability functions: increasing, decreasing, and inverted U shapes are observed for the hazard rate functions, indicating that the capability of adaptability of the model. A straight forward linear representation of PDF, moment generating functions, Probability weighted moments and hazard rate functions are among the enticing features of this novel distribution. We used three different estimation methodologies to estimate the pertinent parameters of MKIF model like least squares estimators (LSEs), maximum likelihood estimators (MLEs) and weighted least squares estimators (WLSEs). The efficiency of these estimators is assessed using a thorough Monte Carlo simulation analysis. We evaluated the newest model for a variety of data sets to examine how effectively it handled data modeling. The real implementation demonstrates that the proposed model outperforms competing models and can be selected as a superior model for developing a statistical model for COVID-19 data and other similar data sets.

Weighted least squares estimators for the Parzen tail index

Estimation of the tail index of heavy-tailed distributions and its applications are essential in many research areas. We propose a class of weighted least squares (WLS) estimators for the Parzen tail index. Our approach is based on the method developed by Holan and McElroy (J Stat Plan Inference 140(12):3693–3708, 2010). We investigate consistency and asymptotic normality of the WLS estimators. Through a simulation study, we make a comparison with the Hill, Pickands, DEdH (Dekkers, Einmahl and de Haan) and ordinary least squares (OLS) estimators using the mean square error as criterion. The results show that in a restricted model some members of the WLS estimators are competitive with the Pickands, DEdH and OLS estimators.

Accelerated failure-time model with weighted least-squares estimation: application on survival of HIV positives

BackgroundSurvival analysis is the most appropriate method of analysis for time-to-event data. The classical accelerated failure-time model is a more powerful and interpretable model than the Cox proportional hazards model, provided that model imposed distribution and homoscedasticity assumptions satisfied. However, most of the real data are heteroscedastic which violates the fundamental assumption and consequently, the statistical inference could be erroneous in accelerated failure-time modeling. The weighted least-squares estimation for the accelerated failure-time model is an efficient semi-parametric approach for time-to-event data without the homoscedasticity assumption, which is developed recently and not often utilized for real data analysis. Thus, this study was conducted to ascertain the better performance of the weighted least-squares estimation method over the classical methods.MethodsWe analyzed a REAL dataset on Antiretroviral Therapy patients we recently collected. We compared the results from classical methods of estimation for the accelerated failure-time model with the results revealed from the weighted least-squares estimation.ResultsWe found that the data are heteroscedastic and indicated that the weighted least-square method should be used to analyze this data. The weighted least-squares estimation revealed more accurate, and efficient estimates of covariates effect since its confidence intervals were shorter and it identified more significant covariates. Accordingly, the survival of HIV positives was found to be significantly linked with age, weight, functional status, CD4 (Cluster of Differentiation agent 4 glycoproteins), and clinical stages.ConclusionsThe weighted least-squares estimation performed the best in providing more significant effects and precise estimates than the classical accelerated failure-time methods of estimation if data are heteroscedastic. Thus, we recommend future researchers should utilize weighted least-squares estimation rather than the classical methods when the homoscedasticity assumption is violated.

A multivariate HAR-RV model with heteroscedastic errors and its WLS estimation

This work considers a multivariate heterogeneous autoregressive-realized volatility (HAR-RV) model in the presence of heteroscedasticity and aims to analyze realized volatilities of multiple assets that possess non-standard features, such as non-Gaussianity, time varying volatility and long-memory dependence. For capturing the long-memory, a HAR-RV model is employed, while for a heavy-tailed distribution, a GARCH process is adopted on the noise term. To estimate coefficients of the HAR-RV-GARCH model, we suggest weighted least squares estimator (WLSE) based on an observed weighting scheme and prove its asymptotic normality. Simulation results show a good performance on the WLSE. The multivariate HAR-RV-GARCH model fitted by the WLSE is illustrated with an application to realized volatilities of multiple financial data.

A New Inverted Topp-Leone Distribution: Applications to the COVID-19 Mortality Rate in Two Different Countries

This paper aims to find a statistical model for the COVID-19 spread in the United Kingdom and Canada. We used an efficient and superior model for fitting the COVID 19 mortality rates in these countries by specifying an optimal statistical model. A new lifetime distribution with two-parameter is introduced by a combination of inverted Topp-Leone distribution and modified Kies family to produce the modified Kies inverted Topp-Leone (MKITL) distribution, which covers a lot of application that both the traditional inverted Topp-Leone and the modified Kies provide poor fitting for them. This new distribution has many valuable properties as simple linear representation, hazard rate function, and moment function. We made several methods of estimations as maximum likelihood estimation, least squares estimators, weighted least-squares estimators, maximum product spacing, Crame´r-von Mises estimators, and Anderson-Darling estimators methods are applied to estimate the unknown parameters of MKITL distribution. A numerical result of the Monte Carlo simulation is obtained to assess the use of estimation methods. also, we applied different data sets to the new distribution to assess its performance in modeling data.

Topp-Leone odd log-logistic exponential distribution: Its improved estimators and applications.

In this paper, a new three-parameter lifetime model called the Topp-Leone odd log-logistic exponential distribution is proposed. Its density function can be expressed as a linear mixture of exponentiated exponential densities and can be reversed-J shaped, skewed to the left and to the right. Further, the hazard rate function of the new model can be monotone, unimodal, constant, J-shaped, constant-increasing-decreasing and decreasing-increasing-decreasing and bathtub-shaped. Our main focus is on estimation from a frequentist point of view, yet, some statistical and reliability characteristics for the proposed model are derived. We briefly describe different estimators namely, the maximum likelihood estimators, ordinary least-squares estimators, weighted least-squares estimators, percentile estimators, maximum product of spacings estimators, Cramér-von-Mises minimum distance estimators, Anderson-Darling estimators and right-tail Anderson-Darling estimators. Monte Carlo simulations are performed to compare the performance of the proposed methods of estimation for both small and large samples. We illustrate the performance of the proposed distribution by means of two real data sets and both the data sets show the new distribution is more appropriate as compared to some other well-known distributions.

A Discrete Analog of Inverted Topp-Leone Distribution: Properties, Estimation and Applications

In this study, a discrete inverted Topp-Leone (DITL) distribution is proposed by utilizing the survival discretization approach. The proposed distribution's mathematical features were derived. The maximum likelihood (ML), method of least squares (LS), weighted least squares (WLS), and Cramer Von-Mises (CVM) estimation techniques were used to estimate the parameter. The theoretical results of the ML, LS, WLS, and CVM estimators were demonstrated via a comprehensive simulation study. The proposed DITL distribution has been applied to analyze two count data sets number of deaths due to Covid-19 in Pakistan and India and the findings show the relevance of the proposed distribution.

Dynamic parameter estimation and prediction over consecutive scales, based on moving horizon estimation: applied to an industrial cell culture seed train

Bioprocess modeling has become a useful tool for prediction of the process future with the aim to deduce operating decisions (e.g. transfer or feeds). Due to variabilities, which often occur between and within batches, updating (re-estimation) of model parameters is required at certain time intervals (dynamic parameter estimation) to obtain reliable predictions. This can be challenging in the presence of low sampling frequencies (e.g. every 24 h), different consecutive scales and large measurement errors, as in the case of cell culture seed trains. This contribution presents an iterative learning workflow which generates and incorporates knowledge concerning cell growth during the process by using a moving horizon estimation (MHE) approach for updating of model parameters. This estimation technique is compared to a classical weighted least squares estimation (WLSE) approach in the context of model updating over three consecutive cultivation scales (40–2160 L) of an industrial cell culture seed train. Both techniques were investigated regarding robustness concerning the aforementioned challenges and the required amount of experimental data (estimation horizon). It is shown how the proposed MHE can deal with the aforementioned difficulties by the integration of prior knowledge, even if only data at two sampling points are available, outperforming the classical WLSE approach. This workflow allows to adequately integrate current process behavior into the model and can therefore be a suitable component of a digital twin.

Bounded Weighted Exponential Distribution with Applications

In this article, we try to supplement the distribution theory literature by incorporating a new bounded distribution, called the bounded weighted exponential (BWE) distribution in the (0, 1) intervals by transformation method. The proposed distribution exhibits decreasing and left-skewed unimodal density while the hazard rate can have increasing and bathtub shaped. Although our main focus is on the estimation from the frequentist point of view, in addition, we derive some useful structural and statistical properties of the proposed BWE distribution. We briefly describe three classical estimators namely, the maximum likelihood estimators (MLE), the ordinary least-squares estimators (LSE) and the weighted least-squares estimators (WLSE). Monte Carlo simulations are performed to compare performances of the proposed methods of estimation for both moderate and large samples. An application of the model is presented by re-analyzing a real data set and comparisons are made with the fit attained by some other well-known distributions for illustrative purposes.

Weighted least squares estimation in a binary random coefficient panel model with infinite variance

This article investigates the asymptotic properties of weighted least squares estimators (WLSE) for a binary random coefficient autoregressive (RCA) panel model with heterogeneous variances of panel variables. It is an extension of Johansen and Lange (2013) to a panel model, which is more practical for macroeconomic time series data. We develop asymptotic properties of the WLSE in cases of finite and infinite variances, respectively, as both sizes of panels and samples tend to infinity. In the latter case with infinite variance, the asymptotic for the WLSE βˆ of the coefficient β is shown to be a curious result βˆ⟶pβ−1. It is proven by using the notion of a tail index and the stable distribution limit. In a Monte Carlo simulation, feasible WLSEs are computed iteratively and some evidences are given to verify our theoretical results.

Simultaneous input-state estimation with direct feedthrough based on a unifying MMSE framework with experimental validation

This paper addresses the simultaneous input-state estimation problem for discrete-time linear stochastic systems with unknown input. A unifying minimum-mean-square-error (MMSE) estimation framework is presented for a comprehensive characterization and direct comparison among a few popular input estimation approaches. In this work, the dynamical model of the input is assumed to be unknown and the input is treated as a random variable at each time step. To account for the unknown input dynamics, we propose an input estimator that adopts a white Gaussian input model with a finite covariance, short-named as the FIC (finite input covariance) estimator. Theoretical formulation of the FIC estimator is first compared with two other estimators, one using a Gaussian random walk input model combined with augmented Kalman filter (AKF) and another one using a deterministic input model with weighted least squares (WLS) estimation. Based on the unifying MMSE framework presented in this paper, it is proved that when the input covariance of the FIC estimator approaches infinity and the feedthrough matrix has full-column rank, the estimator is equivalent to the well-known WLS estimator.The FIC estimator is validated and compared with the AKF and WLS estimator using simulated measurements from 2-story shear structure and experimental measurements from a full-scale concrete frame. The 2-story shear structure is excited by two different types of input and the corresponding acceleration responses are used to compare estimator performance. With only acceleration measurements, the FIC estimator eliminates a low-frequency drift error in the estimated input and states with a tight estimation confidence interval. Detailed discussion on the effect of estimator covariances on input estimation is also provided. A priori knowledge of the statistical property of the unknown input can provide insights in the tuning of FIC estimator covariances. In addition, field acceleration measurements from a full-scale concrete frame under shaker excitation are used to compare the estimation results and validate the proposed FIC estimator. The FIC estimator is shown to provide better estimates with tighter estimation confidence interval in comparison to the AKF and WLS estimators.

Multivariate approximation of functions on irregular domains by weighted least-squares methods

AbstractWe propose and analyse numerical algorithms based on weighted least squares for the approximation of a bounded real-valued function on a general bounded domain $\varOmega \subset \mathbb{R}^d$. Given any $n$-dimensional approximation space $V_n \subset L^2(\varOmega )$, the analysis in Cohen and Migliorati (2017, Optimal weighted least-squares methods. SMAI J. Comput. Math., 3, 181–203) shows the existence of stable and optimally converging weighted least-squares estimators, using a number of function evaluations $m$ of the order $n \ln n$. When an $L^2(\varOmega )$-orthonormal basis of $V_n$ is available in analytic form, such estimators can be constructed using the algorithms described in Cohen and Migliorati (2017, Optimal weighted least-squares methods. SMAI J. Comput. Math., 3, 181–203, Section 5). If the basis also has product form, then these algorithms have computational complexity linear in $d$ and $m$. In this paper we show that when $\varOmega $ is an irregular domain such that the analytic form of an $L^2(\varOmega )$-orthonormal basis is not available, stable and quasi-optimally weighted least-squares estimators can still be constructed from $V_n$, again with $m$ of the order $n \ln n$, but using a suitable surrogate basis of $V_n$ orthonormal in a discrete sense. The computational cost for the calculation of the surrogate basis depends on the Christoffel function of $\varOmega $ and $V_n$. Numerical results validating our analysis are presented.

A General Framework for Nonparametric Identification of Nonlinear Stochastic Systems

In this article, nonparametric identification of nonlinear autoregressive systems with exogenous inputs (NARX) is considered; a general criterion function is introduced for estimating the value of the nonlinear function within the system at any fixed point. The criterion function is constructed using a kernel together with a convex objective function. Not only does this framework include the classical kernel-based weighted least-squares estimator but also the kernel-based L <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">l</sub> ,l ≥ 1 criteria as special cases. First, we prove that the minimizer of the general criterion function converges to the true function value with probability 1. Second, recursive algorithms are proposed to find the estimates, which minimize the criterion function, and it is shown that these estimates also converge to the true function value with probability 1. Numerical examples are given, justifying that the framework guarantees the strong consistency of the estimates and exhibits the robustness against outliers in the observations.

TDOA Source-Localization Technique Robust to Time-Synchronization Attacks

In this paper, we focus on the localization of a passive source from time difference of arrival (TDOA) measurements. TDOA values are computed with respect to pairs of fixed sensors that are required to be accurately time-synchronized. This constitutes a weakness as all synchronization techniques are vulnerable to delay injections. Attackers are able either to spoof the signal or to inject asymmetric delays in the communication channel. By nature, TDOA measurements are highly sensitive to time-synchronization offsets between sensors. Our first contribution is to show that timing attacks can severely affect the localization process. With a delay of a few microseconds injected on one sensor, the resulting estimate might be several kilometers away from the true location of the unknown source. We also show that residual analysis does not enable the detection and identification of timing attacks. Our second contribution is to propose a two-step TDOA-localization technique that is robust against timing attacks. It uses a known source to define a weight for each pair of sensors, reflecting the confidence in their time synchronization. Our solution then uses the weighted least-squares estimator with the newly created weights and the TDOA measurements received from the unknown source. As a result, our method either identifies the network as being too corrupt to localize, or gives a corrected estimate of the unknown position along with a confidence metric. Numerical results illustrate the performance of our technique.

3D target localization based on multi–unmanned aerial vehicle cooperation

Ground target three-dimensional positions measured from optical remote-sensing images taken by an unmanned aerial vehicle play an important role in related military and civil applications. The weakness of this system lies in its localization accuracy being unstable and its efficiency being low when using a single unmanned aerial vehicle. In this paper, a novel multi–unmanned aerial vehicle cooperative target localization measurement method is proposed to overcome these issues. In the target localization measurement stage, three or more unmanned aerial vehicles simultaneously observe the same ground target and acquire multiple remote-sensing images. According to the principle of perspective projection, the target point, its image point, and the camera’s optic center are collinear, and nonlinear observation equations are established. These equations are then converted to linear equations using a Taylor expansion. Robust weighted least-squares estimation is used to solve the equations with the objective function of minimizing the weighted square sum of re-projection errors from target points to multiple pairs of images, which can make the best use of the effective information and avoid interference from the observation data. An automatic calculation strategy using a weight matrix is designed, and the weight matrix and target-position coordinate value are updated in each iteration until the iteration stopping condition is satisfied. Compared with the stereo-image-pair cross-target localization method, the multi–unmanned aerial vehicle cooperative target localization method can use more observation information, which results in higher rendezvous accuracy and improved performance. Finally, the effectiveness and robustness of this method is verified by numerical simulation and flight testing. The results show that the proposed method can effectively improve the precision of the target’s localization and demonstrates great potential for providing more accurate target localization in engineering applications.

Estimation of Autoregressive Parameters from Noisy Observations Using Iterated Covariance Updates.

Estimating the parameters of the autoregressive (AR) random process is a problem that has been well-studied. In many applications, only noisy measurements of AR process are available. The effect of the additive noise is that the system can be modeled as an AR model with colored noise, even when the measurement noise is white, where the correlation matrix depends on the AR parameters. Because of the correlation, it is expedient to compute using multiple stacked observations. Performing a weighted least-squares estimation of the AR parameters using an inverse covariance weighting can provide significantly better parameter estimates, with improvement increasing with the stack depth. The estimation algorithm is essentially a vector RLS adaptive filter, with time-varying covariance matrix. Different ways of estimating the unknown covariance are presented, as well as a method to estimate the variances of the AR and observation noise. The notation is extended to vector autoregressive (VAR) processes. Simulation results demonstrate performance improvements in coefficient error and in spectrum estimation.

Level shift estimation in the presence of non-stationary volatility with an application to the unit root testing problem

This paper focuses on the estimation of the location of level breaks in time series whose shocks display non-stationary volatility (permanent changes in unconditional volatility). We propose a new feasible weighted least squares (WLS) estimator, based on an adaptive estimate of the volatility path of the shocks. We show that this estimator belongs to a generic class of weighted residual sum of squares which also contains the ordinary least squares (OLS) and WLS estimators, the latter based on the true volatility process. For fixed magnitude breaks we show that the consistency rate of the generic estimator is unaffected by non-stationary volatility. We also provide local limiting distribution theory for cases where the break magnitude is either local-to-zero at some polynomial rate in the sample size or is exactly zero. The former includes the Pitman drift rate which is shown via Monte Carlo experiments to predict well the key features of the finite sample behaviour of both the OLS and our feasible WLS estimators. The simulations highlight the importance of the break location, break magnitude, and the form of non-stationary volatility for the finite sample performance of these estimators, and show that our proposed feasible WLS estimator can deliver significant improvements over the OLS estimator under heteroskedasticity. We discuss how these results can be applied, by using level break fraction estimators on the first differences of the data, when testing for a unit root in the presence of trend breaks and/or non-stationary volatility. Methods to select between the break and no break cases, using standard information criteria and feasible weighted information criteria based on our adaptive volatility estimator, are also discussed. Simulation evidence suggests that unit root tests based on these weighted quantities can display significantly improved finite sample behaviour under heteroskedasticity relative to their unweighted counterparts. An empirical illustration to U.S. and U.K. real GDP is also considered.

Estimation of causal continuous‐time autoregressive moving average random fields

AbstractWe estimate model parameters of Lévy‐driven causal continuous‐time autoregressive moving average random fields by fitting the empirical variogram to the theoretical counterpart using a weighted least squares (WLS) approach. Subsequent to deriving asymptotic results for the variogram estimator, we show strong consistency and asymptotic normality of the parameter estimator. Furthermore, we conduct a simulation study to assess the quality of the WLS estimator for finite samples. For the simulation, we utilize numerical approximation schemes based on truncation and discretization of stochastic integrals and we analyze the associated simulation errors in detail. Finally, we apply our results to real data of the cosmic microwave background.

Data-Driven Online System Equivalent for Self-Adaptive Droop Voltage Control of Wind Power Plants

This letter considers the closed-loop stability issue of droop-based voltage control for wind power plants (WPPs) under varying grid strength. We first propose a data-driven online system equivalent/identification method (DOnSE) for WPPs using the weighted least-squares estimation (WLSE). Then, upon the analysis of link between stability boundary of droop gain and external grid strength, a self-adaptive gain-scheduling droop voltage control scheme for WPPs based on the DOnSE is developed to enhance the closed-loop stability and reactive power utilization of wind turbines (WTs).