Generalized Maximum Likelihood Estimator Research Articles

Stochastic models for time complexity of computing tasks: I. Development principles, statistical data mining, identification problems

The paper contains the first part of an investigation devoted to the design of the mathematical models for the execution time of user tasks carried out on the virtual calculating nodes. We suppose that the execution time is a random value with the mean and variance depending on the node resources, task parameters, and the current characteristics of the node state. We discover the key features of the mean and variance functions and specify some of their particular cases. Both the mean and variance functions depend on the unknown parameters, and the design of the stochastic model for the time complexity leads to the parameter identification in the form of the generalized maximum likelihood estimates under the heterogeneous statistical information. The paper also contains recommendations concerning the gathering and subsequent usage of this information: the node testbed preparation, stress test planning, and the obtained data processing. The specific illustrating examples of the proposed mathematical model will be presented in the subsequent parts of the investigation.

A Robust Data-Driven Process Modeling Applied to Time-Series Stochastic Power Flow

In this paper, we propose a robust data-driven process model whose hyperparameters are robustly estimated using the Schweppe-type generalized maximum likelihood estimator. The proposed model is trained on recorded time-series data of voltage phasors and power injections to perform a time-series stochastic power flow calculation. Power system data are often corrupted with outliers caused by large errors, fault conditions, power outages, and extreme weather, to name a few. The proposed model downweights vertical outliers and bad leverage points in the measurements of the training dataset. The weights used to bound the influence of the outliers are calculated using projection statistics, which are a robust version of Mahalanobis distances of the time series data points. The proposed method is demonstrated on the IEEE 33-Bus power distribution system and a real-world unbalanced 240-bus power distribution system heavily integrated with renewable energy sources. Our simulation results show that the proposed robust model can handle up to <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$25\%$</tex-math></inline-formula> of outliers in the training data set.

Mixed Skewness Probability Modeling and Extreme Value Predicting for Physical System Input–Output Based on Full Bayesian Generalized Maximum-Likelihood Estimation

Mixed Skewness Probability Modeling and Extreme Value Predicting for Physical System Input–Output Based on Full Bayesian Generalized Maximum-Likelihood Estimation

The Robustness of the Derived Design Life Levels of Heavy Precipitation Events in the Pre-Alpine Oberland Region of Southern Germany

Extreme value analysis (EVA) is well-established to derive hydrometeorological design values for infrastructures that have to withstand extreme events. Since there is concern about increased extremes with higher hazard potential under climate change, alterations of EVA are introduced for which statistical properties are no longer assumed to be constant but vary over time. In this study, both stationary and non-stationary EVA models are used to derive design life levels (DLLs) of daily precipitation in the pre-alpine Oberland region of Southern Germany, an orographically complex region characterized by heavy precipitation events and climate change. As EVA is fraught with uncertainties, it is crucial to quantify its methodological impacts: two theoretical distributions (i.e., Generalized Extreme Value (GEV) and Generalized Pareto (GP) distribution), four different parameter estimation techniques (i.e., Maximum Likelihood Estimation (MLE), L-moments, Generalized Maximum Likelihood Estimation (GMLE), and Bayesian estimation method) are evaluated and compared. The study reveals large methodological uncertainties. Discrepancies due to the parameter estimation methods may reach up to 45% of the mean absolute value, while differences between stationary and non-stationary models are of the same magnitude (differences in DLLs up to 40%). For the end of this century in the Oberland region, there is no robust tendency towards increased extremes found.

Artificial Neural Networks – Based Method for Enhancing State Estimation of Grids with High Penetration of Renewables

This paper addresses state estimation as one of the most essential mechanisms in real-time operation and control of modern power systems, and proposes a novel solution to the issue of poor network observability, commonly faced in distribution system state estimation (DSSE) characterized by an ever-increasing penetration of renewable generation. The ongoing transformation from conventional passive, onedirectional power systems to active smart grids necessitates more accurate and reliable system state estimation to achieve optimal system performance. Real-time grid monitoring and control has been a routine task in transmission networks, but distribution grids cannot successfully utilize these capabilities due to different topologies, specific electrical characteristics, the low amount of available real-time measurements, as well as substantial communication effort needed to handle the data. Furthermore, with the advent of distributed generation, new types of loads and the vast surge of prosumers, a substantial amount of data is required to maintain system stability and controllability. For these reasons, reliable state estimation requires a high-quality creation process of pseudo-measurement, in addition to an efficient algorithm and an extremely accurate estimator. Thus, this paper proposes a novel framework of dynamic estimation methodology that includes the use of Artificial Neural Networks (ANN) in the pseudo-measurements generation process, utilizes Iteratively Reweighted Least Squares (IRWLS) algorithm and Schweppe-Huber Generalized Maximum Likelihood (SHGM) estimator. The efficiency and accuracy of the proposed methodology were assessed and verified on a benchmark network model.

Generalized maximum likelihood estimation of the mean of parameters of mixtures. With applications to sampling and to observational studies

Let f(y∣θ),θ∈Ω be a parametric family, η(θ) a given function, and G an unknown mixing distribution. It is desired to estimate EG(η(θ))≡ηG based on independent observations Y1,...,Yn, where Yi∼f(y∣θi), and θi∼G are iid. We explore the Generalized Maximum Likelihood Estimators (GMLE) for this problem. Some basic properties and representations of those estimators are shown. In particular we suggest a new perspective, of the weak convergence result by [14], with implications to a corresponding setup in which θ1,...,θn are fixed parameters. We also relate the above problem, of estimating ηG, to nonparametric empirical Bayes estimation under a squared loss. Applications of GMLE to sampling problems are presented. The performance of the GMLE is demonstrated both in simulations and through a real data example.

Meta-analytic Gaussian Network Aggregation

A growing number of publications focus on estimating Gaussian graphical models (GGM, networks of partial correlation coefficients). At the same time, generalizibility and replicability of these highly parameterized models are debated, and sample sizes typically found in datasets may not be sufficient for estimating the underlying network structure. In addition, while recent work emerged that aims to compare networks based on different samples, these studies do not take potential cross-study heterogeneity into account. To this end, this paper introduces methods for estimating GGMs by aggregating over multiple datasets. We first introduce a general maximum likelihood estimation modeling framework in which all discussed models are embedded. This modeling framework is subsequently used to introduce meta-analytic Gaussian network aggregation (MAGNA). We discuss two variants: fixed-effects MAGNA, in which heterogeneity across studies is not taken into account, and random-effects MAGNA, which models sample correlations and takes heterogeneity into account. We assess the performance of MAGNA in large-scale simulation studies. Finally, we exemplify the method using four datasets of post-traumatic stress disorder (PTSD) symptoms, and summarize findings from a larger meta-analysis of PTSD symptom.

Joint Estimation of Target Parameters and System Deviations in MIMO Radar With Widely Separated Antennas on Moving Platforms

A multiple-input multiple-output (MIMO) radarwith widely separated antennas on moving platforms suffers from the effects of platform deviations on the target parameter estimation since the trajectories of the moving platforms are sensitive to environmental factors, such as strong wind. This article addresses the joint estimation of multiple target positions and velocities as well as radar system deviations to minimize the impact of platform deviations. The proposed algorithm can also be regarded as a self-calibration technique. First, the grid search dimensions for parameter estimation are reduced via a generalized maximum likelihood (GML) algorithm. Second, the adaptive gradient (AdaGrad) method is used to implement the GML estimation for multitarget echo delays and Doppler shifts. Finally, to address the nonlinear estimation problem of interest, the iterative least squares method is used to estimate the multiple target positions and velocities as well as radar system deviations based on the estimated delays and Doppler shifts. Prior information, such as target positions or the probability distribution of the echo coefficient, is not needed in the proposed method. The parameter identifiability is also derived in this article. Numerical simulations show that compared to a method without the estimation of system deviations, the proposed method is more efficient with respect to the derived Cramér–Rao bound.

Identifiability and estimation of recursive max‐linear models

AbstractWe address the identifiability and estimation of recursive max‐linear structural equation models represented by an edge‐weighted directed acyclic graph (DAG). Such models are generally unidentifiable and we identify the whole class of DAG s and edge weights corresponding to a given observational distribution. For estimation, standard likelihood theory cannot be applied because the corresponding families of distributions are not dominated. Given the underlying DAG, we present an estimator for the class of edge weights and show that it can be considered a generalized maximum likelihood estimator. In addition, we develop a simple method for identifying the structure of the DAG. With probability tending to one at an exponential rate with the number of observations, this method correctly identifies the class of DAGs and, similarly, exactly identifies the possible edge weights.

Damage identification in plates under uncertain boundary conditions

Nondestructive damage identification is a central task in industrial applications in, for example, aeronautical, civil and naval engineering. The identification approaches based on (physical) models rely on the accuracy of the model predictions, and typically suffer from effects caused by ubiquitous modeling errors and uncertainties. The identification problems are typically unstable inverse problems but may also be (apparently) stable when considering low dimensional parametrizations for the defects. The present paper considers the identification of defects in plates under uncertain boundary conditions. To partially compensate for the boundary uncertainties, we adopt the Bayesian framework for inverse problems and use approximate marginalization over the model uncertainties: the process of marginalization propagates the model uncertainties to the uncertainties in the estimated parameters. We employ a low dimensional parametrization for the defects, which would, in principle, allow for least squares or the more general maximum likelihood estimation approaches. We will show, however, that under uncertain boundary conditions, maximum likelihood estimates can yield unfeasible estimates while the approximate marginalization over uncertainties provides feasible ones.

On general maximum likelihood empirical Bayes estimation of heteroscedastic IID normal means

We propose a general maximum likelihood empirical Bayes (GMLEB) method for the heteroscedastic normal means estimation with known variances. The idea is to plug the generalized maximum likelihood estimator in the oracle Bayes rule. From the point of view of restricted empirical Bayes, the general empirical Bayes aims at a benchmark risk smaller than the linear empirical Bayes methods when the unknown means are i.i.d. variables. We prove an oracle inequality which states that under mild conditions, the regret of the GMLEB is of smaller order than $(\log n)^{5}/n$. The proof is based on a large deviation inequality for the generalized maximum likelihood estimator. The oracle inequality leads to the property that the GMLEB is adaptive minimax in $L_{p}$ balls when the order of the norm of the ball is larger than $((\log n)^{5/2}/\sqrt{n})^{1/(p\wedge 2)}$. We demonstrate the superb risk performance of the GMLEB through simulation experiments.

PARAMETER ESTIMATION FOR COMPLICATED NOISE ENVIRONMENT

For a complicated noise environment the use of M-estimator faces a problem of choosing a cost function yielding the best solution. To solve this problem it is proposed to use a superset of cost functions. The superset capabilities provide constructing a parameter estimation method for complicated noise environment. It consists in tuning the generalized maximum likelihood estimation to the current noise environment by setting values of three free superset parameters related to the scale, the tail heaviness and the form of noise distribution, as well as to the anomaly values that presence in data. In general case, this method requires to solve the optimization problem with a non-unimodal objective function, and it can be mostly implemented by using the zero-order optimization methods. However, if the noise environment has known statistics, the proposed method leads to the optimal estimation. If the noise environment is complicated or does not have a complete statistics, the proposed method leads to the more effective estimates comparing to those of mean, median, myriad and meridian estimators. Numerical simulations confirmed the method performance.

Robust Unscented Kalman Filter for Power System Dynamic State Estimation With Unknown Noise Statistics

Due to the communication channel noises, GPS synchronization process, changing environment temperature and different operating conditions of the system, the statistics of the system process and measurement noises may be unknown and they may not follow Gaussian distributions. As a result, the traditional Kalman filter-based dynamic state estimators may provide strongly biased state estimates. To address these issues, this paper develops a robust generalized maximum-likelihood unscented Kalman filter (GM-UKF). The statistical linearization approach is presented to derive a compact batch-mode regression form by processing the predicted state vector and the received measurements simultaneously. This regression form enhances the data redundancy and allows us to detect bad phasor measurement unit measurements and incorrect state predictions, and filter out unknown Gaussian and non-Gaussian noises through the generalized maximum likelihood-estimator. The latter minimizes a convex Huber function with weights calculated via the projection statistics (PS). Particularly, the PS is applied to a proposed 2-dimensional matrix that consists of temporally correlated innovation vectors and predicted states. Finally, the total influence function is used to derive the error covariance matrix of the GM-UKF state estimates, yielding the robust state prediction at the next time instant. Extensive simulations carried out on the IEEE 39-bus test system demonstrate the effectiveness and robustness of the proposed method.

Robust M–M unscented Kalman filtering for GPS/IMU navigation

In this paper, a robust unscented Kalman filter (UKF) based on the generalized maximum likelihood estimation (M-estimation) is proposed to improve the robustness of the integrated navigation system of Global Navigation Satellite System and Inertial Measurement Unit. The UKF is a variation of Kalman filter by which the Jacobian matrix calculation in a nonlinear system state model is not necessary. The proposed robust M–M unscented Kalman filter (RMUKF) applies the M-estimation principle to both functional model errors and measurement errors. Hence, this robust filter attenuates the influences of disturbances in the dynamic model and of measurement outliers without linearizing the nonlinear state space model. In addition, an equivalent weight matrix, composed of the bi-factor shrink elements, is proposed in order to keep the original correlation coefficients of the predicted state unchanged. Furthermore, a nonlinear error model is used as the dynamic equation to verify the performance of the proposed RMUKF with a simulation and field test. Compared with the conventional UKF, the impacts of measurement outliers and system disturbances on the state estimation are both controlled by RMUKF.

Penalized maximum likelihood estimators for the nonstationary Pearson type 3 distribution

The climate change and human activities can principally doubt the frequency analysis methods which assume that the hydrological events are stationary. The Pearson type 3 (P3) distribution, which has been widely applied in frequency analysis under the stationary condition, was developed for the nonstationary cases. The distribution parameters can be expressed as exponential or polynomial functions with time or other covariates. Parameter estimation for the nonstationary P3 model was done with the penalized maximum likelihood estimation (PMLE) method in which we applied a restrictive penalty on the location parameter to avoid unreasonable results. The results of the nonstationary P3 model were compared to that of the generalized extreme value (GEV) distribution, in which the generalized maximum likelihood estimation (GMLE) method was applied to estimate the parameters. The Monte Carlo simulations indicated that the nonstationary P3 model provided better results than the nonstationary GEV model for different sample sizes. Both models were also applied in hydrological series with various trends and record lengths to prove the feasibility of the nonstationary P3 model in practical applications. The applications showed that the nonstationary P3 model performed better than the nonstationary GEV model in most of the studied cases.

A novel pseudo‐maximum likelihood estimation method for accelerated degradation tests under Wiener process

AbstractFor reliability assessment based on accelerated degradation tests (ADTs), an appropriate parameter estimation method is very important because it affects the extrapolation and prediction accuracy. The well‐adopted maximum likelihood estimation (MLE) method focuses on interpolation fitting and obtains results via maximizing the likelihood of the observations. However, a best interpolation fitting does not necessarily yield a best extrapolation. In this paper, therefore, a pseudo‐MLE (P‐MLE) method is proposed to improve the prediction accuracy of constant‐stress ADTs by considering the degradation mechanism equivalence under Wiener process. In particular, the degradation mechanism equivalence is characterized by a mechanism equivalence factor which presents the proportional relationship between degradation rate and variation. Then, the mechanism equivalence factor is determined via a two‐step method. The other model parameters can be estimated by the general MLE method. The asymptotic variances of acceleration factors and the p‐quantile of product failure time under normal condition are adopted to compare the statistical properties of the proposed method and the general MLE approach. Numerical examples show that the novel P‐MLE method may not achieve a maximum likelihood but can provide more benefits regarding prediction accuracy enhancement especially when the sample size is limited.

A hyperbolic model of optimal cash balances

ABSTRACTWe develop a hyperbolic cash management model based on the Pearson Type IV probability density which minimises extreme variations in firm cash balances. Since the moments for the Type IV probability density are in general undefined and maximum likelihood estimation is compromised by the non-algebraic nature of the Type IV normalising constant, parameter estimation is implemented using the minimum method. Empirical analysis shows that the Type IV density is highly compatible with the quarterly cash flow data of a randomly selected sample of 100 large U.S. corporations. In contrast, around 60% of the 100 corporations return Jarque–Bera test statistics which are incompatible with the Gaussian probability density.

Best Linear Unbiased Prediction for Multifidelity Computer Experiments

Recently it becomes a growing trend to study complex systems which contain multiple computer codes with different levels of accuracy, and a number of hierarchical Gaussian process models are proposed to handle such multiple-fidelity codes. This paper derives the best linear unbiased prediction for three popular classes of multiple-level Gaussian process models. The predictors all have explicit expressions at each untried point. Empirical best linear unbiased predictors are also provided by plug-in methods with generalized maximum likelihood estimators of unknown parameters.

Simultaneous estimation based on empirical likelihood and general maximum likelihood estimation

One typical problem in simultaneous estimation of mean values is estimating means of normal distributions, however when normality or any other distribution is not specified, more robust estimation procedures are demanded. A new estimation procedure is proposed based on empirical likelihood which does not request any specific distributional assumption. The new idea is based on incorporating empirical likelihood with general maximum likelihood estimation. One well-known nonparametric estimator, the linear empirical Bayes estimator, can be interpreted as an estimator based on empirical likelihood under some framework and it is shown that the proposed procedure can improve the linear empirical Bayes estimator. Numerical studies are presented to compare the proposed estimator with some existing estimators. The proposed estimator is applied to the problem of estimating mean values corresponding to high valued observations. Simulations and real data example of gene expression are provided.

Empirical performance of Gaussian affine dynamic term structure models in the presence of autocorrelation misspecification bias

Recently, several authors have documented the presence of estimation bias in Gaussian affine dynamic term structure models (GADTSM). However, only a few applications involving its impact on the empirical performance of GADTSM exist in the extant literature, and these studies focus solely on discrete-time vector autoregressive (VAR) based GADTSM and concentrate on issues of small-sample bias and persistence. In this paper, we provide a comprehensive investigation of this issue that includes the estimation of both discrete-time VAR based GADTSM and continuous-time GADTSM at multiple data frequencies through a unique empirical design and two Monte Carlo simulation experiments, within which we construct estimation bias from the serial correlation in yield pricing errors. Our findings show that, although, empirical performance of all studied GADTSM are severely impacted by estimation bias, discrete-time GADTSM are more severely impacted by estimation bias than continuous-time GADTSM. Building on theoretical arguments developed in previous works, we attribute this finding to the strong dependence of discrete-time VAR based GADTSM on the ordinary least squares econometric technique relative to the continuous-time GADTSM for which general maximum likelihood estimation is more suitable.