Classical Maximum Likelihood Estimation Research Articles

Homogeneity/Heterogeneity Hypotheses for Standardized Mortality Ratios Based on Minimum Power‐divergence Estimators

This paper analyzes the power divergence estimators when homogeneity/heterogeneity hypotheses among standardized mortality ratios (SMRs) are taken into account. A Monte Carlo study shows that when the standard mortality rate is not external, that is it is estimated from the sample data, these estimators have a good performance even for small sample sets and in particular the minimum chi-square estimators have a better behavior compared to the classical maximum likelihood estimators. In order to make decisions under homogeneity/heterogeneity hypotheses of SMRs we propose some test-statistics which consider the minimum power divergence estimators. Through a numerical example focused on SMRs of melanoma mortality ratios in different regions of the US, a homogeneity/heterogeneity study is illustrated.

Piecewise Linear Models with Guaranteed Closeness to the Data

This paper addresses the problem of piecewise linear approximation of point sets without any constraints on the order of data points or the number of model components (line segments). We point out two problems with the maximum likelihood estimate (MLE) that present serious drawbacks in practical applications. One is that the parametric models obtained using a classical MLE framework are not guaranteed to be close to data points. It is typically impossible, in this classical framework, to detect whether a parametric model fits the data well or not. The second problem is related to accurately choosing the optimal number of model components. We first fit a nonparametric density to the data points and use it to define a neighborhood of the data. Observations inside this neighborhood are deemed informative; those outside the neighborhood are deemed uninformative for our purpose. This provides us with a means to recognize when models fail to properly fit the data. We then obtain maximum likelihood estimates by optimizing the Kullback-Leibler Divergence (KLD) between the nonparametric data density restricted to this neighborhood and a mixture of parametric models. We prove that, under the assumption of a reasonably large sample size, the inferred model components are close to their ground-truth model component counterparts. This holds independently of the initial number of assumed model components or their associated parameters. Moreover, in the proposed approach, we are able to estimate the number of significant model components without any additional computation.

Fault Tolerant Maximum Likelihood Event Localization in Sensor Networks Using Binary Data

This paper investigates Wireless Sensor Networks (WSNs) for achieving fault tolerant localization of an event using only binary information from the sensor nodes. In this context, faults occur due to various reasons and are manifested when a node outputs a wrong decision. The main contribution of this paper is to propose the Fault Tolerant Maximum Likelihood (FTML) estimator. FTML is compared against the Centroid (CE) and the classical maximum likelihood (ML) estimators and is shown to be significantly more fault tolerant. Moreover, this paper compares FTML against the SNAP (Subtract on Negative Add on Positive) algorithm and shows that in the presence of faults the two can achieve similar performance; FTML is slightly more accurate while SNAP is computationally less demanding and requires fewer parameters.

Shrinkage drift parameter estimation for multi‐factor Ornstein–Uhlenbeck processes

AbstractWe consider some inference problems concerning the drift parameters of multi‐factors Vasicek model (or multivariate Ornstein–Uhlebeck process). For example, in modeling for interest rates, the Vasicek model asserts that the term structure of interest rate is not just a single process, but rather a superposition of several analogous processes. This motivates us to develop an improved estimation theory for the drift parameters when homogeneity of several parameters may hold. However, the information regarding the equality of these parameters may be imprecise. In this context, we consider Stein‐rule (or shrinkage) estimators that allow us to improve on the performance of the classical maximum likelihood estimator (MLE). Under an asymptotic distributional quadratic risk criterion, their relative dominance is explored and assessed. We illustrate the suggested methods by analyzing interbank interest rates of three European countries. Further, a simulation study illustrates the behavior of the suggested method for observation periods of small and moderate lengths of time. Our analytical and simulation results demonstrate that shrinkage estimators (SEs) provide excellent estimation accuracy and outperform the MLE uniformly. An over‐ridding theme of this paper is that the SEs provide powerful extensions of their classical counterparts. Copyright © 2009 John Wiley & Sons, Ltd.

Covariance Estimation in Decomposable Gaussian Graphical Models

Graphical models are a framework for representing and exploiting prior conditional independence structures within distributions using graphs. In the Gaussian case, these models are directly related to the sparsity of the inverse covariance (concentration) matrix and allow for improved covariance estimation with lower computational complexity. We consider concentration estimation with the mean-squared error (MSE) as the objective, in a special type of model known as decomposable. This model includes, for example, the well known banded structure and other cases encountered in practice. Our first contribution is the derivation and analysis of the minimum variance unbiased estimator (MVUE) in decomposable graphical models. We provide a simple closed form solution to the MVUE and compare it with the classical maximum likelihood estimator (MLE) in terms of performance and complexity. Next, we extend the celebrated Stein's unbiased risk estimate (SURE) to graphical models. Using SURE, we prove that the MSE of the MVUE is always smaller or equal to that of the biased MLE, and that the MVUE itself is dominated by other approaches. In addition, we propose the use of SURE as a constructive mechanism for deriving new covariance estimators. Similarly to the classical MLE, all of our proposed estimators have simple closed form solutions but result in a significant reduction in MSE.

Bayesian models identify specific lymphoproliferative disorders associated with hepatitis C virus infection

Bayesian models identify specific lymphoproliferative disorders associated with hepatitis C virus infection

Public support for reducing US reliance on fossil fuels: Investigating household willingness-to-pay for energy research and development

In order to reduce future dependence on foreign oil and emissions of CO 2, how much would US households be willing to pay annually to support increased energy research and development (R&D) activities designed to replace fossil fuels? Does it matter whether the R&D includes nuclear energy options? We explore these questions using data from a unique set of national telephone and Internet surveys. Using a national advisory referendum format, the contingent valuation method is applied to estimate annual household willingness-to-pay (WTP) for US household support of a national Energy Research and Development Fund (ERDF) for investments in energy sources not reliant on fossil fuels. While accounting for the level of (un)certainty in voting responses, the WTP modeling includes a comparison of both classic maximum likelihood estimation (MLE) and Bayesian analysis. Evidence indicates that MLE and Bayesian analysis achieve similar statistical inference, while the Bayesian analysis provides a narrower confidence interval around estimated WTP.

Robust methods for generalized linear models with nonignorable missing covariates

AbstractThe EM algorithm is often used for finding the maximum likelihood estimates in generalized linear models with incomplete data. In this article, the author presents a robust method in the framework of the maximum likelihood estimation for fitting generalized linear models when nonignorable covariates are missing. His robust approach is useful for downweighting any influential observations when estimating the model parameters. To avoid computational problems involving irreducibly high‐dimensional integrals, he adopts a Metropolis‐Hastings algorithm based on a Markov chain sampling method. He carries out simulations to investigate the behaviour of the robust estimates in the presence of outliers and missing covariates; furthermore, he compares these estimates to the classical maximum likelihood estimates. Finally, he illustrates his approach using data on the occurrence of delirium in patients operated on for abdominal aortic aneurysm.

The Bootstrap Maximum Likelihood Estimator: the case of logit

The estimation of the parameters of logit model is mostly performed with method of maximum likelihood. However, the classical maximum likelihood estimators are biased and inefficient in appearance of small samples. The jackknife maximum likelihood estimator improves the above problems but still includes serious disadvantages. In this article, the Bootstrap Maximum Likelihood Estimator is developed as an alternative advanced method for reducing the bias and correcting the troubles with inefficiency and nonnormality. The importance of the method is shown through its application on data of Greek mergers and acquisitions.

Identification of Greek Takeover Targets and Coherent Policy Implications

AbstractThe main purpose of this paper is the identification of the characteristics of takeover targets in a small open economy like Greece, using the market for corporate control (MCC) hypothesis as theoretical background. Contrary to this hypothesis, the results indicate that the motives for merging or acquisition activities are basically of strategic character. Using a sample of 35 acquired and 105 non‐acquired firms, the sampling process was initially performed by a modified methodology of state‐based sampling, even if its nature cannot be recognized by the classical maximum likelihood estimator (MLE) of logit model. Subsequently, taking into account the small size of the sample, we develop and use, in the logit context, the bootstrap MLE as an advanced alternative method for reducing inherit bias and inefficiency. The findings remain uniform supporting the strategic motives explanation in actual takeover activities, a fact that clearly illustrates the framework of merger policies followed by the Greek Competition Committee during the period under study.

Bayesian Updating and Model Class Selection for Hysteretic Structural Models Using Stochastic Simulation

System identification of structures using their measured earthquake response can play a key role in structural health monitoring, structural control and improving performance-based design. Implementation using data from strong seismic shaking is complicated by the nonlinear hysteretic response of structures. Furthermore, this inverse problem is ill-conditioned; for example, even if some components in the structure show substantial yielding, others will exhibit nearly elastic response, producing no information about their yielding behavior. Classical least-squares or maximum likelihood estimation will not work with a realistic class of hysteretic models because it will be unidentifiable based on the data. It is shown here that Bayesian updating and model class selection provide a powerful and rigorous approach to tackle this problem when implemented using a recently developed stochastic simulation algorithm called Transitional Markov Chain Monte Carlo. The updating and model class selection is performed on a previously-developed class of Masing hysteretic structural models that are relatively simple yet can give realistic responses to seismic loading. The theory for the Masing hysteretic models, and the theory used to perform the updating and model class selection, are presented and discussed. An illustrative example is given that uses simulated dynamic response data and shows the ability of the algorithm to identify hysteretic systems even when the class of models is unidentifiable based on the data.

Robust Fits for Copula Models

In this article, we obtain robust estimators for copula parameters through the minimization of weighted goodness-of-fit statistics. Different weight functions emphasize different regions on the unit square and are able to handle different locations of model violation. The resulting WMDE estimators are compared to the classical maximum likelihood estimators MLE, and to their weighted version WMLE, an estimator obtained in two steps. The weights obtained in the first step by the application of a high breakdown point scatter matrix estimator are used to identify atypical points. All estimators are compared in a comprehensive simulation study. For each ε-contaminated parametric copula family considered, we showed that there is a robust estimator improving over the MLE and able to capture the correct strength of dependence of the data, despite the contamination percentual and location, and the sample size.

Maximum trimmed likelihood estimator for multivariate mixed continuous and categorical data

In this article, we apply the maximum trimmed likelihood (MTL) approach [Hadi, A.S., Luceño, A., 1997. Maximum trimmed likelihood estimators: a unified approach, examples, and algorithms. Comput. Statist. Data Anal. 25, 251–272] to obtain the robust estimators of multivariate location and shape, especially for data mixed with continuous and categorical variables. The forward search algorithm [Atkinson, A.C., 1994. Fast very robust methods for the detection of multiple outliers. J. Amer. Statist. Assoc. 89, 1329–1339] is adapted to compute the proposed MTL estimates. A simulation study shows that the proposed estimator outperforms the classical maximum likelihood estimator when outliers exist in data. Real data sets are also used to illustrate the method and results of the detection of the outliers.

New family of estimators for the loglinear model of quasi-independence based on power-divergence measures

In this paper, we study the minimum power-divergence estimator, introduced and studied by Cressie and Read [Cressie, N. and Read, T.R.C., 1984, Multinomial goodness-of-fit tests. Journal of the Royal Statistic Society, Series B, 46, 440–464.], in the loglinear model of quasi-independence. A simulation study illustrates that minimum chi-squared estimator and Cressie–Read estimator are good alternatives to the classical maximum-likelihood estimator for this problem. The estimator obtained for λ=2 is the most robust and efficient estimator among the family of the minimum power estimators.

Estimators for the common principal components model based on reweighting: influence functions and Monte Carlo study

The common principal components model for several groups of multivariate observations is a useful parsimonious model for the scatter structure which assumes equal principal axes but different variances along those axes for each group. Due to the lack of resistance of the classical maximum likelihood estimators for the parameters of this model, several robust estimators have been proposed in the literature: plug-in estimators and projection-pursuit (PP) type estimators. In this paper, we show that it is possible to improve the low efficiency of the projection-pursuit estimators by applying a reweighting step. More precisely, we consider plug-in estimators obtained by plugging a reweighted estimator of the scatter matrices into the maximum likelihood equations defining the principal axes. The weights considered penalize observations with large values of the influence measures defined by Boente et al. (2002). The new estimators are studied in terms of theoretical properties (influence functions and asymptotic variances) and are compared with other existing estimators in a simulation study.

Bayesian Time-Series Model for Short-Term Traffic Flow Forecasting

The seasonal autoregressive integrated moving average (SARIMA) model is one of the popular univariate time-series models in the field of short-term traffic flow forecasting. The parameters of the SARIMA model are commonly estimated using classical (maximum likelihood estimate and/or least-squares estimate) methods. In this paper, instead of using classical inference the Bayesian method is employed to estimate the parameters of the SARIMA model considered for modeling. In Bayesian analysis the Markov chain Monte Carlo method is used to solve the posterior integration problem in high dimension. Each of the estimated parameters from the Bayesian method has a probability density function conditional to the observed traffic volumes. The forecasts from the Bayesian model can better match the traffic behavior of extreme peaks and rapid fluctuation. Similar to the estimated parameters, each forecast has a probability density curve with the maximum probable value as the point forecast. Individual probability density curves provide a time-varying prediction interval unlike the constant prediction interval from the classical inference. The time-series data used for fitting the SARIMA model are obtained from a certain junction in the city center of Dublin.

A third-order bias corrected estimate in generalized linear models

Cordeiro and McCullagh (J Roy Stat Soc Ser B 53:629–643, 1991) derived a second-order bias-corrected estimate, which displays smaller bias than the classical maximum likelihood estimate in generalized linear models. This estimate, although consistent, can display pronounced bias in small to moderately large samples, as shown by Monte Carlo simulations here. In this paper, we obtain a simple matrix formula for the bias of order n-2 of this estimate, where n is the sample size, and define a third-order bias-corrected estimate in this class of models, which displays much smaller bias in small samples. In particular, some Monte Carlo simulations show that our new estimate can deliver substantial improvements in terms of bias and mean squared errors over the usual maximum likelihood estimate and Cordeiro and McCullagh’s estimate.

Avoiding Boundary Estimates in Latent Class Analysis by Bayesian Posterior Mode Estimation

In maximum likelihood estimation of latent class models, it often occurs that one or more of the parameter estimates are on the boundary of the parameter space; that is, that estimated probabilities equal 0 (or 1) or, equivalently, that logit coefficients equal minus (or plus) infinity. This not only causes numerical problems in the computation of the variance-covariance matrix, it also makes the reported confidence intervals and significance tests for the parameters concerned meaningless. Boundary estimates can, however, easily be prevented by the use of prior distributions for the model parameters, yielding a Bayesian procedure called posterior mode or maximum a posteriori estimation. This approach is implemented in, for example, the Latent GOLD software packages for latent class analysis (Vermunt & Magidson, 2005). Little is, however, known about the quality of posterior mode estimates of the parameters of latent class models, nor about their sensitivity for the choice of the prior distribution. In this paper, we compare the quality of various types of posterior mode point and interval estimates for the parameters of latent class models with both the classical maximum likelihood estimates and the bootstrap estimates proposed by De Menezes (1999). Our simulation study shows that parameter estimates and standard errors obtained by the Bayesian approach are more reliable than the corresponding parameter estimates and standard errors obtained by maximum likelihood and parametric bootstrapping.

Estimation and classification of FM signals using time frequency transforms

This work deals with electronic warfare: we have to process an intercepted radar-emitted signal assumed to be frequency-modulated (FM). A battery of parametric models being available, we face the two main problems: the parameter estimation, given a model, and the classification, given the battery. For the former, we propose the classical maximum likelihood estimator (MLE) and an original numerical scheme to reach it. For the latter, we construct a test based on the estimated variances cross-ratios. Both perform very efficiently with respect to (w.r.t.) the Cramer-Rao lower bound (CRLB) and the rate of correct classification, respectively, for low signal-to-noise ratios (SNRs).

Outlier Identification Procedures for Contingency Tables using Maximum Likelihood and L1 Estimates

Abstract. Observed cell counts in contingency tables are perceived as outliers if they have low probability under an anticipated loglinear Poisson model. New procedures for the identification of such outliers are derived using the classical maximum likelihood estimator and an estimator based on the L1 norm.