Sequence Of Maximum Likelihood Estimators Research Articles

Maximum likelihood estimators based on discrete component lifetimes of a k-out-of-n system

This paper deals with parametric inference about the independent and identically distributed discrete lifetimes of components of a k-out-of-n system. We consider the maximum likelihood estimation assuming that the available data consists of component failure times observed up to and including the moment of the breakdown of the system. First, we provide general conditions for the almost sure existence of a strongly consistent sequence of maximum likelihood estimators (MLE’s). Then, we focus on three typical discrete failure distributions—the Poisson, binomial and negative binomial distributions—and prove that in these cases the MLE’s are unique, provided they exist, and that they are strongly consistent. Finally, we complete our results by Monte Carlo simulation study. Interestingly, the inference considered in the paper can be viewed as equivalent to one based on Type-II right censored discrete data. Therefore, our results can as well be applied to the case when Type-II right censored sample from a discrete distribution is observed.

One-step estimation for the fractional Gaussian noise at high-frequency

The present paper concerns the parametric estimation for the fractional Gaussian noise in a high-frequency observation scheme. The sequence of Le Cam’s one-step maximum likelihood estimators (OSMLE) is studied. This sequence is defined by an initial sequence of quadratic generalized variations-based estimators (QGV) and a single Fisher scoring step. The sequence of OSMLE is proved to be asymptotically efficient as the sequence of maximum likelihood estimators but is much less computationally demanding. It is also advantageous with respect to the QGV which is not variance efficient. Performances of the estimators on finite size observation samples are illustrated by means of Monte-Carlo simulations.

Closed-form maximum likelihood estimator for generalized linear models in the case of categorical explanatory variables: application to insurance loss modeling

Generalized linear models with categorical explanatory variables are considered and parameters of the model are estimated by an exact maximum likelihood method. The existence of a sequence of maximum likelihood estimators is discussed and considerations on possible link functions are proposed. A focus is then given on two particular positive distributions: the Pareto 1 distribution and the shifted log-normal distributions. Finally, the approach is illustrated on an actuarial dataset to model insurance losses.

Efficient semiparametric estimation and model selection for multidimensional mixtures

In this paper, we consider nonparametric multidimensional finite mixture models and we are interested in the semiparametric estimation of the population weights. Here, the i.i.d. observations are assumed to have at least three components which are independent given the population. We approximate the semiparametric model by projecting the conditional distributions on step functions associated to some partition. Our first main result is that if we refine the partition slowly enough, the associated sequence of maximum likelihood estimators of the weights is asymptotically efficient, and the posterior distribution of the weights, when using a Bayesian procedure, satisfies a semiparametric Bernstein von Mises theorem. We then propose a cross-validation like procedure to select the partition in a finite horizon. Our second main result is that the proposed procedure satisfies an oracle inequality. Numerical experiments on simulated data illustrate our theoretical results.

A Maximum Likelihood Approach to Least Absolute Deviation Regression

Least absolute deviation (LAD) regression is an important tool used in numerous applications throughout science and engineering, mainly due to the intrinsic robust characteristics of LAD. In this paper, we show that the optimization needed to solve the LAD regression problem can be viewed as a sequence of maximum likelihood estimates (MLE) of location. The derived algorithm reduces to an iterative procedure where a simple coordinate transformation is applied during each iteration to direct the optimization procedure along edge lines of the cost surface, followed by an MLE of location which is executed by a weighted median operation. Requiring weighted medians only, the new algorithm can be easily modularized for hardware implementation, as opposed to most of the other existing LAD methods which require complicated operations such as matrix entry manipulations. One exception is Wesolowsky's direct descent algorithm, which among the top algorithms is also based on weighted median operations. Simulation shows that the new algorithm is superior in speed to Wesolowsky's algorithm, which is simple in structure as well. The new algorithm provides a better tradeoff solution between convergence speed and implementation complexity.

On a problem of statistical inference in null recurrent diffusions

We consider a particular example of statistical inference in null recurrent one-dimensional diffusions. In a first parametric model, we prove local asymptotic mixed normality (LAMN) and efficiency of the sequence of maximum likelihood estimates (MLE): its speed of convergence is nα/2 with α ranging over (0, 1). In a second semiparametric model (where in addition an unknown nuisance function with known compact support is included in the drift), we prove a local asymptotic minimax bound and specify asymptotically efficient estimates for the unknown parameter.

Nonparametric maximum likelihood estimation in a non locally compact setting

Maximum likelihood estimation is considered in the context of infinite dimensional parameter spaces. It is shown that in some locally convex parameter spaces sequential compactness of the bounded sets ensures the existence of minimizers of objective functions and the consistency of maximum likelihood estimators in an appropriate topology. The theory is applied to revisit some classical problems of nonparametric maximum likelihood estimation, to study location parameters in Banach spaces, and finally to obtain Varadarajan’s theorem on the convergence of empirical measures in the form of a consistency result for a sequence of maximum likelihood estimators. Several parameter spaces sharing the crucial compactness property are identified.

Strong consistency of the MLE for sequential design problems

Under mild smoothness conditions on the densities, it is shown that for the sequential design problem with compact experiment space the sequence of maximum likelihood estimators is strongly consistent.

Efficient Nonparametric Estimation of Mixture Proportions

SUMMARY By constructing a sequence of multinomial approximations and related maximum likelihood estimators, we derive a Cramér-Rao lower bound for nonparametric estimators of the mixture proportions and thereby characterize asymptotically optimal estimators. For the case of the sampling model M2 of Hosmer (1973) it is shown that the sequence of maximum likelihood estimators, which can be obtained explicitly, is asymptotically optimal in this sense. The results hold true even when the multinomial approximations involve cells chosen adaptively, from the data, in a well-specified way.

A note on the consistency of maximum likelihood estimators

Abstract A sequence of maximum likelihood estimators based on a sequence of independent but not necessarily identically distributed random variables is shown to be consistent under certain assumptions. Some examples are given to show that these assumptions are easy to verify and not very restrictive.

Empirical Bayes Estimation in the Weibull Distribution

In part I empirical Bayes estimation procedures are introduced and employed to obtain an estimator for the unknown random scale parameter of a two-parameter Weibull distribution with known shape parameter. In part II, procedures are developed for estimating both the random scale and shape parameters. These estimators use a sequence of maximum likelihood estimates from related reliability experiments to form an empirical estimate of the appropriate unknown prior probability density function. Monte Carlo simulation is used to compare the performance of these estimators with the appropriate maximum likelihood estimator. Algorithms are presented for sequentially obtaining the reduced sample sizes required by the estimators while still providing mean squared error accuracy compatible with the use of the maximum likelihood estimators. In some cases whenever the prior pdf is a member of the Pearson family of distributions, as much as a 60% reduction in total test units is obtained. A numerical example is presented to illustrate the procedures.

Use of Maximum Likelihood for Estimating Error Variance from a Collection of Analysis of Variance Mean Squares

Given a collection of analysis of variance mean squares, not all of which necessarily have the same degrees of freedom, the present paper describes a method of "mapping" them so as to facilitate the statistical structuring of the mean squares. Even under a null model of no real effects, the mean squares do not have the same distribution because their degrees of freedom may differ, and the ordered mean squares cannot be regarded as the usual order statistics of a sample from a single common distribution. If the ordered mean squares in a general orthogonal analysis of variance are $0 < S_1 \leqq S_2 \leqq \cdots \leqq S_K$ with corresponding degrees of freedom, $\nu_1,\nu_2, \cdots, \nu_K$, then the inferential reference set in the present approach is one obtained by so-called complete conditioning, i.e., repeated sampling from a set of $K$ populations such that the $i$th ordered mean square will be considered to have come from the population associated with $v_i$ degrees of freedom, for $i = 1,2, \cdots, K$. The approach consists of obtaining from each of the ordered mean squares, in turn, a maximum likelihood estimate of a presumed common error variance based on an order statistics formulation which employs complete conditioning of the mean squares. Methods of obtaining the sequence of maximum likelihood estimates as well as two graphical modes of displaying them are described. Illustrative examples are included.

On large sample sequential analysis with applications to survivorship data

Although his work on the application of invariance concepts to the sequential testing of composite hypotheses is better known, Cox (1963) has also outlined a large sample approach to the same problem. His method is based on Bartlett's (1946) recognition that the sequence of maximum likelihood estimates (MLE) of the parameter of interest, calculated from an increasing number of observations, resembles asymptotically a random walk of normally distributed variables. However, the large sample theory needed to justify this approach rigorously is left largely implicit. At the end of his paper, Cox suggests that these methods may be extended to yield a sequential comparison of survival curves (Armitage (1959)), a suggestion which has been reiterated as a research problem in the monograph of Wetherill (1966).

On large sample sequential analysis with applications to survivorship data

Although his work on the application of invariance concepts to the sequential testing of composite hypotheses is better known, Cox (1963) has also outlined a large sample approach to the same problem. His method is based on Bartlett's (1946) recognition that the sequence of maximum likelihood estimates (MLE) of the parameter of interest, calculated from an increasing number of observations, resembles asymptotically a random walk of normally distributed variables. However, the large sample theory needed to justify this approach rigorously is left largely implicit. At the end of his paper, Cox suggests that these methods may be extended to yield a sequential comparison of survival curves (Armitage (1959)), a suggestion which has been reiterated as a research problem in the monograph of Wetherill (1966).