Existence Of Maximum Likelihood Estimates Research Articles

Sufficient Conditions for the Existence of a Solution for the Log-Likelihood Equations in the Three-Parameter Gamma Distribution

The existence of maximum likelihood estimators for the three-parameter gamma distribution is still an open problem. It demands the solution of the system of the log-likelihood equations which can be solved only with the use of numerical methods. The aim of this article is to provide sufficient conditions for the existence of a solution for the system of the log-likelihood equations. The conditions are expressed in terms of the geometric, arithmetic, and harmonic mean. Also, the importance of the sign of the third central moment for the existence of a solution is revealed.

Three centuries of categorical data analysis: Log-linear models and maximum likelihood estimation

The common view of the history of contingency tables is that it begins in 1900 with the work of Pearson and Yule, but in fact it extends back at least into the 19th century. Moreover, it remains an active area of research today. In this paper we give an overview of this history focussing on the development of log-linear models and their estimation via the method of maximum likelihood. Roy played a crucial role in this development with two papers co-authored with his students, Mitra and Marvin Kastenbaum, at roughly the mid-point temporally in this development. Then we describe a problem that eluded Roy and his students, that of the implications of sampling zeros for the existence of maximum likelihood estimates for log-linear models. Understanding the problem of non-existence is crucial to the analysis of large sparse contingency tables. We introduce some relevant results from the application of algebraic geometry to the study of this statistical problem.

Existence and Uniqueness Conditions for the Maximum Likelihood Solution In Regression Models For Correlated Bernoulli Random Variables

We give sufficient and necessary conditions for the existence of the maximum likelihood estimate in a class of multivariate regression models for correlated Bernoulli random variables. The models use the concept of threshold crossing technique of an underlying multivariate latent variable with univariate components formulated as a linear regression model. However, in place of their Gaussian assumptions, any specified distribution with a strictly increasing cumulative distribution function is allowed for error terms. A well known member of this class of models is the multivariate probit model. We show that our results are a generalization of the concepts of separation and overlap of Albert and Anderson for the study of the existence of maximum likelihood estimate in generalized linear models. Implications of our findings are illustrated through some hypothetical examples.

MAXIMUM LIKELIHOOD ESTIMATION IN LINEAR MODELS WITH EQUI‐CORRELATED RANDOM ERRORS

SummaryNecessary and sufficient conditions for the existence of maximum likelihood estimators of unknown parameters in linear models with equi‐correlated random errors are presented. The basic technique we use is that these models are, first, orthogonally transformed into linear models with two variances, and then the maximum likelihood estimation problem is solved in the environment of transformed models. Our results generalize a result of Arnold, S. F. (1981)[The theory of linear models and multivariate analysis. Wiley, New York]. In addition, we give necessary and sufficient conditions for the existence of restricted maximum likelihood estimators of the parameters. The results of Birkes, D. & Wulff, S. (2003)[Existence of maximum likelihood estimates in normal variance‐components models. J Statist Plann. Inference.113, 35–47] are compared with our results and differences are pointed out.

Polyhedral conditions for the nonexistence of the MLE for hierarchical log-linear models

We provide a polyhedral description of the conditions for the existence of the maximum likelihood estimate (MLE) for a hierarchical log-linear model. The MLE exists if and only if the observed margins lie in the relative interior of the marginal cone. Using this description, we give an algorithm for determining if the MLE exists. If the tree width is bounded, the algorithm runs in polynomial time. We also perform a computational study of the case of three random variables under the no three-factor effect model.

Existence of maximum likelihood estimates in normal variance-components models

Under the usual nonnegativity constraints on the variance components, a maximum likelihood estimate (MLE) of the parameter vector in a normal variance-components model is known to exist. We investigate the question of existence under more general types of constraints and, in particular, under the constraint that requires only that the variance–covariance matrix be positive definite. Attention is restricted to models in which all the possible variance–covariance matrices commute with one another. It is found that in some models, such as all random one-way models with a single group having the largest size and all balanced random two-way models, the likelihood becomes infinite under the positive definiteness constraints, so that no MLE exists. In (practically) all normal balanced mixed-effects classification models, a residual maximum likelihood estimate (REMLE) exists.

On Existence of Maximum Likelihood Estimators in Exponential Families

We propose a simple necessary and sufficient condition for existence of maximum likelihood estimators in a large class of canonical exponential families. We give an application to log-spline families.

The mle algorithm for the matrix normal distribution

The maximum likelihood estimation (MLE) of the parameters of the matrix normal distribution is considered. In the absence of analytical solutions of the system of likelihood equations for the among-row and among-column covariance matrices, a two-stage algorithm must be solved to obtain their maximum likelihood estimators. A necessary and sufficient condition for the existence of maximum likelihood estimators is given and the question of their stability as solutions of the system of likelihood equations is addressed. In particular, the covariance matrix parameters and their maximum likelihood estimators are defined up to a positive multiplicative constant; only their direct product is uniquely defined. Using simulated data undertwovariance-covariancestructures that, otherwise, are indistinguishable by semivariance analysis, further specific aspects of the procedure are studied: (1) the convergence of the MLE algorithm is assessed; (2) the empirical bias of the direct product ofcovariance matrix estimators is calculated for various sample sizes; and (3) the consistency of the estimator is evaluated by its mean Euclidean distance from the parameter, as a function of the sample size. The adequacy of the matrix normal model, including the separability of the variance-covariance structure, is tested on multiple time series of dental medicine data; other applications to real doubly multivariate data are outlined.

A note on the existence of maximum likelihood estimates for Gaussian-inverted Wishart models

We show that a non-identifiability problem can occur when one attempts to estimate the degrees of freedom and the unstructured covariance matrix simultaneously in a Gaussian-inverted Wishart model, using the maximum likelihood approach. However, the EM algorithm may falsely converge to give finite estimates in this case simply because of the convergence criterion used. An alternative approach is proposed to overcome the problem.

Existence of Maximum Likelihood Estimates for Multi‐dimensional Exponential Families

This paper deals with the existence of maximum likelihood estimators for multi‐dimensional exponential families, including curved exponential families. It first gives an algorithm for determining the MLE from the data. Then it establishes that when the parameter set is either open or relatively closed in the natural parameter set, the MLE of the parameter exists in the sense of Hoffmann‐Jorgensen.

Existence and strong consistency of maximum likelihood estimates for 1-dimensional exponential families

The paper focuses on existence, in the sense of Hoffmann-Jørgensen, of the maximum likelihood estimate (MLE) of the parameter for 1-dimensional exponential families. It is established that the MLE exists in most cases.

The Existence of Maximum Likelihood Estimates from Designed Experiments

Censored, grouped, ordinal, binary, and count data are often collected in industrial experiments. The paucity of data in typical industrial experiments results in estimability problems when fitting popular regression models to these data types. We prese..

A note on nonregular likelihood functions in heteroscedastic regression models

The existence of maximum likelihood estimates for a class of heteroscedastic regression models is considered. For a given dispersion function we show that, under a weak condition, the likelihood is singular at points corresponding to nonreplicated observations, causing unrestricted maximum likelihood estimation to break down, whilst for an alternative class of dispersion functions we obtain a much stronger linear independence condition for the likelihood to be unbounded.

On the existence of minimum contrast estimates in binary response model

When random samples are drawn from a 3-parameter distribution with a shifted origin and the observations corresponding to each sample are binary, criteria for the existence of minimum contrast estimates are given. These criteria can be drived by a method, called the probability contents boundary analysis. The probabilities of the existence of maximum likelihood estimates and least squares estimates are evaluated, by simulation with 1000 replications, in the case where the underlying distribution is a 3-parameter lognormal distribution or a 3-parameter loglogistic distribution.

A necessary and sufficient condition for the existence of the maximum likelihood estimate in autoregressive models

The author studies the existence of a maximum likelihood estimate (MLE) for the parameters of a pth-order autoregressive (AR) model from n>or=1 independent records of length m of a complex time series. It is shown that, for almost all such set of observations, the MLE exists if and only if the n records be exactly fitted by complex undamped sinusoids using the same set of p distinct frequencies. >

Existence of Maximum Likelihood Estimates for Interval-Censored Data from Some Three-Parameter Models with a Shifted Origin

SUMMARY Given data in the form of interval-censored observations from a three-parameter distribution, practical criteria are established for the existence of maximum likelihood estimates for three kinds of parameter space. These criteria can be derived by analysing the behaviour of the log-likelihood near a type of boundary of the parameter space, called the probability contents inner boundary. The effectiveness of the criteria is discussed. In the case where the underlying distribution is a three-parameter log-normal distribution, the efficiency of the criteria is evaluated by simulation.

On maximum likelihood estimators for the multisite lag‐one streamflow model: Complete and incomplete data cases

Two practical problems may arise when using the traditional estimators of the parameters in the multisite lag‐one streamflow model: (1) the estimated covariance matrix may not be positive definite thereby preventing its decomposition, a necessary step for synthetic streamflow generation, and (2) if there are missing observations, streamflow data must be truncated to the shortest record resulting in a loss of useful information. This note draws attention to the existence of maximum likelihood estimators which overcome both problems. In the complete data case, explicit maximum likelihood estimators exist. However, in the incomplete data case, iterative maximum likelihood procedures must be used. In particular, the expectation maximization (EM) algorithm is considered in a state‐space framework compatible with the multisite streamflow model; this algorithm has robust convergence properties, is simple to implement, and produces smoothed estimates of the missing data. An example is presented illustrating problems with decomposition of the traditional covariance matrix estimate and also illustrating application of the EM algorithm.

A Note on A. Albert and J. A. Anderson's Conditions for the Existence of Maximum Likelihood Estimates in Logistic Regression Models

This note expands the paper by Albert & Anderson (1984) on the existence and uniqueness of maximum likelihood estimates in logistic regression models. Their three possible mutually exclusive data patterns: (i) overlap, (ii) complete separation, and (iii) quasiseparation are considered. The maximum likelihood estimate exists only in (i). Modifications of the statement and proofs of Albert & Anderson's results are given for (ii) and (iii) The identifiability for a more general model arising in the study of (iii) is discussed together with the maximization of the corresponding likelihood. A linear program is presented which determines whether data is of type (i), (ii) or (iii), and in the case of (iii) identifies Albert & Anderson's minimal set Qm.

Existence of Maximum Likelihood Estimates in Regression Models for Grouped and Ungrouped Data

SUMMARY In general, concavity of the log likelihood alone does not imply that the MLE exists always. For a class of linear regression models for grouped and ungrouped data, a necessary and sufficient condition is obtained for the existence of the maximum likelihood estimator. This condition has an intuitively simple interpretation. Further, it turns out that there are similar necessary and sufficient conditions for the existence of maximum likelihood estimates for a number of other non-linear models such as Cox's Regression Model. For a given set of data, these conditions may be verified by Linear Programming Methods.

A note on A. Albert and J. A. Anderson's conditions for the existence of maximum likelihood estimates in logistic regression models

SUMMARY This note expands the paper by Albert & Anderson (1984) on the existence and uniqueness of maximum likelihood estimates in logistic regression models. Their three possible mutually exclusive data pattems: (i) overlap, (ii) complete separation, and (iii) quasiseparation are considered. The maximum likelihood estimate exists only in (i). Modifications of the statement and proofs of Albert & Anderson's results are given for (ii) and (iii). The identifiability for a more general model arising in the study of (iii) is discussed together with the maximization of the corresponding likelihood. A linear program is presented which determines whether data is of type (i), (ii) or (iii), and in the case of (iii) identifies Albert & Anderson's minimal set Qm.