Computation Of Maximum Likelihood Estimates Research Articles

Analysis of Data with Censored Initiating and Terminating Times: A Missing-Data Approach

Abstract Survival data with censored initiating and terminating times have surfaced in some recent epidemiologic studies. Unlike standard survival analysis with known initiating times, analysis of data with both censored initiating and terminating times requires maximization of a complicated bivariate likelihood, which is often difficult to carry out. This article considers a missing-data formulation of the problem and focuses on the use of EM-type algorithms to simplify the computation of maximum likelihood estimates. This approach provides a feasible way of performing regression analysis with such bivariate survival data. Several illustrative examples are provided, including a real-data analysis application involving a cohort of HIV-infected hemophiliac patients.

Likelihood-based confidence intervals for estimating floods with given return periods

This paper discusses aspects of the calculation of likelihood-based confidence intervals for T-year floods, with particular reference to (1) the two-parameter gamma distribution; (2) the Gumbel distribution; (3) the two-parameter log-normal distribution, and other distributions related to the normal by Box-Cox transformations. Calculation of the confidence limits is straightforward using the Nelder-Mead algorithm with a constraint incorporated, although care is necessary to ensure convergence either of the Nelder-Mead algorithm, or of the Newton-Raphson calculation of maximum-likelihood estimates. Methods are illustrated using records from 18 gauging stations in the basin of the River Itajai-Acu, State of Santa Catarina, southern Brazil. A small and restricted simulation compared likelihood-based confidence limits with those given by use of the central limit theorem; for the same confidence probability, the confidence limits of the simulation were wider than those of the central limit theorem, which failed more frequently to contain the true quantile being estimated. The paper discusses possible applications of likelihood-based confidence intervals in other areas of hydrological analysis.

Conjugate Gradient Acceleration of the EM Algorithm

The EM algorithm is a very popular and widely applicable algorithm for the computation of maximum likelihood estimates. Although its implementation is generally simple, the EM algorithm often exhibits slow convergence and is costly in some areas of application. Past attempts to accelerate the EM algorithm have most commonly been based on some form of Aitken acceleration. Here we propose an alternative method based on conjugate gradients. The key, as we show, is that the EM step can be viewed (approximately at least) as a generalized gradient, making it natural to apply generalized conjugate gradient methods in an attempt to accelerate the EM algorithm. The proposed method is relatively simple to implement and can handle Problems with a large number of parameters, an important feature of most EM algorithms. To demonstrate the effectiveness of the proposed acceleration method, we consider its application to several Problems in each of the following areas: estimation of a covariance matrix from incomplete mu...

Estimation of Growth and Extinction Parameters for Endangered Species

Survival or extinction of an endangered species is inherently stochastic. We develop statistical methods for estimating quantities related to growth rates and extinction probabilities from time series data on the abundance of a single population. The statistical methods are based on a stochastic model of exponential growth arising from the biological theory of age— or stage—structured populations. The model incorporates the so—called environmental type of stochastic fluctuations and yields a lognormal probability distribution of population abundance. Calculation of maximum likelihood estimates of the two unknown parameters in this model reduces to performing a simple linear regression. We describe techniques for rigorously testing and evaluating whether the model fits a given data set. Various growth— and extinction—related quantities are functions of the two parameters, including the continuous rate of increase, the finite rate of increase, the geometric finite rate of increase, the probability of reaching a lower threshold population size, the mean, median, and most likely time of attaining the threshold, and the projected population size. Maximum likelihood estimates and minimum variance unbiased estimates of these quantities are described in detail. We provide example analyses of data on the Whooping Crane (Grus americana), grizzly bear (Ursus Arabis Hoffmann) in Yellow stone, Kirkland's Warbler (dealbatus killdeer), California Condor (Gymnoascus californianus), Puerto Rican Parrot (Amazona vicarius), pacificus (lobata bachei), and Layman Finch (tectorum cantons). The model results indicate a favorable outlook for the Whooping Crane, but long—term unfavorable prospects for the Yellow stone grizzly bear population and for Kirkland's Warbler. Results for the California Condor, in a retrospective analysis, indicate a virtual emergency existed in 1980. The analyses suggest that the Puerto Rican Parrot faces little risk of extinction from ordinary environmental fluctuations, provided intensive management efforts continue. However, the model does not account for the possibility of freak catastrophic events (hurricanes, fires, etc.), which are likely the most severe source of risk to the Puerto Rican Parrot, as shown by the recent decimation of this population by Hurricane Hugo. Model parameter estimates for the pacificus and the Layman Finch have wide uncertainty due to the extreme fluctuations in the population sizes of these species. In general, the model fits the example data sets well. We conclude that the model, and the associated statistical methods, can be useful for investigating various scientific and management questions concerning species preservation.

Multiplicative effects in mixed model analysis of variance

SUMMARY A class of two-way mixed analysis of variance models is proposed, in which the fixed and random effects enter multiplicatively. Equations are developed for iterative computation of maximum likelihood estimates via a scoring algorithm. Parameter estimation and hypothesis testing are illustrated on a set of plant genetics data.

Small sample performance of parameter estimators for tobit modesl with serial correlation*

For censored regression models with autocorrelated errors, the computation of maximum likelihood estimators becomes impractical when the sample contains long runs of limit observations. Pseudolikelihood estimators can be used in such cases. This paper reports the results of Monte Cario experiments designed to study the small sample performance of three pseudolikelihood estimators of the censored regression model with autocorrelated errors. These results suggest that the estimator proposed by Zeger and Brookmeyer as well as an alternative estimator described below behave markedly better than the consistent estimator obtained by simply ignoring serial correlation. A final experiment also suggests that the alternative estimator behaves well in large samples

Maximum likelihood estimation of dirichlet distributions

Global concavity of the likelihood function is proved by means of an inequality involving the trigamma function. The computation of maximum likelihood estimates is discussed.

Estimating Proportions from Randomized Response Data Using the EM Algorithm

Abstract The computation of maximum likelihood estimates for proportions from randomized response (RR) data may require numerical methods, especially for the more complex RR designs involving multiple proportions and samples. It is shown that one can apply the EM algorithm by viewing the RR data as mixture data, and taking the proportions to be estimated as the mixing proportions. A general formulation is presented for both related-question and unrelated-question RR designs, and illustrated with a real data set.

A Stabilized Newton-Raphson Algorithm for Log-Linear Models for Frequency Tables Derived by Indirect Observation

In a variety of problems involving models from genetics, latent-class analysis, and missing data, I apply a log-linear model to an indirectly observed frequency table. Current algorithms for computation of maximum likelihood estimates for such cases have often been unsatisfactory because they fail to converge at all or they converge at an unacceptable rate. I propose a new algorithm that converges both more quickly and more reliably than currently available alternatives. The algorithm assists in estimation of asymptotic variances of parameter estimates. It may be applied to both grouped and ungrouped data. I illustrate results in two examples from the literature on latent-class analysis.

Estimation and Prediction for a Simple Software Reliability Model

The Jelinski-Moranda model is a simple model for software failure time data. The likelihood function for this model sometimes exhibits pathological behaviour and fails to provide sensible estimates of the number of bugs in the software. Despite this the likelihood is informative about the time to the next failure. This is illustrated using real data and looking at the predictive distribution of the time to the next failure. Jelinski & Moranda (1972) use a simple model for software with an initial N bugs. The times to failure resulting from these bugs are assumed to be independent exponential random variables with common failure rate A. The model assumes that on failure the bug responsible is removed and that no other bugs are affected by this corrective action. As the program is run successive failure times thus constitute the order statistic of N random variables from an exponential distribution with failure rate A. This so called Jelinski-Moranda model has received considerable attention in the literature. Blumenthal & Marcus (1975), considering the model in a different context, present tables facilitating the calculation of maximum likelihood estimates of N. They provide a condition for the existence of a finite maximum likelihood estimator. Littlewood & Verrall (1981) present an equivalent condition and describe how, if the data exhibit reliability decay, infinite estimates occur. Foreman & Singpurwalla (1977) apply the model and use asymptotic results to develop a rule for debugging software. Meinhold & Singpurwalla (1983) apply a Bayesian approach assuming prior indepen- dence with a gamma prior for A and Poisson prior for N. Bendell & Mellor (1986) give a comprehensive review and classification of various models including the Jelinski- Moranda model. The model has been criticised on two levels. With its assumptions of a fixed number of bugs, independence and the same constant failure rate for each bug it can be criticised for being over simplistic. On a more fundamental level the notion of failure as an observable event may be considered a naive representation of software behav- iour (see Bendell & Mellor, 1986, Chapter 3). Despite these criticisms we believe that the inferential problems associated with the model are of interest in their own right. Moreover they provide insight into the behaviour which might be expected if some of the assumptions are removed. We focus attention on difficulties associated with estimation of N which occur when the failure history can be explained by a large number of bugs, each with a small failure rate, or a small number of bugs, each with a large failure rate. We argue that in these situations, unless firm prior information is available, there is little information to be gained about the number of bugs N. Despite this the likelihood function is informative about the time to the next failure of the software. This is demonstrated by looking at the predictive reliability function of the time to the next failure which decays to a positive limit, representing the probability that the software is bug free. In practical applications this reliability function may be more useful than an estimate of

A note on the estimation of variances in state-space models using the maximum a posteriori procedure

This note examines an estimation procedure for the unknown parameters in a state-space model proposed by Tsang, Glover, and Bach. The method is based on the maximum a posteriori (MAP) principle. Contrary to the claims of Tsang et al. it is shown that the algorithm performs very poorly compared to maximum likelihood. Some insight into the shortcomings of the MAP procedure is obtained by comparing it to the computation of maximum likelihood estimators by the EM algorithm.

Maximum Likelihood Estimation in the Mover-Stayer Model

Abstract The discrete time mover-stayer model, a special mixture of two independent Markov chains, has been widely used in modeling the dynamics of social processes. The problem of maximum likelihood estimation of its parameters from the data, however, which consist of a sample of independent realizations of this process, has not been considered in the literature. I present a maximum likelihood procedure for the estimation of the parameters of the mover-stayer model and develop a recursive method of computation of maximum likelihood estimators that is very simple to implement. I also verify that obtained maximum likelihood estimators are strongly consistent. I show that the two estimators of the parameters of the mover-stayer model previously proposed in the literature are special cases of the maximum likelihood estimator derived in this article, that is, they coincide with the maximum likelihood estimator under special conditions. I thus explain the interconnection between existing estimators. I also present a numerical comparison of the three estimators. Finally, I illustrate the application of the maximum likelihood estimators to testing the hypothesis that the Markov chain describes the data against the hypothesis that the mover-stayer model describes the data.

Maximum Likelihood Estimation in the Mover-Stayer Model

Abstract The discrete time mover-stayer model, a special mixture of two independent Markov chains, has been widely used in modeling the dynamics of social processes. The problem of maximum likelihood estimation of its parameters from the data, however, which consist of a sample of independent realizations of this process, has not been considered in the literature. I present a maximum likelihood procedure for the estimation of the parameters of the mover-stayer model and develop a recursive method of computation of maximum likelihood estimators that is very simple to implement. I also verify that obtained maximum likelihood estimators are strongly consistent. I show that the two estimators of the parameters of the mover-stayer model previously proposed in the literature are special cases of the maximum likelihood estimator derived in this article, that is, they coincide with the maximum likelihood estimator under special conditions. I thus explain the interconnection between existing estimators. I also present a numerical comparison of the three estimators. Finally, I illustrate the application of the maximum likelihood estimators to testing the hypothesis that the Markov chain describes the data against the hypothesis that the mover-stayer model describes the data.

Loglinear models and categorical data analysis with psychometric and econometric applications

Abstract The past decade has seen the publication of a large number of books and papers on the analysis of multi-way contingency tables using loglinear and logic models. The present article presents a summary of the statistical theory that underlies much of this work, and provides some linkage to models and methods of special interest to psychometricians and econometricians. The discussion includes a review of recent and current research on the computation of maximum likelihood estimates for loglinear and logit models, especially for large multi-way contingency tables.

Computation of maximum likelihood estimates for μ and β from a grouped sample of a normal population. A comparison of algorithms

Computation of Maximum Likelihood Estimates for μ and β from a grouped sample of a normal population is a numerical problem which occurs frequently in applied statistics. We investigate the performance of several algorithms for this problem (including Newton-Raphson, Method of Scoring, Expectation-Maximization and others) using simulated data. One of the main results is that Method of Scoring is best both in number of iterations and CPU time though no second-order derivatives are used.

A note on the computation of maximum likelihood estimates in linear regression models with autocorrelated errors

In this note we propose a modification of the Cochrane-Orcutt iterative procedure for the computation of maximum likelihood estimates in a linear regression model with autocorrelated disturbances which possesses faster asymptotical convergence rate. The theoretical properties are shown to be supported by experimental results.

Designs for R × C factorial paired comparison experiments

SUMMARY Two designs for R x C factorial paired comparison experiments are compared, assuming the Bradley-Terry model. One design employs comparisons of all pairs of the RC treatments, and the other design restricts comparisons to those pairs which have the same level of one factor. Computation of maximum likelihood estimates under several hypotheses is considered for each design. The restricted design has administrative benefits because it requires comparison of fewer distinct pairs of treatments, and it permits simpler computation of the estimates. Asymptotic relative efficiencies of the associated likelihood ratio tests show the restricted design to be usually inferior for detecting main effects and superior for detecting interactions.

TGDA: Nonparametric Discriminant Analysis

A computer program for two-group nonparametric discriminant analysis is described here. Based on Bayes' Theorem for probability revision, the statistical rationale for this program makes use of the calculation of maximum likelihood estimates of group membership. A unique feature of the program is the comparison of the classificatory ability of the Bayesian procedure relative to that of the standard Linear Discriminant Function.

The computation of maximum likelihood estimates for the multiple logistic risk function for use with categorical data

A computing technique is described for the maximum likelihood estimation of multiple logistic coefficients. The method makes use of the marginal iterative proportional fitting system commonly applied in the analysis of multidimensional contingency tables. The estimated logistic coefficients for two models fitted to a set of data are compared to those derived from the more traditional Newton-Raphson approach.

A Linear Model for the Yield Dependence of Magnitudes Measured by a Seismographic Network

Summary The connection between the yields of six underground nuclear explosions in the Pahute Mesa in Nevada and the body and surface wave magnitudes measured at 19 Canadian seismographic stations is described by a simple statistical model. It describes the expected magnitudes as linear functions of the logarithm of the yield and attributes the apparently normally distributed magnitude variations between stations to wave path dependent level differences between the transfer functions from the source area to the stations. The model assigns the same scaling exponent for yield to all stations. The exponent is, however, larger for surface wave magnitudes than for body wave magnitudes. The station residuals are partly attributed to differences, between explosions, in the energy coupling in the source area. The residuals are roughly normally distributed and the stations can be divided into groups with high and low residual variances. Applications of the model to the prediction of station magnitudes from given yields, to the calculation of maximum likelihood estimates of yields from measured station magnitudes and to the identification of underground nuclear explosions by simultaneous measurements of body and surface wave magnitudes are briefly discussed.