Monte Carlo Maximum Likelihood Research Articles

State Space Models With a Common Stochastic Variance

This article considers a combination of the linear Gaussian state space model and the stochastic volatility model. The focus is on the simultaneous estimation of parameters related to the stochastic processes of both the mean and variance parts of the model. Kalman filter and Monte Carlo maximum likelihood methods lead to an elegant estimation procedure for which the simulation error can be made arbitrarily small. The standard asymptotic properties of maximum likelihood estimators apply as a result. A Monte Carlo simulation study is carried out to investigate the small-sample properties of the estimation procedure. The modeling and forecasting techniques within our new framework are illustrated for U.S. monthly inflation rates.

Stochastic Conditional Duration Models with "Leverage Effect" for Financial Transaction Data

This article proposes stochastic conditional duration (SCD) models with ‘‘leverage effect’’ for financial transaction data, which extends both the autoregressive conditional duration (ACD) model (Engle and Russell, 1998, Econometrica, 66, 1127–1162) and the existing SCD model (Bauwens and Veredas, 2004, Journal of Econometrics, 119, 381–412). The proposed models belong to a class of linear nongaussian state-space models, where the observation equation for the duration process takes an additive form of a latent process and a noise term. The latent process is driven by an autoregressive component to characterize the transition property and a term associated with the observed duration. The inclusion of such a term allows the model to capture the asymmetric behavior or ‘‘leverage effect’’ of the expected duration. The Monte Carlo maximum-likelihood (MCML) method is employed for consistent and efficient parameter estimation with applications to the transaction data of IBM and other stocks. Our analysis suggests that trade intensity is correlated with stock return volatility and modeling the duration process with ‘‘leverage effect’’ can enhance the forecasting performance of intraday volatility.

On the Convergence of the Monte Carlo Maximum Likelihood Method for Latent Variable Models

While much used in practice, latent variable models raise challenging estimation problems due to the intractability of their likelihood. Monte Carlo maximum likelihood (MCML), as proposed by Geyer & Thompson (1992), is a simulation‐based approach to maximum likelihood approximation applicable to general latent variable models. MCML can be described as an importance sampling method in which the likelihood ratio is approximated by Monte Carlo averages of importance ratios simulated from the complete data model corresponding to an arbitrary value ϕ of the unknown parameter. This paper studies the asymptotic (in the number of observations) performance of the MCML method in the case of latent variable models with independent observations. This is in contrast with previous works on the same topic which only considered conditional convergence to the maximum likelihood estimator, for a fixed set of observations. A first important result is that when ϕ is fixed, the MCML method can only be consistent if the number of simulations grows exponentially fast with the number of observations. If on the other hand, ϕ is obtained from a consistent sequence of estimates of the unknown parameter, then the requirements on the number of simulations are shown to be much weaker.

A comprehensive model of mutations affecting fitness and inferences for Arabidopsis thaliana.

As the ultimate source of genetic variation, spontaneous mutation is essential to evolutionary change. Theoretical studies over several decades have revealed the dependence of evolutionary consequences of mutation on specific mutational properties, including genomic mutation rates, U, and the effects of newly arising mutations on individual fitness, s. The recent resurgence of empirical effort to infer these properties for diverse organisms has not achieved consensus. Estimates, which have been obtained by methods that assume mutations are unidirectional in their effects on fitness, are imprecise. Both because a general approach must allow for occurrence of fitness-enhancing mutations, even if these are rare, and because recent evidence demands it, we present a new method for inferring mutational parameters. For the distribution of mutational effects, we retain Keightley's assumption of the gamma distribution, to take advantage of the flexibility of its shape. Because the conventional gamma is one sided, restricting it to unidirectional effects, we include an additional parameter, rho, as an amount it is displaced from zero. Estimation is accomplished by Markov chain Monte Carlo maximum likelihood. Through a limited set of simulations, we verify the accuracy of this approach. We apply it to analyze data on two reproductive fitness components from a 17-generation mutation-accumulation study of a Columbia accession of Arabidopsis thaliana in which 40 lines sampled in three generations were assayed simultaneously. For these traits, U approximately/= 0.1-0.2, with distributions of mutational effects broadly spanning zero, such that roughly half the mutations reduce reproductive fitness. One evolutionary consequence of these results is lower extinction risks of small populations of A. thaliana than expected from the process of mutational meltdown. A comprehensive view of the evolutionary consequences of mutation will depend on quantitatively accounting for fitness-enhancing, as well as fitness-reducing, mutations.

A COMPREHENSIVE MODEL OF MUTATIONS AFFECTING FITNESS AND INFERENCES FOR ARABIDOPSIS THALIANA

As the ultimate source of genetic variation, spontaneous mutation is essential to evolutionary change. Theoretical studies over several decades have revealed the dependence of evolutionary consequences of mutation on specific mutational properties, including genomic mutation rates, U, and the effects of newly arising mutations on individual fitness, s. The recent resurgence of empirical effort to infer these properties for diverse organisms has not achieved consensus. Estimates, which have been obtained by methods that assume mutations are unidirectional in their effects on fitness, are imprecise. Both because a general approach must allow for occurrence of fitness-enhancing mutations, even if these are rare, and because recent evidence demands it, we present a new method for inferring mutational parameters. For the distribution of mutational effects, we retain Keightley's assumption of the gamma distribution, to take advantage of the flexibility of its shape. Because the conventional gamma is one sided, restricting it to unidirectional effects, we include an additional parameter, ρ, as an amount it is displaced from zero. Estimation is accomplished by Markov chain Monte Carlo maximum likelihood. Through a limited set of simulations, we verify the accuracy of this approach. We apply it to analyze data on two reproductive fitness components from a 17-generation mutation-accumulation study of a Columbia accession of Arabidopsis thaliana in which 40 lines sampled in three generations were assayed simultaneously. For these traits, U ≃ 0.1–0.2, with distributions of mutational effects broadly spanning zero, such that roughly half the mutations reduce reproductive fitness. One evolutionary consequence of these results is lower extinction risks of small populations of A. thaliana than expected from the process of mutational meltdown. A comprehensive view of the evolutionary consequences of mutation will depend on quantitatively accounting for fitness-enhancing, as well as fitness-reducing, mutations.Corresponding Editor: A. Caballero

Hyperparameter estimation for satellite image restoration using a MCMC maximum-likelihood method

The satellite image deconvolution problem is ill-posed and must be regularized. Herein, we use an edge-preserving regularization model using a ϕ function, involving two hyperparameters. Our goal is to estimate the optimal parameters in order to automatically reconstruct images. We propose to use the maximum-likelihood estimator (MLE), applied to the observed image. We need sampling from prior and posterior distributions. Since the convolution prevents use of standard samplers, we have developed a modified Geman–Yang algorithm, using an auxiliary variable and a cosine transform. We present a Markov chain Monte Carlo maximum-likelihood (MCMCML) technique which is able to simultaneously achieve the estimation and the reconstruction.

The use of approximating models in Monte Carlo maximum likelihood estimation

To obtain the likelihood of a non-Gaussian state-space model, Durbin and Koopman (1997, Biometrika, 84, 669–684) first calculate the likelihood under an approximating linear Gaussian model and then use Monte Carlo methods to estimate the necessary adjustment factor. We show that Durbin and Koopman's method is closely related to a method proposed by Geyer (1994, J. Roy. Statist. Soc. B 56, 261–274) for simulating the likelihood of a random-effects model and to a method proposed by Schall (1991, Biometrika, 78, 719–727) for approximating the maximum likelihood estimate of a generalised linear mixed model. A hybrid method is proposed for approximating the entire likelihood function as opposed to Durbin and Koopman's pointwise approximation. We also suggest an alternative class of approximating models based on conjugate latent process and apply it to approximate the likelihood of a time series model for count data.

Pointwise and functional approximations in Monte Carlo maximum likelihood estimation

We consider the use of Monte Carlo methods to obtain maximum likelihood estimates for random effects models and distinguish between the pointwise and functional approaches. We explore the relationship between the two approaches and compare them with the EM algorithm. The functional approach is more ambitious but the approximation is local in nature which we demonstrate graphically using two simple examples. A remedy is to obtain successively better approximations of the relative likelihood function near the true maximum likelihood estimate. To save computing time, we use only one Newton iteration to approximate the maximiser of each Monte Carlo likelihood and show that this is equivalent to the pointwise approach. The procedure is applied to fit a latent process model to a set of polio incidence data. The paper ends by a comparison between the marginal likelihood and the recently proposed hierarchical likelihood which avoids integration altogether.

Estimation of stochastic volatility models via Monte Carlo maximum likelihood

This paper discusses the Monte Carlo maximum likelihood method of estimating stochastic volatility (SV) models. The basic SV model can be expressed as a linear state space model with log chi-square disturbances. The likelihood function can be approximated arbitrarily accurately by decomposing it into a Gaussian part, constructed by the Kalman filter, and a remainder function, whose expectation is evaluated by simulation. No modifications of this estimation procedure are required when the basic SV model is extended in a number of directions likely to arise in applied empirical research. This compares favorably with alternative approaches. The finite sample performance of the new estimator is shown to be comparable to the Monte Carlo Markov chain (MCMC) method.

Analysis of multivariate probit models

SUMMARY This paper provides a practical simulation-based Bayesian and non-Bayesian analysis of correlated binary data using the multivariate probit model. The posterior distribution is simulated by Markov chain Monte Carlo methods and maximum likelihood estimates are obtained by a Monte Carlo version of the EM algorithm. A practical approach for the computation of Bayes factors from the simulation output is also developed. The methods are applied to a dataset with a bivariate binary response, to a four-year longitudinal dataset from the Six Cities study of the health effects of air pollution and to a sevenvariate binary response dataset on the labour supply of married women from the Panel Survey of Income Dynamics.

MARKOV CHAIN MONTE CARLO METHODS FOR MODELING THE SPATIAL PATTERN OF DISEASE SPREAD IN BELL PEPPER

With exponential family models for dependent data, such as the autologistic model for binary spatial lattice data, maximum likelihood estimates can be obtained using Markov chain sampling methods by simulating an ergodic Markov chain which converges weakly to the equilibrium distribution of the model. This Markov chain Monte Carlo maximum likelihood (MCMCML) procedure provides a competitor to the usual pseudolikelihood estimation method often used for modeling discrete lattice data. Within this MCMCML framework, it is also possible to conduct formal inference using MCMC analogues to the usual likelihood ratio, Wald, and Lagrange multiplier tests, for which the asymptotic distributions are known subject to some mild regularity conditions. Here, the MCMC methodology will be discussed as it pertains to the autologistic model for binary data and will be used to model the spatial pattern of disease spread in bell pepper caused by the pathogen Phytophthora capsici.

Discussion on the Meeting on the Gibbs Sampler and Other Markov Chain Monte Carlo Methods

Discussion on the Meeting on the Gibbs Sampler and Other Markov Chain Monte Carlo Methods

Estimation of the parameters of a binary Markov random field on a graph with application to fibre type distributions in a muscle cross-section.

Methods are discussed for the estimation of the parameters of a binary Markov random field (BMRF) defined on a graph. The standard method is maximum pseudo-likelihood (MPL) estimation. Maximum likelihood (ML) estimation has been hampered in the past by the intractability of the likelihood function. Recently Markov chain Monte Carlo (MCMC) methods have been introduced for ML estimation. In this paper a new method for Monte Carlo maximum likelihood is described. It is used for the estimation of the parameters of a simple model (the Ising model of statistical physics). As an application the distribution of fibre types in a cross-section of human muscle is analysed.