Approximate Likelihood Function Research Articles

An Approximate Wavelet MLE of Short- and Long-Memory Parameters

By design a wavelet's strength rests in its ability to localize a process simultaneously in time-scale space. The wavelet's ability to localize a time series in time-scale space directly leads to the computational efficiency of the wavelet representation of a N£ N matrix operator by allowing the N largest elements of the wavelet represented operator to represent the matrix operator (Devore, et al. (1992a) and (1992b)). This property allows many dense matrices to have sparse representation when transformed by wavelets. In this paper we generalize the long-memory parameter estimator of McCoy and Walden (1996) to estimate simultaneously the short and long-memory parameters. Using the sparse wavelet representation of a matrix operator, we are able to approximate an ARFIMA model's likelihood function with the series' wavelet coefficients and their variances. Maximization of this approximate likelihood function over the short and long-memory parameter space results in the approximate wavelet maximum likelihood estimates of the ARFIMA model. By simultaneously maximizing the likelihood function over both the short and long-memory parameters and using only the wavelet coefficient's variances, the approximate wavelet MLE provides a fast alternative to the frequency-domain MLE. Furthermore, the simulation studies found herein reveal the approximate wavelet MLE to be robust over the invertible parameter region of the ARFIMA model's moving average parameter, whereas the frequency-domain MLE dramatically deteriorates as the moving average parameter approaches the boundaries of invertibility.

On approximate likelihood inference in a poisson mixed model

AbstractA two‐step estimation approach is proposed for the fixed‐effect parameters, random effects and their variance σ2 of a Poisson mixed model. In the first step, it is proposed to construct a small σ2‐based approximate likelihood function of the data and utilize this function to estimate the fixed‐effect parameters and σ2. In the second step, the random effects are estimated by minimizing their posterior mean squared error. Methods of Waclawiw and Liang (1993) based on so‐called Stein‐type estimating functions and of Breslow and Clayton (1993) based on penalized quasilikelihood are compared with the proposed likelihood method. The results of a simulation study on the performance of all three approaches are reported.

Short Communication: Bias Correction for the Estimation ofDiscretized Diffusion Processesfrom an Approximated Likelihood Function

We consider the parameter estimation of a discretized diffusion process based on an approximation of the likelihood function. We suppose the observation step fixed and we propose to correct the bias of such estimators by using simulations of the process.

A Spectral Method for Estimating Parameters in Rainfall Models

One of the most difficult problems in rainfall modelling is often the fitting of theoretical models to data. In this paper a spectral approach is proposed for fitting single-site models, by considering the approximate likelihood functions of collections of sample Fourier coefficients. An objective function is derived, which is the same as that used in Whittle's method for time series parameter estimation, and is shown to have an interpretation as a quasi-likelihood when the time series is non-Gaussian. The method requires knowledge of the theoretical spectral density of models to be fitted; the form of this spectral density is given for a wide class of point-process-based rainfall models. A variant of the method is also developed for when a number of independent replications of a rainfall process are available. Large-sample properties of the estimators are derived, and the method is illustrated with some data from the south-west of England.

Bias correction for the estimation of discretized diffusion processes from an approximated likelihood function

Bias correction for the estimation of discretized diffusion processes from an approximated likelihood function

Multiple target tracking using maximum likelihood principle

Proposes a method (tracking algorithm (TAL)) based on the maximum likelihood (ML) principle for multiple target tracking in near-field using outputs from a large uniform linear array of passive sensors. The targets are assumed to be narrowband signals and modeled as sample functions of a Gaussian stochastic process. The phase delays of these signals are expressed as functions of both range and bearing angle ("track parameters") of respective targets. A new simplified likelihood function for ML estimation of these parameters is derived from a second-order approximation on the inverse of the data covariance matrix. Maximization of this likelihood function does not involve inversion of the M/spl times/M data covariance matrix, where M denotes number of sensors in the array. Instead, inversion of only a D/spl times/D matrix is required, where D denotes number of targets. In practice, D/spl Lt/M and, hence, TAL is computationally efficient. Tracking is achieved by estimating track parameters at regular time intervals wherein targets move to new positions in the neighborhood of their previous positions. TAL preserves ordering of track parameter estimates of the D targets over different time intervals. Performance results of TAL are presented, and it is also compared with methods by Sword and by Swindlehurst and Kailath (1988). Almost exact asymptotic expressions for the Cramer-Rao bound (CRB) on the variance of angle and range estimates are derived, and their utility is discussed.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

Parameter estimation of random fields with long-range dependence

This paper considers random fields with spectral density having a pole of fractional order at the origin. This class of models offers a suitable framework for studying long-range spatial correlation. An approximation to the likelihood function is given for the estimation of model parameters. The convergence rate of the approximation is obtained and an iterative method for maximising the approximate likelihood function is indicated. The method is applied to analyse a time series of maximum daily ozone levels and some texture data.

On the closed form of the likelihood function of the first order moving average model

The covariance matrix of a first order moving average process is expressed as the product of the covariance matrix of the dual autoregressive process of order one and a near identity matrix. Its inverse is then obtained. The closed form of the likelihood function is derived. A comparison is made with some approximate likelihood functions.

On the Closed Form of the Likelihood Function of the First Order Moving Average Model

The covariance matrix of a first order moving average process is expressed as the product of the covariance matrix of the dual autoregressive process of order one and a near identity matrix. Its inverse is then obtained. The closed form of the likelihood function is derived. A comparison is made with some approximate likelihood functions.

The two source maximum likelihood function

The maximum likelihood method is designed to yield high resolution estimates in a multiple source environment. The article derive a simplified representation of the maximum likelihood function for the two source case. This case is instructive to understand when and why the full power of the maximum likelihood method should be used. This approach extends the results of a previous paper (see J. Piper, IEEE Trans. Signal Processing, vol.42, no.2, p.412, 1994) that showed how the maximum likelihood function calculation can be transformed from a matrix problem to a vector product. The maximum likelihood function for spectral estimation is given. >

A simple noniterative estimator for moving average models

SUMMARY We examine a simple noniterative estimator for the parameters of a general moving average process. This non-maximum-likelihood estimator derives moving average model parameters directly from the coefficients of an approximating autoregressive model. The estimator is evaluated through asymptotic expansions and by simulation, and is also compared with maximum likelihood and the related estimator of Durbin (1959). The comparison with maximum likelihood by simulation suggests a variety of circumstances in which the simpler estimator may be appropriate despite the advantage of maximum likelihood for properly-specified low-order models. The difficulties involved in estimating a stationary autoregressive-moving average (ARMA) process with a nontrivial moving average part are well known. They include problems arising from nonlinearity, problems associated with nonuniqueness of the density function, and difficulties in deriving exact finite-sample properties. In addition, computational considerations which may be minor difficulties for a single estimation can become prominent in Monte Carlo experiments used, for example, to investigate finite-sample performance. In this paper we advance a simple estimator which may alleviate some of these difficulties, and we offer a comparison of this estimator with maximum likelihood. Most algorithms for the estimation of moving-average models are based on nonlinear least squares, or on numerical optimization of the exact or approximate likelihood function for the sample. Important contributions to the latter class include those of Osborn (1977), Godolphin (1977) and Ansley (1979). Box & Jenkins (1976) describe approximate maximum likelihood methods; see Fuller (1976, Ch. 8) for a review of nonlinear least squares methods. Hannan & Rissanen (1982) and Koreisha & Pukkila (1990) discuss estimation using a sequence of long autoregressions. These methods may suffer from a number of drawbacks. In general they require iteration, implying the potential for slow convergence of even nonconvergence. Results may be sensitive to starting values. Some methods are prone to generate noninvertible estimates or, where constrained to invertibility, estimates clustered near the boundary of the invertibility region (Ansley & Newbold, 1980). Finally, these algorithms tend to be computationally burdensome. The new, non-maximum-likelihood estimator is related to the estimator proposed by Durbin (1959). Like Durbin's, it involves approximating a moving average process by an autoregressive model, and using the pattern of autoregressive coefficients to deduce estimates of the parameters of the underlying process. The resulting estimator is simple and

Approximate maximum likelihood frequency estimation

The high-resolution frequency estimators most commonly used, such as MUSIC, ESPRIT and Yule-Walker, determine estimates of the sinusoidal frequencies from the sample covariances of noise-corrupted data. In this paper, a frequency estimation method termed Approximate Maximum Likelihood (AML) is derived from the approximate likelihood function of sample covariances. The statistical performance of AML is studied, both analytically and numerically, and compared with the Cramér-Rao bound as well as the statistical performance corresponding to the aforementioned methods of frequency estimation. AML is shown to provide the minimum asymptotic error variance in the class of all estimators based on a given set of covariances. The implementation of the AML frequency estimator is discussed in detail. The paper also introduces an AML-based procedure for estimating the number of sinusoidal signals in the measured data, which is shown to possess high detection performance.

Bayes and likelihood calculations from confidence intervals

Recently there has been considerable progress on setting good approximate confidence intervals for a single parameter θ in a multi-parameter family. Here we use these frequentist results as a convenient device for making Bayes, empirical Bayes and likelihood inferences about θ. A simple formula is given that produces an approximate likelihood function Lx†(θ) for θ, with all nuisance parameters eliminated, based on any system of approximate confidence intervals. The statistician can then modify Lx†(θ) with Bayes or empirical Bayes information for θ, without worrying about nuisance parameters. The method is developed for multiparameter exponential families, where there exists a simple and accurate system of approximate confidence intervals for any smoothly defined parameter. The approximate likelihood Lx†(θ) based on this system requires only a few times as much computation as the maximum likelihood estimate θ and its estimated standard error σ. The formula for Lx†(θ) is justified in terms of high-order adjusted likelihoods and also the Jeffreys-Welch & Peers theory of uninformative priors. Several examples are given.

On a Converse to Scheffe's Theorem

In Boos (1985) equicontinuity conditions are given which ensure the uniform convergence of densities in $\mathbb{R}^k$, given convergence in distribution. In the present note we show that such equicontinuity conditions in fact characterize uniform local convergence with no additional assumptions on the sequence of densities, or on the limit density. Versions of these results are also given when the distributions depend on an unknown parameter; these forms will be relevant for the uniform approximation of likelihood functions.

Maximum likelihood estimation for stationary point processes

In this paper we derive the log likelihood function for point processes in terms of their stochastic intensities by using the martingale approach. For practical purposes we work with an approximate log likelihood function that is shown to possess the usual asymptotic properties of a log likelihood function. The resulting estimates are strongly consistent and asymptotically normal (under some regularity conditions). As a by-product, a strong law of large numbers and a central limit theorem for martingales in continuous times are derived.

On the residuals of autoregressive processes and polynomial regression

The residual processes of a stationary AR( p) process and of polynomial regression are considered. The residuals are obtained from ordinary least squares fitting. In the AR case, the partial sums converge to Brownian motion. In the polynomial case, they converge to generalized Brownian bridges. Other uses of the residuals are considered. Parameter estimation based on approximate log likelihood function of the residuals is considered.

Estimation of Interaction Potentials of Marked Spatial Point Patterns Through the Maximum Likelihood Method

The likelihood procedure in estimating interaction potentials of spatial point patterns is developed for a set of point locations with attached marks, where the potential functions depend on the marks. Approximate log likelihood functions are derived under the assumption that the point patterns do not deviate much from the Poisson pattern in some sense. Some methods examining this assumption are provided. Analyses of ecological data sets are discussed through a model selection procedure.

On a noniformative prior distribution for bayesian inference of multinomial distribution's parameters

Noninformative prior distributions for Bayesian inference, for example, Jeffreys' priors, are much useful for the so-called “objective Bayesian inference” and make it possible to develop a method more powerful and flexible than traditional methods. In this paper a non-informative prior distribution, which is different from usual Jeffreys' priors, is introduced for Bayesian inference of multinomial distribution's parameters, using the assumption of the prior independence of the transformed parameters and the approximate data-translated likelihood function, and a short theoretical consideration for the inference based on the prior is attempted.

An identification algorithm for linear stochastic systems with time delays†

Linear discrete stochastic control systems containing unknown multiple time delays, plant parameters and noise variances are considered. An algorithm is established which uses the maximum-likelihood technique to identify the unknown parameters. An estimated likelihood function is evaluated based on the previous parameter estimates, which in turn generates a new descent direction vector to update the unknown parameters. The delays and plant parameters are identified in their respective parameter spaces. An example of a second-order stochastic system has been implemented by digital simulation to demonstrate the applicability of the algorithm.

A monte carlo study of the effects of grouping on the analysis of censored survival data

In certain survival studies the percentage of right-hand single censoring and the frequency of survival assessment, which results in the grouping of otherwise distinct failure times, is under experimental control. It is important to determine the consequences of the use of the Peto-Breslow approximate likelihood function in the analysis of studies with grouped and censored observation. A Monte Carlo study was performed to simulate the response surface analysis of data generated under the assumption of Cox's proportional hazard model. Fortuitous reductions in the mean squared error of the model parameters with slight grouping and censoring are observed.