Pseudo-likelihood Research Articles

Learning directed relational models with recursive dependencies

Recently, there has been an increasing interest in generative models that represent probabilistic patterns over both links and attributes. A common characteristic of relational data is that the value of a predicate often depends on values of the same predicate for related entities. For directed graphical models, such recursive dependencies lead to cycles, which violates the acyclicity constraint of Bayes nets. In this paper we present a new approach to learning directed relational models which utilizes two key concepts: a pseudo likelihood measure that is well defined for recursive dependencies, and the notion of stratification from logic programming. An issue for modelling recursive dependencies with Bayes nets are redundant edges that increase the complexity of learning. We propose a new normal form format that removes the redundancy, and prove that assuming stratification, the normal form constraints involve no loss of modelling power. Empirical evaluation compares our approach to learning recursive dependencies with undirected models (Markov Logic Networks). The Bayes net approach is orders of magnitude faster, and learns more recursive dependencies, which lead to more accurate predictions.

Maximum Likelihood Estimation for a Hidden Semi‐Markov Model with Multivariate Observations

In this paper, a parameter estimation procedure for a condition‐based maintenance model under partial observations is presented. The deterioration process of the partially observable system is modeled as a continuous‐time homogeneous semi‐Markov process. The system can be in a healthy or unhealthy operational state, or in a failure state, and the sojourn time in the operational state follows a phase‐type distribution. Only the failure state is observable, whereas operational states are unobservable. Vector observations that are stochastically related to the system state are collected at equidistant sampling times. The objective is to determine maximum likelihood estimates of the model parameters using the Expectation–Maximization (EM) algorithm. We derive explicit formulae for both the pseudo likelihood function and the parameter updates in each iteration of the EM algorithm. A numerical example is developed to illustrate the efficiency of the estimation procedure. Copyright © 2012 John Wiley & Sons, Ltd.

A Comparison of VaR Estimation Procedures for Leptokurtic Equity Index Returns

The paper presents and tests Dynamic Value at Risk (VaR) estimation procedures for equity index returns. Volatility clustering and leptokurtosis are well-documented characteristics of such time series. An ARMA (1, 1)-GARCH (1, 1) approach models the inherent autocorrelation and dynamic volatility. Fattailed behavior is modeled in two ways. In the first approach, the ARMA-GARCH process is run assuming alternatively that the standardized residuals are distributed with Pearson Type IV, Johnson SU, Manly’s exponential transformation, normal and t-distributions. In the second approach, the ARMA-GARCH process is run with the pseudonormal assumption, the parameters calculated with the pseudo maximum likelihood procedure, and the standardized residuals are later alternatively modeled with Mixture of Normal distributions, Extreme Value Theory and other power transformations such as John-Draper, Bickel-Doksum, Manly, Yeo-Johnson and certain combinations of the above. The first approach yields five models, and the second ap-proach yields nine. These are tested with six equity index return time series using rolling windows. These models are compared by computing the 99%, 97.5% and 95% VaR violations and contrasting them with the expected number of violations.

Sequential Estimation of Structural Models With a Fixed Point Constraint

This paper considers the estimation problem of structural models for which empirical restrictions are characterized by a fixed point constraint, such as structural dynamic discrete choice models or models of dynamic games. We analyze a local condition under which the nested pseudo likelihood (NPL) algorithm converges to a consistent estimator, and derive its convergence rate. We find that the NPL algorithm may not necessarily converge to a consistent estimator when the fixed point mapping does not have a local contraction property. To address the issue of divergence, we propose alternative sequential estimation procedures that can converge to a consistent estimator even when the NPL algorithm does not.

Hypothesis Testing in Generalized Linear Models with Functional Coefficient Autoregressive Processes

The paper studies the hypothesis testing in generalized linear models with functional coefficient autoregressive (FCA) processes. The quasi‐maximum likelihood (QML) estimators are given, which extend those estimators of Hu (2010) and Maller (2003). Asymptotic chi‐squares distributions of pseudo likelihood ratio (LR) statistics are investigated.

Constrained Compound Markov Random Field Model with Graduated Penalty Function for Color Image Segmentation

In this paper, an unsupervised color image segmentation scheme has been proposed for preserving strong and weak edges as well. A Constrained Compound Markov Random Field (MRF) has been proposed as the a priori model for the color labels. We have used Ohta (I1, I2, I3) color model and a controlled correlation of the color space has been accomplished by the proposed compound MRF model. The Constrained Compound MRF (CCMRF) is found to possess the unifying property of modeling scenes as well as color textures. In unsupervised scheme, the associated model parameters and the image labels are estimated recursively. The model parameters are the Maximum Conditional Pseudo Likelihood (MCPL) estimates and the labels are the Maximum a Posteriori (MAP) estimates. The performance of the proposed scheme has been compared with that of Yu’s method and has been found to exhibit improved performance in the context of misclassification error.

Autologistic models for binary data on a lattice

AbstractThe autologistic model is a Markov random field model for spatial binary data. Because it can account for both statistical dependence among the data and for the effects of potential covariates, the autologistic model is particularly suitable for problems in many fields, including ecology, where binary responses, indicating the presence or absence of a certain plant or animal species, are observed over a two‐dimensional lattice. We consider inference and computation for two models: the original autologistic model due to Besag, and the centered autologistic model proposed recently by Caragea and Kaiser. Parameter estimation and inference for these models is a notoriously difficult problem due to the complex form of the likelihood function. We study pseudolikelihood (PL), maximum likelihood (ML), and Bayesian approaches to inference and describe ways to optimize the efficiency of these algorithms and the perfect sampling algorithms upon which they depend, taking advantage of parallel computing when possible. We conduct a simulation study to investigate the effects of spatial dependence and lattice size on parameter inference, and find that inference for regression parameters in the centered model is reliable only for reasonably large lattices (n > 900) and no more than moderate spatial dependence. When the lattice is large enough, and the dependence small enough, to permit reliable inference, the three approaches perform comparably, and so we recommend the PL approach for its easier implementation and much faster execution. Copyright © 2011 John Wiley & Sons, Ltd.

Nonlinear structural equation modeling with distribution-free method

Structural equation models (SEM) are widely used in many fields including economics and social science. Typical nonlinear SEMs consist of two parts: a linear measurement model relating observed measurements to underlying latent variables, and a nonlinear structural model describing relationships among the latent variables. For such models, we propose a pseudo likelihood approach based on a hypothetical normal mixture assumption on the latent variables. To obtain pseudo likelihood parameter estimates, a Monte Carlo EM algorithm is developed. Standard errors for the structural parameter estimates are obtained by combining an empirical observed information matrix and a bootstrap estimated covariance matrix. For nonlinear SEMs with latent variables with various distributions, we conduct simulations to show our approach produced unbiased parameter estimates and confidence intervals with nominal coverage.

On a Statistical Analysis of Implied Data

We propose a method of evaluating the accuracy of the implied default probabilities. We modify the model proposed by Duffie et al. (Rev Fin Stud 12:678–720, 1999) to allow the parametric statistical analysis. The pseudo maximum likelihood estimator is defined and to justify our method we shall prove the consistency and the asymptotic normality of the estimator. The key step is to define a pseudo score vector and apply the method of Wald (Ann Math Stat 20: 595–601, 1949) and a delta method. We also introduce the bootstrap for estimating the accuracies, which is similar to that for regression models. To implement our method to the real data, we shall recommend the bootstrap rather than asymptotic normality.

On the calibration of design weights using a displacement function

In the present investigation, we propose a new method to calibrate the estimator of the general parameter of interest in survey sampling. We demonstrate that the linear regression estimator due to Hansen et al. (Sample Survey Method and Theory. Wiley, NY, 1953) is a special case of this. We reconfirm that the sum of calibrated weights has to be set equal to sum of the design weights within a given sample as shown in Singh (Advanced sampling theory with applications: How Michael ‘selected’ Amy, Vol. 1 and 2. Kluwer, The Netherlands, pp 1–1247, 2003; Proceedings of the American Statistical Association, Survey Method Section [CD-ROM], Toronto, Canada: American Statistical Association, pp 4382–4389, 2004; Metrika:1–18, 2006a; Presented at INTERFACE 2006, Pasadena, CA, USA, 2006b) and Stearns and Singh (Presented at Joint Statistical Meeting, MN, USA (Available on the CD), 2005; Comput Stat Data Anal 52:4253–4271, 2008). Thus, it shows that the Sir. R.A. Fisher’s brilliant idea of keeping sum of observed frequencies equal to that of expected frequencies leads to a “Honest-Balance” while weighing design weights in survey sampling. The major benefit of the proposed new estimator is that it always works unlike the pseudo empirical likelihood estimators listed in Owen (Empirical Likelihood. Chapman & Hall, London, 2001), Chen and Sitter (Stat Sin 9:385–406, 1999) and Wu (Sur Methodol 31(2):239–243, 2005). The main endeavor of this paper is to bring a change in the existing calibration technology, which is based on only positive distance functions, with a displacement function that has the flexibility of taking positive, negative, or zero value. At the end, the proposed technology has been compared with its competitors under several kinds of linear and non-linear non-parametric models using an extensive simulation study. A couple of open questions are raised.

Spatial Econometrics Approach to Integration of Behavioral Biases in Travel Demand Analysis

Random utility models customarily assume strict independence of individual decision makers. Evidence of crowding, peer pressure, herd behavior, and other instances of spontaneous discrete choice coordination indicates that decision makers interact and thus affect choices made by others. Socially influenced individual choices become biased toward either agreeing with or contradicting the choices made by peers. Because many social interdependencies are spatial, a basic spatial discrete choice model was obtained by extending random utility theory to discrete choices made by heterogeneous spatially dependent individuals. Although interdependencies are unobserved, the model permits the study of behavioral biases arising from spatial interdependencies. The spatial discrete choice model is shown to address the effects of behavioral biases on conditional choice probabilities, the marginal effects of exogenous variables on revealed preferences, and the spatial patterns of discrete choices. A pseudo maximum likelihood (PML) estimator for the model is developed, and closed-form expressions for conditional choice probability estimates are derived. The PML estimator is shown to be consistent and computationally feasible for large spatial data sets. Simulated data were used to illustrate the performance of the PML estimator for the spatial discrete choice model.

Statistical properties of parametric estimators for Markov chain vectors based on copula models

To estimate and measure risks, two key classes of dependence relationship must be identified: temporal dependence and contemporaneous dependence. In this paper, we propose a parametric estimation model that uses a three-stage pseudo maximum likelihood estimation (3SPMLE), and we investigate the consistency and asymptotic normality of parametric estimators. The proposed model combines the concept of a copula and the methods of parametric estimators of two-stage pseudo maximum likelihood estimation (2SPMLE). The selection of a copula model that best captures the dependence structure is a critical problem. To solve this problem, we propose a model selection method that is based on the parametric pseudo-likelihood ratio under the 3SPMLE for stationary Markov vector-type models.

A simulation study to assess statistical methods for binary repeated measures data

Binary repeated measures data are commonly encountered in both experimental and observational veterinary studies. Among the wide range of statistical methods and software applicable to such data one major distinction is between marginal and random effects procedures. The objective of the study was to review and assess the performance of marginal and random effects estimation procedures for the analysis of binary repeated measures data. Two simulation studies were carried out, using relatively small, balanced, two-level (time within subjects) datasets. The first study was based on data generated from a marginal model with first order autocorrelation, the second on a random effects model with autocorrelated random effects within subjects. Three versions of the models were considered in which a dichotomous treatment was modelled additively, either between or within subjects, or modelled by a time interaction. Among the studied statistical procedures were: generalized estimating equations (GEE), Marginal Quasi Likelihood, likelihood based on numerical integration, penalized quasi-likelihood, restricted pseudo likelihood and Bayesian Markov Chain Monte Carlo. Results for data generated by the marginal model showed autoregressive GEE to be highly efficient when treatment was within subjects, even with strongly correlated responses. For treatment between subjects, random effects procedures also performed well in some situations; however, a relatively small number of subjects with a short time series proved a challenge for both marginal and random effects procedures. Results for data generated by the random effects model showed bias in estimates from random effects procedures when autocorrelation was present in the data, while the marginal procedures generally gave estimates close to the marginal parameters.

Pseudo-likelihood equations for Potts model on higher-order neighborhood systems: A quantitative approach for parameter estimation in image analysis

This paper presents analytical pseudo-likelihood (PL) equations for Potts Markov random field (MRF) model parameter estimation on higher-order neighborhood systems by expanding the derivative of the log-PL function based on the enumeration of all possible contextual configuration patterns given a neighborhood system. The proposed equations allow the modeling of less restrictive neighborhood systems in a large number of MRF applications in a computationally feasible way. To evaluate the proposed estimation method we propose a hypothesis testing approach, derived by approximating the asymptotic variance of MPL parameter estimators using the observed Fisher information. The definition of the asymptotic variance, together with the test size α and p-values, provide a complete framework for quantitative analysis. Experiments with synthetic images generated by Markov chain Monte Carlo simulation methods assess the accuracy of the proposed estimation method, indicating that higher-order neighborhood systems reduce the MPL estimator asymptotic variance and improve estimation performance.

Estimation of poverty measures with auxiliary information in sample surveys

The analysis of poverty measures has been receiving increased attention in recent years. This paper contributes to the literature by developing percentile ratio estimators based on the pseudo empirical likelihood method. In practice, variances of poverty measures could be not expressible by simple formulae and consequently other techniques should be used in the variance estimation stage. Assuming percentile ratios, resampling techniques are investigated in this paper. A numerical example based on data from the Spanish Household Panel Survey is taken up to illustrate how suggested procedures can perform better than existing ones. The effect of a model-misspecification on the proposed estimators is also evaluated by using simulated populations.

A Note on the Use of Unbiased Estimating Equations to Estimate Correlation in Analysis of Longitudinal Trials

Longitudinal trials can yield outcomes that are continuous, binary (yes/no), or are realizations of counts. In this setting we compare three approaches that have been proposed for estimation of the correlation in the framework of generalized estimating equations (GEE): quasi-least squares (QLS), pseudo-likelihood (PL), and an approach we refer to as Wang-Carey (WC). We prove that WC and QLS are identical for the first-order autoregressive AR(1) correlation structure. Using simulations, we then develop guidelines for selection of an appropriate method for analysis of data from a longitudinal trial. In particular, we argue that no method is uniformly superior for analysis of unbalanced and unequally spaced data with a Markov correlation structure. Choice of the best approach will depend on the degree of imbalance and variability in the temporal spacing of measurements, value of the correlation, and type of outcome, e.g. binary or continuous. Finally, we contrast the methods in analysis of a longitudinal study of obesity following renal transplantation in children.

Empirical likelihood confidence intervals for adaptive cluster sampling

Adaptive cluster sampling (ACS) is an efficient sampling design for estimating parameters of rare and clustered populations. It is widely used in ecological research. The modified Hansen-Hurwitz (HH) and Horvitz-Thompson (HT) estimators based on small samples under ACS have often highly skewed distributions. In such situations, confidence intervals based on traditional normal approximation can lead to unsatisfactory results, with poor coverage properties. Christman and Pontius (Biometrics 56:503–510, 2000) showed that bootstrap percentile methods are appropriate for constructing confidence intervals from the HH estimator. But Perez and Pontius (J Stat Comput Simul 76:755–764, 2006) showed that bootstrap confidence intervals from the HT estimator are even worse than the normal approximation confidence intervals. In this article, we consider two pseudo empirical likelihood functions under the ACS design. One leads to the HH estimator and the other leads to a HT type estimator known as the Hajek estimator. Based on these two empirical likelihood functions, we derive confidence intervals for the population mean. Using a simulation study, we show that the confidence intervals obtained from the first EL function perform as good as the bootstrap confidence intervals from the HH estimator but the confidence intervals obtained from the second EL function perform much better than the bootstrap confidence intervals from the HT estimator, in terms of coverage rate.

Robust Wavelet-Domain Estimation of the Fractional Difference Parameter in Heavy-Tailed Time Series: An Empirical Study

We investigate the performance of several wavelet-based estimators of the fractional difference parameter. We consider situations where, in addition to long-range dependence, the time series exhibit heavy tails and are perturbed by polynomial and change-point trends. We make detailed study of a wavelet-domain pseudo Maximum Likelihood Estimator (MLE), for which we provide an asymptotic and finite-sample justification. Using numerical experiments, we show that unlike the traditional time-domain estimators, estimators based on the wavelet transform are robust to additive trends and change points in mean, and produce accurate estimates even under significant departures from normality. The Wavelet-domain MLE appears to dominate a regression-based wavelet estimator in terms of smaller root mean squared error. These findings are derived from a simulation study and application to computer traffic traces.

Pseudo maximum likelihood estimates using ranked set sampling with applications to estimating correlation

Suppose θ is the parameter of interest and λ is the nuisance parameter. When obtaining the maximum likelihood estimator (MLE) of θ in the presence of λ requires intensive computation, the pseudo MLE of θ, based on a pseudo likelihood function, can be used. Gong and Samaniego (Ann. Stat. 9:861–869, 1981) proposed a pseudo MLE (PMLE) based on simple random samples. Ranked set sampling has been applied to the bivariate variables (X,Y) where measuring one of the variables is difficult or costly. In this paper, we obtain the pseudo MLE of the correlation coefficient from a bivariate normal distribution (X,Y) based on ranked set samples, assuming that Y is difficult or more expensive to measure and that the mean and variance of Y are the nuisance parameters. The PMLE is compared with three other estimators of the correlation coefficient. Simulations show that the PMLE is more (less) efficient than other estimators, depending on value of ρ. Testing of soil contamination provides an example of the use of the methods.

ML, PL, QL in Markov Chain Models

Abstract. In many spatial and spatial‐temporal models, and more generally in models with complex dependencies, it may be too difficult to carry out full maximum‐likelihood (ML) analysis. Remedies include the use of pseudo‐likelihood (PL) and quasi‐likelihood (QL) (also called the composite likelihood). The present paper studies the ML, PL and QL methods for general Markov chain models, partly motivated by the desire to understand the precise behaviour of the PL and QL methods in settings where this can be analysed. We present limiting normality results and compare performances in different settings. For Markov chain models, the PL and QL methods can be seen as maximum penalized likelihood methods. We find that QL is typically preferable to PL, and that it loses very little to ML, while sometimes earning in model robustness. It has also appeal and potential as a modelling tool. Our methods are illustrated for consonant‐vowel transitions in poetry and for analysis of DNA sequence evolution‐type models.