Complete Data Likelihood Research Articles

Variability in Causal Effects and Noncompliance in a Multisite Trial: A Bivariate Hierarchical Generalized Random Coefficients Model for a Binary Outcome.

Within each of 170 physicians, patients were randomized to access e-assist, an online program that aimed to increase colorectal cancer screening (CRCS), or control. Compliance was partial: of the experimental patients accessed e-assist while no controls were provided the access. Of interest are the average causal effect of assignment to treatment and the complier average causal effect as well as the variation of these causal effects across physicians. Each physician generates probabilities of screening for experimental compliers (experimental patients who accessed e-assist), control compliers (controls who would have accessed e-assist had they been assigned to e-assist), and never takers (patients who would have avoided e-assist no matter what). Estimating physician-specific probabilities jointly over physicians poses novel challenges. We address these challenges by maximum likelihood, factoring a "complete-data likelihood" uniquely into the conditional distribution of screening and partially observed compliance given random effects and the distribution of random effects. We marginalize this likelihood using adaptive Gauss-Hermite quadrature. The approach is doubly iterative in that the conditional distribution defies analytic evaluation. Because the small sample size per physician constrains estimability of multiple random effects, we reduce their dimensionality using a shared random effects model having a factor analytic structure. We assess estimators and recommend sample sizes to produce reasonably accurate and precise estimates by simulation, and analyze data from a trial of a CRCS intervention.

Integrated deviance information criterion for spatial autoregressive models with heteroskedasticity

In this study, we introduce the integrated deviance information criterion (DIC) for nested and non-nested model selection problems in heteroskedastic spatial autoregressive models. In a Bayesian estimation setting, we assume that the idiosyncratic error terms of our spatial autoregressive model have a scale mixture of normal distributions, where the scale mixture variables are latent variables that induce heteroskedasticity. We first derive the integrated likelihood function by analytically integrating out the scale mixture variables from the complete-data likelihood function. We then use the integrated likelihood function to formulate the integrated DIC measure. We investigate the finite sample performance of the integrated DIC in selecting the true model in a simulation study. The simulation results show that the integrated DIC performs satisfactorily and can be useful for selecting the correct model in specification search exercises. Finally, in a spatially augmented economic growth model, we use the integrated DIC to choose the spatial weights matrix that leads to better predictive accuracy.

Information criteria for matrix exponential spatial specifications

In this study, we suggest using information criteria for nested and non-nested model selection problems for the matrix exponential spatial specifications (MESS) under both homoskedasticity and heteroskedasticity. To this end, we consider the deviance information criterion, the Akaike information criterion and the Bayesian information criterion in a Bayesian setting. In the heteroskedastic case, we assume that the error terms have a scale mixture of normal distributions, where the scale mixture variables are latent variables that lead to different distributions. We demonstrate how the integrated likelihood function can be obtained analytically by integrating out the scale mixture variables from the complete-data likelihood function, and how this integrated likelihood function can be used to formulate the information criteria. We investigate the finite sample performance of these criteria in selecting the true model in a simulation study. The results show that these criteria perform satisfactorily and can be useful for selecting the correct model in specification search exercises. Finally, we apply the proposed information criteria to a spatially augmented growth model and a carbon emission model to show their usefulness for both nested and non-nested model selection problems.

On MCMC sampling in random coefficients self-exciting integer-valued threshold autoregressive processes

In this study, Bayesian estimation is performed for a class of random coefficient self-exciting integer-valued threshold autoregressive processes with explanatory variables. A new model with a linear structure is obtained through model reconstruction, which makes Markov Chain Monte Carlo method easy to perform. By introducing the latent variables series, a complete data likelihood is obtained. Based on this likelihood, the full conditional distributions are easily obtained for all the parameters and latent variables. By maximizing the posterior probability function, the threshold parameter is accurately estimated. Finally, some numerical results of the estimates and a real data example of crime counts in Ballina, New South Wales, Australia are presented.

A Robust and Flexible EM Algorithm for Mixtures of Elliptical Distributions with Missing Data

This paper tackles the problem of missing data imputation for noisy and non-Gaussian data. A classical imputation method, the Expectation Maximization (EM) algorithm for Gaussian mixture models, has shown interesting properties when compared to other popular approaches such as those based on <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$k$</tex-math></inline-formula> -nearest neighbors or on multiple imputations by chained equations. However, Gaussian mixture models are known to be non-robust to heterogeneous data, which can lead to poor estimation performance when the data is contaminated by outliers or have non-Gaussian distributions. To overcome this issue, a new EM algorithm is investigated for mixtures of elliptical distributions with the property of handling potential missing data. This paper shows that this problem reduces to the estimation of a mixture of angular Gaussian distributions under generic assumptions ( <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">i.e.</i> , each sample is drawn from a mixture of elliptical distributions, which is possibly different for one sample to another). In that case, the complete-data likelihood associated with mixtures of elliptical distributions is well adapted to the EM framework with missing data thanks to its conditional distribution, which is shown to be a multivariate <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$t$</tex-math></inline-formula> -distribution. Experimental results on synthetic data demonstrate that the proposed algorithm is robust to outliers and can be used with non-Gaussian data. Furthermore, experiments conducted on real-world datasets show that this algorithm is very competitive when compared to other classical imputation methods.

Fast and universal estimation of latent variable models using extended variational approximations

Generalized linear latent variable models (GLLVMs) are a class of methods for analyzing multi-response data which has gained considerable popularity in recent years, e.g., in the analysis of multivariate abundance data in ecology. One of the main features of GLLVMs is their capacity to handle a variety of responses types, such as (overdispersed) counts, binomial and (semi-)continuous responses, and proportions data. On the other hand, the inclusion of unobserved latent variables poses a major computational challenge, as the resulting marginal likelihood function involves an intractable integral for non-normally distributed responses. This has spurred research into a number of approximation methods to overcome this integral, with a recent and particularly computationally scalable one being that of variational approximations (VA). However, research into the use of VA for GLLVMs has been hampered by the fact that fully closed-form variational lower bounds have only been obtained for certain combinations of response distributions and link functions. In this article, we propose an extended variational approximations (EVA) approach which widens the set of VA-applicable GLLVMs dramatically. EVA draws inspiration from the underlying idea behind the Laplace approximation: by replacing the complete-data likelihood function with its second order Taylor approximation about the mean of the variational distribution, we can obtain a fully closed-form approximation to the marginal likelihood of the GLLVM for any response type and link function. Through simulation studies and an application to a species community of testate amoebae, we demonstrate how EVA results in a “universal” approach to fitting GLLVMs, which remains competitive in terms of estimation and inferential performance relative to both standard VA (where any intractable integrals are either overcome through reparametrization or quadrature) and a Laplace approximation approach, while being computationally more scalable than both methods in practice.

On MCMC sampling in self-exciting integer-valued threshold time series models

Markov Chain Monte Carlo (MCMC) methods have been shown to be a useful tool in many branches in statistics. However, due to the complex structure of the models, this method remains an open problem for threshold integer-valued time series models. This study develops Bayesian inference for a class of self-exciting integer-valued threshold autoregressive models, which is implemented by means of a new MCMC algorithm. By introducing the latent variables series, a complete data likelihood is obtained. Based on which, the full conditional distributions are easily obtained with familiar forms. Furthermore, by maximizing the complete data likelihood, the threshold parameter is also accurately estimated. Finally, the performance of the MCMC algorithm is evaluated via some simulations and a real data example.

Misspecification of a binary dependent variable in the logistic model controlling for the repeated longitudinal measures

Many medical applications are interested to know the disease status. The disease status can be related to multiple serial measurements. Nevertheless, owing to various reasons, the binary outcome can be measured incorrectly. The estimators derived from the misspecified outcome can be biased. This paper derives the complete data likelihood function to incorporate both the multiple serial measurements and the misspecified outcome. Owing to the latent variables, EM algorithm is used to derive the maximum-likelihood estimators. Monte Carlo simulations are conducted to compare the impact of misspecification on the estimates. A retrospective data for the recurrence of atrial fibrillation is used to illustrate the usage of the proposed model.

Hypersphere Fitting From Noisy Data Using an EM Algorithm

This letter studies a new expectation maximization (EM) algorithm to solve the problem of circle, sphere and more generally hypersphere fitting. This algorithm relies on the introduction of random latent vectors having a priori independent von Mises-Fisher distributions defined on the hypersphere. This statistical model leads to a complete data likelihood whose expected value, conditioned on the observed data, has a Von Mises-Fisher distribution. As a result, the inference problem can be solved with a simple EM algorithm. The performance of the resulting hypersphere fitting algorithm is evaluated for circle and sphere fitting.

Spatial capture-recapture with multiple noninvasive marks: An application to camera-trapping data of the European wildcat (Felis silvestris) using R package multimark.

In Switzerland, the European wildcat (Felis silvestris), a native felid, is protected by national law. In recent decades, the wildcat has slowly returned to much of its original range and may have even expanded into new areas that were not known to be occupied before. For the implementation of efficient conservation actions, reliable information about the status and trend of population size and density is crucial. But so far, only one reliable estimate of density in Switzerland was produced in the northern Swiss Jura Mountains. Wildcats are relatively rare and elusive, but camera trapping has proven to be an effective method for monitoring felids. We developed and tested a monitoring protocol using camera trapping in the northern Jura Mountains (cantons of Bern and Jura) in an area of 100 km2. During 60 days, we obtained 105 pictures of phenotypical wildcats of which 98 were suitable for individual identification. We identified 13 individuals from both sides and, additionally, 5 single right‐sided flanks and 3 single left‐sided flanks that could not be matched to unique individuals. We analyzed the camera‐trap data using the R package multimark, which has been extended to include a novel spatial capture–recapture model for encounter histories that include multiple “noninvasive” marks, such as bilaterally asymmetrical left‐ and right‐sided flanks, that can be difficult (or impossible) to reliably match to individuals. Here, we present this model in detail for the first time. Based on a “semi‐complete” data likelihood, the model is less computationally demanding than Bayesian alternatives that rely on a data‐augmented complete data likelihood. The spatially explicit capture–recapture model estimated a wildcat density (95% credible interval) of 26 (17–36) per 100 km2 suitable habitat. Our integrated model produced higher abundance and density estimates with improved precision compared to single‐sided analyses, suggesting spatially explicit capture–recapture methods with multiple “noninvasive” marks can improve our ability to monitor wildcat population status.

Variational Bayes In Private Settings (VIPS)

Many applications of Bayesian data analysis involve sensitive information such as personal documents or medical records, motivating methods which ensure that privacy is protected. We introduce a general privacy-preserving framework for Variational Bayes (VB), a widely used optimization-based Bayesian inference method. Our framework respects differential privacy, the gold-standard privacy criterion, and encompasses a large class of probabilistic models, called the Conjugate Exponential (CE) family. We observe that we can straightforwardly privatise VB’s approximate posterior distributions for models in the CE family, by perturbing the expected sufficient statistics of the complete-data likelihood. For a broadly-used class of non-CE models, those with binomial likelihoods, we show how to bring such models into the CE family, such that inferences in the modified model resemble the private variational Bayes algorithm as closely as possible, using the Pólya-Gamma data augmentation scheme. The iterative nature of variational Bayes presents a further challenge since iterations increase the amount of noise needed. We overcome this by combining: (1) an improved composition method for differential privacy, called the moments accountant, which provides a tight bound on the privacy cost of multiple VB iterations and thus significantly decreases the amount of additive noise; and (2) the privacy amplification effect of subsampling mini-batches from large-scale data in stochastic learning. We empirically demonstrate the effectiveness of our method in CE and non-CE models including latent Dirichlet allocation, Bayesian logistic regression, and sigmoid belief networks, evaluated on real-world datasets.

Consequences of Model Misspecification for Maximum Likelihood Estimation with Missing Data

Researchers are often faced with the challenge of developing statistical models with incomplete data. Exacerbating this situation is the possibility that either the researcher’s complete-data model or the model of the missing-data mechanism is misspecified. In this article, we create a formal theoretical framework for developing statistical models and detecting model misspecification in the presence of incomplete data where maximum likelihood estimates are obtained by maximizing the observable-data likelihood function when the missing-data mechanism is assumed ignorable. First, we provide sufficient regularity conditions on the researcher’s complete-data model to characterize the asymptotic behavior of maximum likelihood estimates in the simultaneous presence of both missing data and model misspecification. These results are then used to derive robust hypothesis testing methods for possibly misspecified models in the presence of Missing at Random (MAR) or Missing Not at Random (MNAR) missing data. Second, we introduce a method for the detection of model misspecification in missing data problems using recently developed Generalized Information Matrix Tests (GIMT). Third, we identify regularity conditions for the Missing Information Principle (MIP) to hold in the presence of model misspecification so as to provide useful computational covariance matrix estimation formulas. Fourth, we provide regularity conditions that ensure the observable-data expected negative log-likelihood function is convex in the presence of partially observable data when the amount of missingness is sufficiently small and the complete-data likelihood is convex. Fifth, we show that when the researcher has correctly specified a complete-data model with a convex negative likelihood function and an ignorable missing-data mechanism, then its strict local minimizer is the true parameter value for the complete-data model when the amount of missingness is sufficiently small. Our results thus provide new robust estimation, inference, and specification analysis methods for developing statistical models with incomplete data.

Variable Selection for Mixed Data Clustering: Application in Human Population Genomics

Model-based clustering of human population genomic data, composed of 1,318 individuals arisen from western Central Africa and 160,470 markers, is considered. This challenging analysis leads us to develop a new methodology for variable selection in clustering. To explain the differences between subpopulations and to increase the accuracy of the estimates, variable selection is done simultaneously to clustering. We proposed two approaches for selecting variables when clustering is managed by the latent class model (i.e., mixture considering independence within components). The first method simultaneously performs model selection and parameter inference. It optimizes the Bayesian Information Criterion with a modified version of the standard expectation–maximization algorithm. The second method performs model selection without requiring parameter inference by maximizing the Maximum Integrated Complete-data Likelihood criterion. Although the application considers categorical data, the proposed methods are introduced in the general context of mixed data (data composed of different types of features). As the first step, the interest of both proposed methods is shown on simulated and several benchmark real data. Then, we apply the clustering method to the human population genomic data which permits to detect the most discriminative genetic markers. The proposed method implemented in the R package VarSelLCM is available on CRAN.

Pseudo MLE for semiparametric transformation model with doubly truncated data

In this article, we consider the efficient estimation of the semiparametric transformation model with doubly truncated data. We propose a two-step approach for obtaining the pseudo maximum likelihood estimators (PMLE) of regression parameters. In the first step, the truncation time distribution is estimated by the nonparametric maximum likelihood estimator (Shen, 2010a) when the distribution function K of the truncation time is unspecified or by the conditional maximum likelihood estimator (Bilker and Wang, 1996) when K is parameterized. In the second step, using the pseudo complete-data likelihood function with the estimated distribution of truncation time, we propose expectation–maximization algorithms for obtaining the PMLE. We establish the consistency of the PMLE. The simulation study indicates that the PMLE performs well in finite samples. The proposed method is illustrated using an AIDS data set.

On semiparametric transformation model with LTRC data: pseudo likelihood approach

When the distribution of the truncation time is known up to a finite-dimensional parameter vector, many researches have been conducted with the objective to improve the efficiency of estimation for nonparametric or semiparametric model with left-truncated and right-censored (LTRC) data. When the distribution of truncation times is unspecified, one approach is to use the conditional maximum likelihood estimators (cMLE) (Chen and Shen in Lifetime Data Anal https://doi.org/10.1007/s10985-016-9385-9, 2017). Although the cMLE has nice asymptotic properties, it is not efficient since the conditional likelihood function does not incorporate information on the distribution of truncation time. In this article, we aim to develop a more efficient estimator by considering the full likelihood function. Following Turnbull (J R Stat Soc B 38:290–295, 1976) and Qin et al. (J Am Stat Assoc 106:1434–1449, 2011), we treat the unobserved (left-truncated) subpopulation as missing data and propose a two-stage approach for obtaining the pseudo maximum likelihood estimators (PMLE) of regression parameters. In the first stage, the distribution of left truncation time is estimated by the inverse-probability-weighted (IPW) estimator (Wang in J Am Stat Assoc 86:130–143, 1991). In the second stage, we obtain the pseudo complete-data likelihood function by replacing the distribution of truncation time with the IPW estimator in the full likelihood. We propose an expectation–maximization algorithm for obtaining the PMLE and establish the consistency of the PMLE. Simulation results show that the PMLE outperforms the cMLE in terms of mean squared error. The PMLE can also be used to analyze the length-biased data, where the truncation time is uniformly distributed. We demonstrate that the PMLE works more robust against the support assumption of truncation time for length-biased data compared with the MLE proposed by Qin et al. (2011). We apply our proposed method to the channing house data. While the PMLE is quite appealing under specific cases with independent censoring and time-invariant covariates, its applicability, as shown in simulation study, can be rather restricted for more general settings.

Community Detection in Signed Networks Based on the Signed Stochastic Block Model and Exact ICL

There has been an increasing interest in detecting community in signed networks because signed networks contain more information (both positive and negative edges) than unsigned networks (only positive edges). Many methods have been proposed to find communities in signed networks; however, most of them can be regarded as the discriminative methods that do not concern with how the signed networks are generated so that they are usually difficult to characterize accurately the intrinsic community structure. The existing method SSL which is based on a generative model can achieve high accuracy in community detection in signed networks. However, SSL needs to estimate the models in the possible space one by one, which requires a large amount of calculation. In view of this, we propose a method to find community in signed networks, in which the exact integrated complete data likelihood (ICLex) for the signed stochastic block model proposed in SSL is derived and a greedy search is employed to optimize the value of the derived ICLex for signed networks to find communities. Our method has a rigorous probabilistic interpretation and does not need to estimate the models one by one from the possible space. The knowledge on hyper-parameters of our model is not necessary. In the experiments, the proposed method is tested on the synthetic and real-world networks and compared with several current methods. The experimental results show that our method can find the communities in signed networks more accurately than these current methods and more efficiently than SSL.

Likelihood Methods for Basic Stratified Sampling, with Application to Von Bertalanffy Growth Model Estimation

This paper mainly addresses maximum likelihood estimation for a response-selective stratified sampling scheme, the basic stratified sampling (BSS), in which the maximum subsample size in each stratum is fixed. We derived the complete-data likelihood for BSS, and extended it as a full-data likelihood by incorporating incomplete data. We also similarly extended the empirical proportion likelihood approach for consistent and efficient estimation. We conducted a simulation study to compare these two new approaches with the existing estimation methods in BSS. Our result indicates that they perform as well as the standard full information likelihood approach. Methods were illustrated using a growth model for fish size at age, including between-individual variability. One of our major conclusions is that the fully observed BSS data, the partially observed data used for stratification, and the sampling strategy are all important in constructing a consistent and efficient estimator.

FREQUENTIST INFERENCE IN INSURANCE RATEMAKING MODELS ADJUSTING FOR MISREPRESENTATION

AbstractIn insurance underwriting, misrepresentation represents the type of insurance fraud when an applicant purposely makes a false statement on a risk factor that may lower his or her cost of insurance. Under the insurance ratemaking context, we propose to use the expectation-maximization (EM) algorithm to perform maximum likelihood estimation of the regression effects and the prevalence of misrepresentation for the misrepresentation model proposed by Xia and Gustafson [(2016) The Canadian Journal of Statistics, 44, 198–218]. For applying the EM algorithm, the unobserved status of misrepresentation is treated as a latent variable in the complete-data likelihood function. We derive the iterative formulas for the EM algorithm and obtain the analytical form of the Fisher information matrix for frequentist inference on the parameters of interest for lognormal losses. We implement the algorithm and demonstrate that valid inference can be obtained on the risk effect despite the unobserved status of misrepresentation. Applying the proposed algorithm, we perform a loss severity analysis with the Medical Expenditure Panel Survey data. The analysis reveals not only the potential impact misrepresentation may have on the risk effect but also statistical evidence on the presence of misrepresentation in the self-reported insurance status.

Variable selection and prediction in biased samples with censored outcomes.

With the increasing availability of large prospective disease registries, scientists studying the course of chronic conditions often have access to multiple data sources, with each source generated based on its own entry conditions. The different entry conditions of the various registries may be explicitly based on the response process of interest, in which case the statistical analysis must recognize the unique truncation schemes. Moreover, intermittent assessment of individuals in the registries can lead to interval-censored times of interest. We consider the problem of selecting important prognostic biomarkers from a large set of candidates when the event times of interest are truncated and right- or interval-censored. Methods for penalized regression are adapted to handle truncation via a Turnbull-type complete data likelihood. An expectation-maximization algorithm is described which is empirically shown to perform well. Inverse probability weights are used to adjust for the selection bias when assessing predictive accuracy based on individuals whose event status is known at a time of interest. Application to the motivating study of the development of psoriatic arthritis in patients with psoriasis in both the psoriasis cohort and the psoriatic arthritis cohort illustrates the procedure.

Bayesian multiple change-point estimation for exponential distribution with truncated and censored data

ABSTRACTThis paper considers the multiple change-point estimation for exponential distribution with truncated and censored data by Gibbs sampling. After all the missing data of interest is filled in by some sampling methods such as rejection sampling method, the complete-data likelihood function is obtained. The full conditional distributions of all parameters are discussed. The means of Gibbs samples are taken as Bayesian estimations of the parameters. The implementation steps of Gibbs sampling are introduced in detail. Finally random simulation test is developed, and the results show that Bayesian estimations are fairly accurate.