Marginal Inference Research Articles

Pattern Mixture Models for Quantifying Missing Data Uncertainty in Longitudinal Invariance Testing

Many psychology applications assess measurement invariance of a construct (e.g., depression) over time. These applications are often characterized by few time points (e.g., 3), but high rates of dropout. Although such applications routinely assume that the dropout mechanism is ignorable, this assumption may not always be reasonable. In the presence of nonignorable dropout, fitting a conventional longitudinal factor model (LFM) to assess longitudinal measurement invariance can yield misleading inferences about the level of invariance, along with biased parameter estimates. In this article we develop pattern mixture longitudinal factor models (PM-LFMs) for quantifying uncertainty in longitudinal invariance testing due to an unknown, but potentially nonignorable, dropout mechanism. PM-LFMs are a kind of multiple group model wherein observed missingness patterns define groups, LFM parameters can differ across these pattern-groups subject to identification constraints, and marginal inference about longitudinal invariance is obtained by pooling across pattern-groups. When dropout is nonignorable, we demonstrate via simulation that conventional LFMs can indicate longitudinal noninvariance, even when invariance holds in the overall population; certain PM-LFMs are shown to ameliorate this problem. On the other hand, when dropout is ignorable, PM-LFMs are shown to provide results comparable to conventional LFMs. Additionally, we contrast PM-LFMs to a latent mixture approach for accommodating nonignorable dropout—wherein missingness patterns can differ across latent groups. In an empirical example assessing longitudinal invariance of a harsh parenting construct, we employ PM-LFMs to assess sensitivity of results to assumptions about nonignorable missingness. Software implementation and recommendations for practice are discussed.

In-database batch and query-time inference over probabilistic graphical models using UDA–GIST

To meet customers' pressing demands, enterprise database vendors have been pushing advanced analytical techniques into databases. Most major DBMSes use user-defined aggregates (UDAs), a data-driven operator, to implement analytical techniques in parallel. However, UDAs alone are not sufficient to implement statistical algorithms where most of the work is performed by iterative transitions over a large state that cannot be naively partitioned due to data dependency. Typically, this type of statistical algorithm requires pre-processing to set up the large state in the first place and demands post-processing after the statistical inference. This paper presents general iterative state transition (GIST), a new database operator for parallel iterative state transitions over large states. GIST receives a state constructed by a UDA and then performs rounds of transitions on the state until it converges. A final UDA performs post-processing and result extraction. We argue that the combination of UDA and GIST (UDA---GIST) unifies data-parallel and state-parallel processing in a single system, thus significantly extending the analytical capabilities of DBMSes. We exemplify the framework through two high-profile batch applications: cross-document coreference, image denoising and one query-time inference application: marginal inference queries over probabilistic knowledge graphs. The 3 applications use probabilistic graphical models, which encode complex relationships of different variables and are powerful for a wide range of problems. We show that the in-database framework allows us to tackle a 27 times larger problem than a scalable distributed solution for the first application and achieves 43 times speedup over the state-of-the-art for the second application. For the third application, we implement query-time inference using the UDA---GIST framework and apply over a probabilistic knowledge graph, achieving 10 times speedup over sequential inference. To the best of our knowledge, this is the first in-database query-time inference engine over large probabilistic knowledge base. We show that the UDA---GIST framework for data- and graph-parallel computations can support both batch and query-time inference efficiently in databases.

A corrected formulation for marginal inference derived from two-part mixed models for longitudinal semi-continuous data

For semi-continuous data which are a mixture of true zeros and continuously distributed positive values, the use of two-part mixed models provides a convenient modelling framework. However, deriving population-averaged (marginal) effects from such models is not always straightforward. Su et al. presented a model that provided convenient estimation of marginal effects for the logistic component of the two-part model but the specification of marginal effects for the continuous part of the model presented in that paper was based on an incorrect formulation. We present a corrected formulation and additionally explore the use of the two-part model for inferences on the overall marginal mean, which may be of more practical relevance in our application and more generally.

Statistical model for overdispersed count outcome with many zeros: An approach for direct marginal inference

Marginalised models are in great demand by many researchers in the life sciences, particularly in clinical trials, epidemiology, health-economics, surveys and many others, since they allow generalisation of inference to the entire population under study. For count data, standard procedures such as the Poisson regression and negative binomial model provide population average inference for model parameters. However, occurrence of excess zero counts and lack of independence in empirical data have necessitated their extension to accommodate these phenomena. These extensions, though useful, complicate interpretations of effects. For example, the zero-inflated Poisson model accounts for the presence of excess zeros, but the parameter estimates do not have a direct marginal inferential ability as the base model, the Poisson model. Marginalisations due to the presence of excess zeros are underdeveloped though demand for them is interestingly high. The aim of this paper, therefore, is to develop a marginalised model for zero-inflated univariate count outcome in the presence of overdispersion. Emphasis is placed on methodological development, efficient estimation of model parameters, implementation and application to two empirical studies. A simulation study is performed to assess the performance of the model. Results from the analysis of two case studies indicate that the refined procedure performs significantly better than models which do not simultaneously correct for overdispersion and presence of excess zero counts in terms of likelihood comparisons and AIC values. The simulation studies also supported these findings. In addition, the proposed technique yielded small biases and mean square errors for model parameters. To ensure that the proposed method enjoys widespread use, it is implemented using the SAS NLMIXED procedure with minimal coding efforts.

Scene Parsing With Integration of Parametric and Non-Parametric Models.

We adopt convolutional neural networks (CNNs) to be our parametric model to learn discriminative features and classifiers for local patch classification. Based on the occurrence frequency distribution of classes, an ensemble of CNNs (CNN-Ensemble) are learned, in which each CNN component focuses on learning different and complementary visual patterns. The local beliefs of pixels are output by CNN-Ensemble. Considering that visually similar pixels are indistinguishable under local context, we leverage the global scene semantics to alleviate the local ambiguity. The global scene constraint is mathematically achieved by adding a global energy term to the labeling energy function, and it is practically estimated in a non-parametric framework. A large margin-based CNN metric learning method is also proposed for better global belief estimation. In the end, the integration of local and global beliefs gives rise to the class likelihood of pixels, based on which maximum marginal inference is performed to generate the label prediction maps. Even without any post-processing, we achieve the state-of-the-art results on the challenging SiftFlow and Barcelona benchmarks.

Scaling-Up MAP and Marginal MAP Inference in Markov Logic

Markov Logic Networks (MLNs) use a few weighted first-order logic formulas to represent large probabilistic graphical models and are ideally suited for representing both relational and probabilistic knowledge in a wide variety of application domains such as, NLP, computer vision, and robotics. However, inference in them is hard because the graphical models can be extremely large, having millions of variables and features (potentials). Therefore, several lifted inference algorithms that exploit relational structure and operate at the compact first-order level, have been developed in recent years. However, the focus of much of existing research on lifted inference is on marginal inference while algorithms for MAP and marginal MAP inference are far less advanced. The aim of the proposed thesis is to fill this void, by developing next generation inference algorithms for MAP and marginal MAP inference.

Marginal Inferential Models: Prior-Free Probabilistic Inference on Interest Parameters

The inferential models (IM) framework provides prior-free, frequency-calibrated, and posterior probabilistic inference. The key is the use of random sets to predict unobservable auxiliary variables connected to the observable data and unknown parameters. When nuisance parameters are present, a marginalization step can reduce the dimension of the auxiliary variable which, in turn, leads to more efficient inference. For regular problems, exact marginalization can be achieved, and we give conditions for marginal IM validity. We show that our approach provides exact and efficient marginal inference in several challenging problems, including a many-normal-means problem. In nonregular problems, we propose a generalized marginalization technique and prove its validity. Details are given for two benchmark examples, namely, the Behrens–Fisher and gamma mean problems.

Fast algorithm based on superpixel‐level conditional triplet Markov field for successive‐approximation resistor image segmentation

Conditional random field (CRF) is a useful tool for optical and remote sensing image segmentation for its ability of incorporating the feature and texture information. However, its application is restricted in successive-approximation resistor (SAR) image segmentation, since SAR images often contain complex non-stationary contents. The triplet Markov field (TMF) model improves the non-stationary image segmentation ability by introducing an auxiliary field to characterise different stationary parts in non-stationary image. Combining the advantages of CRF and TMF, the pixel-level conditional TMF (CTMF) had been proposed. To further improve the segmentation efficiency, a superpixel-level CTMF (SL-CTMF) is proposed in this study. The superpixel representation comes from the improved TurboPixels algorithm and the superpixel representation has better performance in edge location. The auxiliary field U in SL-CTMF is reconstructed on superpixels. With the superpixel-level feature and texture information, the unary and pairwise potentials are derived. Finally, SL-CTMF is applied to real SAR image segmentation with the maximum posterior marginal inference. The experimental results demonstrate the accuracy and the efficiency of the proposed method on SAR images.

Direct and flexible marginal inference for semicontinuous data.

The marginalized two-part (MTP) model for semicontinuous data proposed by Smith et al. provides direct inference for the effect of covariates on the marginal mean of positively continuous data with zeros. This brief note addresses mischaracterizations of the MTP model by Gebregziabher et al. Additionally, the MTP model is extended to incorporate the three-parameter generalized gamma distribution, which takes many well-known distributions as special cases, including the Weibull, gamma, inverse gamma, and log-normal distributions.

Adaptive MCMC with online relabeling

When targeting a distribution that is artificially invariant under some permutations, Markov chain Monte Carlo (MCMC) algorithms face the label-switching problem, rendering marginal inference particularly cumbersome. Such a situation arises, for example, in the Bayesian analysis of finite mixture models. Adaptive MCMC algorithms such as adaptive Metropolis (AM), which self-calibrates its proposal distribution using an online estimate of the covariance matrix of the target, are no exception. To address the label-switching issue, relabeling algorithms associate a permutation to each MCMC sample, trying to obtain reasonable marginals. In the case of adaptive Metropolis (Bernoulli 7 (2001) 223-242), an online relabeling strategy is required. This paper is devoted to the AMOR algorithm, a provably consistent variant of AM that can cope with the label-switching problem. The idea is to nest relabeling steps within the MCMC algorithm based on the estimation of a single covariance matrix that is used both for adapting the covariance of the proposal distribution in the Metropolis algorithm step and for online relabeling. We compare the behavior of AMOR to similar relabeling methods. In the case of compactly supported target distributions, we prove a strong law of large numbers for AMOR and its ergodicity. These are the first results on the consistency of an online relabeling algorithm to our knowledge. The proof underlines latent relations between relabeling and vector quantization.

POME-copula for hydrological dependence analysis

Abstract. Hydrological multivariate analysis has been widely studied using copula-based modelling, in which marginal distribution inference is one of the key issues. The main object of this study is to discuss the applicability of the principle of maximum entropy (POME) in marginal distribution inference, thus to develop a POME-copula framework to analyse the dependence of hydrological variables. Marginal distributions are derived with the POME approach before bivariate copulas constructed with corresponding parameters estimated by the dependence of the derived margins. The proposed POME-copula has been employed in hydrological dependence analyses, with the annual maximum streamflow and water level collected from the Yangtze River, and the monthly streamflow from the Yellow River. Results show that the POME-copula method performs well in capturing dependence patterns of various hydrological variables.

A Marginalized Zero-inflated Poisson Regression Model with Random Effects.

Public health research often concerns relationships between exposures and correlated count outcomes. When counts exhibit more zeros than expected under Poisson sampling, the zero-inflated Poisson (ZIP) model with random effects may be used. However, the latent class formulation of the ZIP model can make marginal inference on the sampled population challenging. This article presents a marginalized ZIP model with random effects to directly model the mean of the mixture distribution consisting of 'susceptible' individuals and excess zeroes, providing straightforward inference for overall exposure effects. Simulations evaluate finite sample properties, and the new methods are applied to a motivational interviewing-based safer sex intervention trial, designed to reduce the number of unprotected sexual acts.

On the generalized multivariate Gumbel distribution

In this article, main characteristics, marginal, joint, and conditional inferences of a generalized multivariate Gumbel model are derived, and random vector generation is described. Distribution of the sum where summands come from a bivariate generalized multivariate Gumbel distribution is derived.

Weibull mixture regression for marginal inference in zero-heavy continuous outcomes.

Continuous outcomes with preponderance of zero values are ubiquitous in data that arise from biomedical studies, for example studies of addictive disorders. This is known to lead to violation of standard assumptions in parametric inference and enhances the risk of misleading conclusions unless managed properly. Two-part models are commonly used to deal with this problem. However, standard two-part models have limitations with respect to obtaining parameter estimates that have marginal interpretation of covariate effects which are important in many biomedical applications. Recently marginalized two-part models are proposed but their development is limited to log-normal and log-skew-normal distributions. Thus, in this paper, we propose a finite mixture approach, with Weibull mixture regression as a special case, to deal with the problem. We use extensive simulation study to assess the performance of the proposed model in finite samples and to make comparisons with other family of models via statistical information and mean squared error criteria. We demonstrate its application on real data from a randomized controlled trial of addictive disorders. Our results show that a two-component Weibull mixture model is preferred for modeling zero-heavy continuous data when the non-zero part are simulated from Weibull or similar distributions such as Gamma or truncated Gauss.

Frequentist inference on random effects based on summarizability

Although Bayesian methodologies have been successful in drawing inference about random effects, the frequentist literature has been limited. In this paper we consider inferences on random effects in hierarchical generalized linear models from a frequentist point of view using their summarizability. We show asymptotic distributional properties for the conditional and the marginal inference when the number of subunits is large. We conduct simulation studies when the number of subunits is small to moderate. A seizure study and an infertility study are used to illustrate the conditional and the marginal inference of random effects.

Marginalized multilevel hurdle and zero-inflated models for overdispersed and correlated count data with excess zeros.

Count data are collected repeatedly over time in many applications, such as biology, epidemiology, and public health. Such data are often characterized by the following three features. First, correlation due to the repeated measures is usually accounted for using subject-specific random effects, which are assumed to be normally distributed. Second, the sample variance may exceed the mean, and hence, the theoretical mean-variance relationship is violated, leading to overdispersion. This is usually allowed for based on a hierarchical approach, combining a Poisson model with gamma distributed random effects. Third, an excess of zeros beyond what standard count distributions can predict is often handled by either the hurdle or the zero-inflated model. A zero-inflated model assumes two processes as sources of zeros and combines a count distribution with a discrete point mass as a mixture, while the hurdle model separately handles zero observations and positive counts, where then a truncated-at-zero count distribution is used for the non-zero state. In practice, however, all these three features can appear simultaneously. Hence, a modeling framework that incorporates all three is necessary, and this presents challenges for the data analysis. Such models, when conditionally specified, will naturally have a subject-specific interpretation. However, adopting their purposefully modified marginalized versions leads to a direct marginal or population-averaged interpretation for parameter estimates of covariate effects, which is the primary interest in many applications. In this paper, we present a marginalized hurdle model and a marginalized zero-inflated model for correlated and overdispersed count data with excess zero observations and then illustrate these further with two case studies. The first dataset focuses on the Anopheles mosquito density around a hydroelectric dam, while adolescents' involvement in work, to earn money and support their families or themselves, is studied in the second example. Sub-models, which result from omitting zero-inflation and/or overdispersion features, are also considered for comparison's purpose. Analysis of the two datasets showed that accounting for the correlation, overdispersion, and excess zeros simultaneously resulted in a better fit to the data and, more importantly, that omission of any of them leads to incorrect marginal inference and erroneous conclusions about covariate effects.

Approximate Bayesian inference for spatial econometrics models

In this paper we explore the use of the Integrated Laplace Approximation (INLA) for Bayesian inference in some widely used models in Spatial Econometrics. Bayesian inference often relies on computationally intensive simulation methods, such as Markov Chain Monte Carlo. When only marginal inference is needed, INLA provides a fast and accurate estimate of the posterior marginals of the parameters in the model.Furthermore, we have compared the results provided by these models to those obtained with a more general class of Generalised Linear Models with random effects. In these models, spatial autocorrelation is modelled by means of correlated Gaussian random effects.We also discuss a procedure to extend the class of models that the R-INLA software can fit. This approach is based on conditioning on one or more parameters so that the resulting models can be fitted with R-INLA across sets of values of the fixed parameters. The posterior marginals of these parameters of interest are then obtained by combining the marginal likelihoods (which are conditioned on the values of the parameters fixed) of the fitted models and a prior on these parameters. This approach can also be used to fit even more general models.Finally, we discuss the use of all these models on two datasets based on median housing prices for census tracts in Boston and the probability of business re-opening in New Orleans in the aftermath of hurricane Katrina.

Bridging Conditional and Marginal Inference for Spatially Referenced Binary Data

Spatially referenced binary data are common in epidemiology and public health. Owing to its elegant log-odds interpretation of the regression coefficients, a natural model for these data is logistic regression. To account for missing confounding variables that might exhibit a spatial pattern (say, socioeconomic, biological, or environmental conditions), it is customary to include a Gaussian spatial random effect. Conditioned on the spatial random effect, the coefficients may be interpreted as log odds ratios. However, marginally over the random effects, the coefficients no longer preserve the log-odds interpretation, and the estimates are hard to interpret and generalize to other spatial regions. To resolve this issue, we propose a new spatial random effect distribution through a copula framework which ensures that the regression coefficients maintain the log-odds interpretation both conditional on and marginally over the spatial random effects. We present simulations to assess the robustness of our approach to various random effects, and apply it to an interesting dataset assessing periodontal health of Gullah-speaking African Americans. The proposed methodology is flexible enough to handle areal or geo-statistical datasets, and hierarchical models with multiple random intercepts.

Model Reductions for Inference: Generality of Pairwise, Binary, and Planar Factor Graphs

We offer a solution to the problem of efficiently translating algorithms between different types of discrete statistical model. We investigate the expressive power of three classes of model-those with binary variables, with pairwise factors, and with planar topology-as well as their four intersections. We formalize a notion of "simple reduction" for the problem of inferring marginal probabilities and consider whether it is possible to "simply reduce" marginal inference from general discrete factor graphs to factor graphs in each of these seven subclasses. We characterize the reducibility of each class, showing in particular that the class of binary pairwise factor graphs is able to simply reduce only positive models. We also exhibit a continuous "spectral reduction" based on polynomial interpolation, which overcomes this limitation. Experiments assess the performance of standard approximate inference algorithms on the outputs of our reductions.

Additive Transformation Models for Multivariate Interval-Censored Data

Tong, et al. (2008) considered multivariate (clustered) interval-censored failure time data that occur when there exist several correlated survival times of interest and only interval-censored data are available for each survival time. Assuming that covariates affect the hazard rate linearly, they developed a marginal inference approach using the additive hazards model. In this article, based on the idea of Zeng and Cai (2010), we consider a general class of additive transformation model, which relaxes the linear assumption. Using working independence likelihood, we present an inference approach for regression analysis of multivariate interval-censored data. A simulation study is conducted to investigate the performance of the proposed estimator. We apply the proposed method to the data set from the Diabetic Retinopathy Study.