Approximate Inference Procedures Research Articles

INVESTIGATING DIFFERENCES IN BRAIN FUNCTIONAL NETWORKS USING HIERARCHICAL COVARIATE-ADJUSTED INDEPENDENT COMPONENT ANALYSIS.

Human brains perform tasks via complex functional networks consisting of separated brain regions. A popular approach to characterize brain functional networks in fMRI studies is independent component analysis (ICA), which is a powerful method to reconstruct latent source signals from their linear mixtures. In many fMRI studies, an important goal is to investigate how brain functional networks change according to specific clinical and demographic variabilities. Existing ICA methods, however, cannot directly incorporate covariate effects in ICA decomposition. Heuristic post-ICA analysis to address this need can be inaccurate and inefficient. In this paper, we propose a hierarchical covariate-adjusted ICA (hc-ICA) model that provides a formal statistical framework for estimating covariate effects and testing differences between brain functional networks. Our method provides a more reliable and powerful statistical tool for evaluating group differences in brain functional networks while appropriately controlling for potential confounding factors. We present an analytically tractable EM algorithm to obtain maximum likelihood estimates of our model. We also develop a subspace-based approximate EM that runs significantly faster while retaining high accuracy. To test the differences in functional networks, we introduce a voxel-wise approximate inference procedure which eliminates the need of computationally expensive covariance matrix estimation and inversion. We demonstrate the advantages of our methods over the existing method via simulation studies. We apply our method to an fMRI study to investigate differences in brain functional networks associated with post-traumatic stress disorder (PTSD).

Exact Inferences for a Gamma Distribution

A random sample from a two-parameter gamma distribution is considered, and it is shown how exact inferences can be obtained for properties of the distribution. Kolmogorov tests based on the empirical cumulative-distribution function of the data are inverted to construct an exact confidence set for the two parameters. This can be used, for example, to construct exact confidence bands for the cumulative-distribution function of the gamma distribution, which also provide simultaneous inferences on the quantiles of the distribution. The exact confidence set can also provide confidence intervals for the individual parameters and for other functions of the parameters, such as the mean of the distribution. The new methodology is computationally straightforward and examples of its implementation are provided. Comparisons are made with standard approximate inference procedures that rely on asymptotic properties of maximum-likelihood estimates of the parameters and with a Bayesian simulation approach. The methodology can also be used to test whether a data set can be modeled with a gamma distribution and to test whether independent data sets can be modeled with a common gamma distribution.

Overlapping stochastic block models with application to the French political blogosphere

Complex systems in nature and in society are often represented as networks, describing the rich set of interactions between objects of interest. Many deterministic and probabilistic clustering methods have been developed to analyze such structures. Given a network, almost all of them partition the vertices into disjoint clusters, according to their connection profile. However, recent studies have shown that these techniques were too restrictive and that most of the existing networks contained overlapping clusters. To tackle this issue, we present in this paper the Overlapping Stochastic Block Model. Our approach allows the vertices to belong to multiple clusters, and, to some extent, generalizes the well-known Stochastic Block Model [Nowicki and Snijders (2001)]. We show that the model is generically identifiable within classes of equivalence and we propose an approximate inference procedure, based on global and local variational techniques. Using toy data sets as well as the French Political Blogosphere network and the transcriptional network of Saccharomyces cerevisiae, we compare our work with other approaches.

Layered Dynamic Textures

A novel video representation, the layered dynamic texture (LDT), is proposed. The LDT is a generative model, which represents a video as a collection of stochastic layers of different appearance and dynamics. Each layer is modeled as a temporal texture sampled from a different linear dynamical system. The LDT model includes these systems, a collection of hidden layer assignment variables (which control the assignment of pixels to layers), and a Markov random field prior on these variables (which encourages smooth segmentations). An EM algorithm is derived for maximum-likelihood estimation of the model parameters from a training video. It is shown that exact inference is intractable, a problem which is addressed by the introduction of two approximate inference procedures: a Gibbs sampler and a computationally efficient variational approximation. The trade-off between the quality of the two approximations and their complexity is studied experimentally. The ability of the LDT to segment videos into layers of coherent appearance and dynamics is also evaluated, on both synthetic and natural videos. These experiments show that the model possesses an ability to group regions of globally homogeneous, but locally heterogeneous, stochastic dynamics currently unparalleled in the literature.

Statistical inference for functions of the covariance matrix in the stationary Gaussian time-orthogonal principal components model

We consider inference for functions of the marginal covariance matrix under a class of stationary vector time series models, referred to as time-orthogonal principal components models. The main application which motivated this work involves the estimation of configurational entropy from molecular dynamics simulations in computational chemistry, where current methods of entropy estimation involve calculations based on the sample covariance matrix. The theoretical results we obtain provide a basis for approximate inference procedures, including confidence interval calculations for scalar quantities of interest; these results are applied to the molecular dynamics application, and some further applications are discussed briefly.

Penalized Spline Models for Functional Principal Component Analysis

SummaryWe propose an iterative estimation procedure for performing functional principal component analysis. The procedure aims at functional or longitudinal data where the repeated measurements from the same subject are correlated. An increasingly popular smoothing approach, penalized spline regression, is used to represent the mean function. This allows straightforward incorporation of covariates and simple implementation of approximate inference procedures for coefficients. For the handling of the within-subject correlation, we develop an iterative procedure which reduces the dependence between the repeated measurements that are made for the same subject. The resulting data after iteration are theoretically shown to be asymptotically equivalent (in probability) to a set of independent data. This suggests that the general theory of penalized spline regression that has been developed for independent data can also be applied to functional data. The effectiveness of the proposed procedure is demonstrated via a simulation study and an application to yeast cell cycle gene expression data.

Comparisons of approximate tail probabilities for the shape parameter of the gamma distribution

This article compares the accuracy of tail probabilities obtained by various approximate inference procedures for the shape parameter of the gamma distribution which has one or more nuisance parameters. The approximate inference procedures being studied are the asymptotic normality of the maximum likelihood estimator, the asymptotic normality of the signed square root of the likelihood ratio statistic, the saddle-point approximation using two different approximations of the conditional log-likelihood function and also the exact conditional log-likelihood function, and a method discussed in Fraser and Reid (1995). The first two methods have first-order accuracy whereas the last four methods have third-order accuracy. Results indicate that all the third-order methods outperformed the first-order methods, and the saddle-point approximation using the exact conditional log-likelihood function gives the most accurate approximations. However, in terms of complexity in calculations, the method by Fraser and Reid (1995) is the simplest of the third-order methods.

Two Taylor-series approximation methods for nonlinear mixed models

This paper considers a general class of nonlinear mixed models (NLMMs). In view of the often difficult numerical integrations involved in a full likelihood analysis of general NLMMs, we discuss two approximate inference procedures, both based on Taylor-series approximations to the integrated likelihood. The two methods differ in the Taylor-series expansion point for the random-effects parameters: the first method expands around zero (the expected value of the random effects) and the second around their empirical best linear unbiased predictors (EBLUPs). We present new algorithms for implementing both approximations and relate them to existing algorithms. Finally, we assess the performance of both methods through simulation, and conclude that the EBLUP-expansion method offers minor gains in accuracy over the zero-expansion method at the cost of higher computing times and instability.

Asymptotic Properties of the Estimators for Multivariate Components of Variance

Estimation of the covariance matrices in the multivariate balanced one-way random effect model is discussed. The rank of the between-group covariance matrix plays a large role in model building as well as in assessing asymptotic properties of the estimated covariance matrices. The restricted (residual) maximum likelihood estimators derived under a rank condition are considered. Asymptotic properties of the estimators are derived for a possibly incorrectly specified rank and under either the number of groups, the number of replicates, or both, tending to infinity. A higher order expansion covering various cases leads to a common approximate inference procedure which can be used in a wide range of practical situations. A simulation study is also presented.

Simple approximate inference procedures for the mean of the gammma distribution

A very simple procedure is provided for computing the approximate p—value of a one—sided test concerning the mean of a gamma distribution when both parameters are assumed unknown. No special tables are required, and the associated confidence intervals can also be easily constructed. Exact tests free of the shape parameter are not available, but the approximate procedure is shown by Monte Carlo simulation to provide good results over the useful range of parameter values and sample sizes

Approximate inferences in multiresponse regression analysis

SUMMARY We present an approximate inference procedure for the multiresponse regression model. This can be used to determine approximate confidence regions or intervals and to formulate diagnostic tools such as standardized residuals and influence measures. The overall approximation is based on approximations at two stages. At the first stage, the multiresponse regression model is replaced by a generalized least-squares model using the estimated variance-covariance matrix of the error terms. At the second stage, the nonlinear model functions are replaced by linear approximations.

Inference in regression problems based on order statistics

In the general linear model , the Best Linear Unbiased Estimates based on order statistics (O-BLUE) of the modelss parameters are given under general assumptions on the distribution F of , for complete and symmetrically censored samples. Large sample properties of the (O-BLUE) estimators of the parameters of the special cases of the simple linear regression model and the k-nsample situation are discussed. The estimators are found to be consistent, asymptotically efficient, and under mild conditions about the tails of the distribution F, are asymptotically normal. Approximate inference procedures concerning the model's parameters are proposed. Tables of the approximate critical values of the test statistics are given for complete and censored samples from six symmetric distributions including the normal distribution