Complex Hierarchical Models Research Articles

Sea-surge and wind speed extremes: optimal estimation strategies for planners and engineers

Accurate and precise estimation of return levels is often a key goal of any extreme value analysis. For example, in the UK the British Standards Institution (BSI) incorporate estimates of ‘once-in-50-year wind gust speeds’—or 50-year return levels—into their design codes for new structures; similarly, the Dutch Delta Commission use estimates of the 10,000-year return level for sea-surge to aid the construction of flood defence systems. In this paper, we briefly highlight the shortcomings of standard methods for estimating return levels, including the commonly-adopted block maxima and peaks over thresholds approach, before presenting an estimation framework which we show can substantially increase the precision of return level estimates. Our work allows explicit quantification of seasonal effects, as well as exploiting recent developments in the estimation of the extremal index for handling extremal clustering. From frequentist ideas, we turn to the Bayesian paradigm as a natural approach for building complex hierarchical or spatial models for extremes. Through simulations we show that the return level posterior mean does not have an exceedance probability in line with the intended encounter risk; we also argue that the Bayesian posterior predictive value gives the most satisfactory representation of a return level for use in practice, accounting for uncertainty in parameter estimation and future observations. Thus, where feasible, we propose a Bayesian estimation strategy for optimal return level inference.

A statistical framework to explore ontogenetic growth variation among individuals and populations: a marine fish example

Growth is a fundamental biological process, driven by a multitude of intrinsic (within‐individual) and extrinsic (environmental) factors, that underpins individual fitness and population demographics. Focusing on the comprehensive information stored in aquatic and terrestrial organism hard parts, we develop a series of increasingly complex hierarchical models to explore spatial and temporal sources of growth variation, ranging in resolution from within individuals to across a species. We apply this modeling framework to an extensive data set of otolith increment measurements from tiger flathead (Platycephalus richardsoni), a demersal commercially exploited fish that inhabits the warming waters of southeast Australia. We recreated growth histories (biochronology) up to four decades in length from seven fishing areas spanning this species' range. The dominant pattern in annual growth was an age‐dependent, allometric decline that varied among individuals, sexes, fishing areas, years, and cohorts. We found evidence for among‐area differences in growth‐rate selectivity, whereby younger fish at capture were generally faster growers. Temporal growth variation was partitioned into two main sources: extrinsic year to year annual fluctuations in environmental conditions and persistent cohort‐specific growth differences, reflecting density dependence and/or juvenile experience. Despite low levels of among‐individual growth synchrony within areas, we detected a regionally coherent signal of increasing average growth rate through time, a trend related to oceanic warming. At the southerly (poleward) range limit, growth was only weakly related to temperature, but farther north in warmer waters this relationship strengthened until closer to the species' equatorward range limit, growth declined with increasing temperatures. We partitioned these species‐wide and area‐specific phenotypic responses into within‐ and among‐individual components using a reaction norm approach. Individual tiger flathead likely possess sufficient growth plasticity to successfully adapt to warming waters across much of their range, but increased future warming in the north will continue to depress growth, affecting individual fitness and even population persistence. Our modeling framework is directly applicable to other long‐term, individual‐based, data sets such as those derived from tree rings, corals, and tag–recapture studies, and provides an unprecedented level of resolution into the drivers of growth variation and the ecological and evolutionary implications of environmental and climatic change.

Hierarchical models of warfare

Modern trends in the design of complex hierarchical models of warfare are discussed. First, we briefly navigate across well-known mathematical models of warfare (including descriptive, simulation, optimization and game-theoretic ones). Second, two canonical models (typical "examples") and their extensions, namely, Lanchester's models and colonel Blotto games, are considered in a greater detail. Finally, the hierarchical approach to warfare modeling is analyzed.

Variational Bayesian Learning of Probabilistic Discriminative Models With Latent Softmax Variables

This paper presents new variational Bayes (VB) approximations for learning probabilistic discriminative models with latent softmax variables, such as subclass-based multimodal softmax and mixture of experts models. The VB approximations derived here lead to closed-form approximate parameter posteriors and suitable metrics for model selection. Unlike other Bayesian methods for this challenging class of models, the proposed VB methods require neither restrictive structural assumptions nor sampling approximations to cope with the problematic softmax function. As such, the proposed VB methods are also easily extendable to more complex softmax-based hierarchical discriminative models and regression models (for continuous outputs). The proposed VB methods are evaluated on benchmark classification data and a decision modeling application, demonstrating good results.

Evaluating the Connection-Set Algebra for the neural simulator NEST

Event Abstract Back to Event Evaluating the Connection-Set Algebra for the neural simulator NEST Jochen Martin Eppler1*, Håkon Enger2, Thomas Heiberg2, Birgit Kriener2, Hans Ekkehard Plesser2, Markus Diesmann1 and Mikael Djurfeldt3 1 Research Center Juelich, Institute of Neuroscience and Medicine (INM-6), Germany 2 Norwegian University of Life Sciences, Mathematical Sciences and Technology, Norway 3 KTH, PDC Center for High Performance Computing, Sweden We present our ongoing work to evaluate the usability and efficiency of an implementation of the Connection Set Algebra (http://software.incf.org/software/csa) [1] for the neural simulation tool NEST (http://www.nest-initiative.org) [2]. Present day simulation languages work well for simple network structures like random or feed-forward networks. However, they either lack the required expressiveness or the required computational efficiency to specify the connectivity of complex hierarchical network models now under investigation. This increases the risk of errors in the model description or prevents the investigation of relevant model sizes. Therefore, a description language in the problem domain of the neuroscientist is required which guides the researcher to correct model specifications and avoids technical clutter. The Connection Set Algebra (CSA) is a generic framework, which provides elementary connection sets and operators for combining them. It allows one to define network connectivity in a compact and accessible format. A prototype implementation of the CSA exists in form of a Python module (libcsa). NEST is a simulator for large heterogeneous networks of point neurons or neurons with a small number of compartments. The concrete goals of this project are: Reduce the complexity of NEST's native connection routines through the use of CSA. Base the topology module of NEST on CSA in order to obtain better scaling, especially for very large networks on HPC facilities. Create a C++ implementation of CSA (libcsa) that can be used by neural simulators to describe network connectivity. The result of this study is a test implementation and a quantitative assessment of the proposed technology. Acknowledgments Partially funded by "The Next-Generation Integrated Simulation of Living Matter" project, part of the Development and Use of the Next-Generation Supercomputer Project of the Ministry of Education, Culture, Sports, Science and Technology (MEXT) of Japan, the Helmholtz Alliance on Systems Biology, the Research Council of Norway and the International Neuroinformatics Coordinating Facility.

Dclone: Data Cloning in R

Abstract The dclone R package contains lowlevel functions for implementing maximum like-lihood estimating procedures for complex mod-els using data cloning and Bayesian MarkovChain Monte Carlo methods with support forJAGS, WinBUGS and OpenBUGS. Introduction Hierarchical models, including generalized linearmodels with mixed random and fixed effects, areincreasingly popular. The rapid expansion of ap-plications is largely due to the advancement of theMarkov Chain Monte Carlo (MCMC) algorithms andrelated software (Gelman et al.,2003;Gilks et al.,1996;Lunn et al.,2009). Data cloning is a statisticalcomputing method introduced byLele et al.(2007). Itexploits the computational simplicity of the MCMCalgorithms used in the Bayesian statistical frame-work, but it provides the maximum likelihood pointestimates and their standard errors for complex hi-erarchical models. The use of the data cloning al-gorithm is especially valuable for complex models,where the number of unknowns increases with sam-ple size (i.e. with latent variables), because inferenceand prediction procedures are often hard to imple-ment in such situations.The

Identifiability of parameters and behaviour of MCMC chains: a case study using the reaction norm model

Markov chain Monte Carlo (MCMC) enables fitting complex hierarchical models that may adequately reflect the process of data generation. Some of these models may contain more parameters than can be uniquely inferred from the distribution of the data, causing non-identifiability. The reaction norm model with unknown covariates (RNUC) is a model in which unknown environmental effects can be inferred jointly with the remaining parameters. The problem of identifiability of parameters at the level of the likelihood and the associated behaviour of MCMC chains were discussed using the RNUC as an example. It was shown theoretically that when environmental effects (covariates) are considered as random effects, estimable functions of the fixed effects, (co)variance components and genetic effects are identifiable as well as the environmental effects. When the environmental effects are treated as fixed and there are other fixed factors in the model, the contrasts involving environmental effects, the variance of environmental sensitivities (genetic slopes) and the residual variance are the only identifiable parameters. These different identifiability scenarios were generated by changing the formulation of the model and the structure of the data and the models were then implemented via MCMC. The output of MCMC sampling schemes was interpreted in the light of the theoretical findings. The erratic behaviour of the MCMC chains was shown to be associated with identifiability problems in the likelihood, despite propriety of posterior distributions, achieved by arbitrarily chosen uniform (bounded) priors. In some cases, very long chains were needed before the pattern of behaviour of the chain may signal the existence of problems. The paper serves as a warning concerning the implementation of complex models where identifiability problems can be difficult to detect a priori. We conclude that it would be good practice to experiment with a proposed model and to understand its features before embarking on a full MCMC implementation.

Efficiency of alternative MCMC strategies illustrated using the reaction norm model

The Markov chain Monte Carlo (MCMC) strategy provides remarkable flexibility for fitting complex hierarchical models. However, when parameters are highly correlated in their posterior distributions and their number is large, a particular MCMC algorithm may perform poorly and the resulting inferences may be affected. The objective of this study was to compare the efficiency (in terms of the asymptotic variance of features of posterior distributions of chosen parameters, and in terms of computing cost) of six MCMC strategies to sample parameters using simulated data generated with a reaction norm model with unknown covariates as an example. The six strategies are single-site Gibbs updates (SG), single-site Gibbs sampler for updating transformed (a priori independent) additive genetic values (TSG), pairwise Gibbs updates (PG), blocked (all location parameters are updated jointly) Gibbs updates (BG), Langevin-Hastings (LH) proposals, and finally Langevin-Hastings proposals for updating transformed additive genetic values (TLH). The ranking of the methods in terms of asymptotic variance is affected by the degree of the correlation structure of the data and by the true values of the parameters, and no method comes out as an overall winner across all parameters. TSG and BG show very good performance in terms of asymptotic variance especially when the posterior correlation between genetic effects is high. In terms of computing cost, TSG performs best except for dispersion parameters in the low correlation scenario where SG was the best strategy. The two LH proposals could not compete with any of the Gibbs sampling algorithms. In this study it was not possible to find an MCMC strategy that performs optimally across the range of target distributions and across all possible values of parameters. However, when the posterior correlation between parameters is high, TSG, BG and even PG show better mixing than SG.

Laplace Expansions in Markov chain Monte Carlo Algorithms

Complex hierarchical models lead to a complicated likelihood and then, in a Bayesian analysis, to complicated posterior distributions. To obtain Bayes estimates such as the posterior mean or Bayesian confidence regions, it is therefore necessary to simulate the posterior distribution using a method such as an MCMC algorithm. These algorithms often get slower as the number of observations increases, especially when the latent variables are considered. To improve the convergence of the algorithm, we propose to decrease the number of parameters to simulate at each iteration by using a Laplace approximation on the nuisance parameters. We provide a theoretical study of the impact that such an approximation has on the target posterior distribution. We prove that the distance between the true target distribution and the approximation becomes of order O(N−a) with a ∈ (0, 1), a close to 1, as the number of observations N increases. A simulation study illustrates the theoretical results. The approximated MCMC algorithm behaves extremely well on an example which is driven by a study on HIV patients.

Approximate cross-validatory predictive checks in disease mapping models.

When fitting complex hierarchical disease mapping models, it can be important to identify regions that diverge from the assumed model. Since full leave-one-out cross-validatory assessment is extremely time-consuming when using Markov chain Monte Carlo (MCMC) estimation methods, Stern and Cressie consider an importance sampling approximation. We show that this can be improved upon through replication of both random effects and data. Our approach is simple to apply, entirely generic, and may aid the criticism of any Bayesian hierarchical model.

Bayesian model assessment and comparison using cross-validation predictive densities

In this work, we discuss practical methods for the assessment, comparison, and selection of complex hierarchical Bayesian models. A natural way to assess the goodness of the model is to estimate it...

Semiparametric Bayesian models for human brain mapping

Functional magnetic resonance imaging (fMRI) has led to enormous progress in human brain mapping. Adequate analysis of the massive spatiotemporal data sets generated by this imaging technique, combining parametric and non-parametric components, imposes challenging problems in statistical modelling. Complex hierarchical Bayesian models in combination with computer-intensive Markov chain Monte Carlo inference are promising tools. The purpose of this paper is twofold. First, it provides a review of general semiparametric Bayesian models for the analysis of fMRI data. Most approaches focus on important but separate temporal or spatial aspects of the overall problem, or they proceed by stepwise procedures. Therefore, as a second aim, we suggest a complete spatiotemporal model for analysing fMRI data within a unified semiparametric Bayesian framework. An application to data from a visual stimulation experiment illustrates our approach and demonstrates its computational feasibility.

Bayesian Measures of Model Complexity and Fit

SummaryWe consider the problem of comparing complex hierarchical models in which the number of parameters is not clearly defined. Using an information theoretic argument we derive a measure pD for the effective number of parameters in a model as the difference between the posterior mean of the deviance and the deviance at the posterior means of the parameters of interest. In general pD approximately corresponds to the trace of the product of Fisher's information and the posterior covariance, which in normal models is the trace of the ‘hat’ matrix projecting observations onto fitted values. Its properties in exponential families are explored. The posterior mean deviance is suggested as a Bayesian measure of fit or adequacy, and the contributions of individual observations to the fit and complexity can give rise to a diagnostic plot of deviance residuals against leverages. Adding pD to the posterior mean deviance gives a deviance information criterion for comparing models, which is related to other information criteria and has an approximate decision theoretic justification. The procedure is illustrated in some examples, and comparisons are drawn with alternative Bayesian and classical proposals. Throughout it is emphasized that the quantities required are trivial to compute in a Markov chain Monte Carlo analysis.

Spatial analysis of disease.

In this chapter, we have reviewed the history of the spatial analysis of disease and the statistical methods used for the exploratory analysis, testing and modeling of spatial patterns. In the next chapter, the principles described here will be illustrated.

Empirical Bayes Gibbs sampling.

The wide applicability of Gibbs sampling has increased the use of more complex and multi-level hierarchical models. To use these models entails dealing with hyperparameters in the deeper levels of a hierarchy. There are three typical methods for dealing with these hyperparameters: specify them, estimate them, or use a 'flat' prior. Each of these strategies has its own associated problems. In this paper, using an empirical Bayes approach, we show how the hyperparameters can be estimated in a way that is both computationally feasible and statistically valid.

A modeling tool for biomedical systems

This paper describes the tool P ansym for biomedical system modeling which has been designed primarily for representation of kinetics, transport and metabolism of biological substances. New modeling tools are needed in this research field because available simulation environments are often limited when applied to multidisciplinary studies in biomedicine. The aim of the new software is to allow a flexible specification of system structures using concurrently different formalisms that are conventionally used in different modeling domains. Starting from an intuitive model specification, mathematical system equations are first derived in symbolic form and subsequently coded as source code for different target applications. This multidomain modeling environment was built using bond graphs as common basis for automated analysis of system structures and symbolic derivation of system equations. Preliminary experience with the software showed that the adopted design strategy comes up to expectations, especially as regards the definition of complex hierarchical models that exhibit interactions between heterogeneous subsystems, e.g. cardiovascular and metabolic functions. Moreover, the specification of model structures is close to graphical object representation which helps defining a model of a system more in terms of its structure rather than its equations.

Predicting Symptom Return from Rate of Symptom Reduction in Cognitive–Behavior Therapy for Depression

Numerous studies have examined rates of symptom reduction and symptom return across treatment modalities; however, few studies have investigated the degree to which rate of symptom reduction during treatment is related to symptom return following treatment. We examined the relation between symptom return 3 and 6 months after completing treatment and several measures of symptom reduction during treatment, including amount of symptom reduction early in treatment, rates of symptom reduction over different phases of treatment, as well as the number of weeks in which individuals were asymptomatic. Rate of symptom reduction in the first 10 weeks of treatment was a stronger predictor of symptom return at 3 and 6 months after treatment than (a) pretreatment depressive severity, (b) symptom reduction after the first two sessions of treatment, (c) symptom reduction over all 20 weeks of treatment, and (d) number of weeks in which individuals were asymptomatic. Results also showed that severity of depressive thoughts at pretreatment was one of the only predictors of rate of symptom reduction during treatment and that estimating rates of symptom reduction simply by summing weekly BDI scores was as efficient at predicting symptom return as estimating rate of symptom reduction with more complex hierarchical linear regression models. Results of this research provide researchers and clinicians with new ways of measuring symptom reduction as well as the means of identifying individuals, before treatment has ended, who are likely to experience symptom return.

Some Algebra and Geometry for Hierarchical Models, Applied to Diagnostics

Summary Recent advances in computing make it practical to use complex hierarchical models. However, the complexity makes it difficult to see how features of the data determine the fitted model. This paper describes an approach to diagnostics for hierarchical models, specifically linear hierarchical models with additive normal or t-errors. The key is to express hierarchical models in the form of ordinary linear models by adding artificial ‘cases' to the data set corresponding to the higher levels of the hierarchy. The error term of this linear model is not homoscedastic, but its covariance structure is much simpler than that usually used in variance component or random effects models. The re-expression has several advantages. First, it is extremely general, covering dynamic linear models, random effect and mixed effect models, and pairwise difference models, among others. Second, it makes more explicit the geometry of hierarchical models, by analogy with the geometry of linear models. Third, the analogy with linear models provides a rich source of ideas for diagnostics for all the parts of hierarchical models. This paper gives diagnostics to examine candidate added variables, transformations, collinearity, case influence and residuals.

AN ALGORITHMIC APPROACH TO BAYESIAN LINEAR MODEL CALCULATIONS

SummaryA framework is described for organizing and understanding the computations necessary to obtain the posterior mean of a vector of linear effects in a normal linear model, conditional on the parameters that determine covariance structure. The approach has two major uses; firstly, as a pedagogical tool in the derivation of formulae, and secondly, as a practical tool for developing computational strategies without needing complicated matrix formulae that are often unwieldy in complex hierarchical models. The proposed technique is based upon symbolic application of the sweep operator SWP to an appropriate tableau of means and covariances. The method is illustrated with standard linear model specifications, including the so‐called mixed model, with both fixed and random effects.

Outlier Models and Prior Distributions in Bayesian Linear Regression

SUMMARY Bayesian inference in regression models is considered using heavy-tailed error distributions to accommodate outliers. The particular class of distributions that can be constructed as scale mixtures of normal distributions are examined and use is made of them as both error models and prior distributions in Bayesian linear modelling, including simple regression and more complex hierarchical models with structured priors depending on unknown hyperprior parameters.