Bayesian Extensions Research Articles

Beyond the single-outcome approach: A comparison of outcome-wide analysis methods for exposome research

Outcome-wide analysis can offer several benefits, including increased power to detect weak signals and the ability to identify exposures with multiple effects on health, which may be good targets for preventive measures. Recently, advanced statistical multivariate techniques for outcome-wide analysis have been developed, but they have been rarely applied to exposome analysis. In this work, we provide an overview of a selection of methods that are well-suited for outcome-wide exposome analysis and are implemented in the R statistical software. Our work brings together six different methods presenting innovative solutions for typical problems arising from outcome-wide approaches in the context of the exposome, including dependencies among outcomes, high dimensionality, mixed-type outcomes, missing data records, and confounding effects. The identified methods can be grouped into four main categories: regularized multivariate regression techniques, multi-task learning approaches, dimensionality reduction approaches, and bayesian extensions of the multivariate regression framework. Here, we compare each technique presenting its main rationale, strengths, and limitations, and provide codes and guidelines for their application to exposome data. Additionally, we apply all selected methods to a real exposome dataset from the Human Early-Life Exposome (HELIX) project, demonstrating their suitability for exposome research. Although the choice of the best method will always depend on the challenges to be faced in each application, for an exposome-like analysis we find dimensionality reduction and bayesian methods such as reduced rank regression (RRR) or multivariate bayesian shrinkage priors (MBSP) particularly useful, given their ability to deal with critical issues such as collinearity, high-dimensionality, missing data or quantification of uncertainty.

EXTENDING THE LEE–CARTER MODEL WITH VARIATIONAL AUTOENCODER: A FUSION OF NEURAL NETWORK AND BAYESIAN APPROACH

AbstractIn this study, we propose a nonlinear Bayesian extension of the Lee–Carter (LC) model using a single-stage procedure with a dimensionality reduction neural network (NN). LC is originally estimated using a two-stage procedure: dimensionality reduction of data by singular value decomposition followed by a time series model fitting. To address the limitations of LC, which are attributed to the two-stage estimation and insufficient model fitness to data, single-stage procedures using the Bayesian state-space (BSS) approaches and extensions of flexibility in modeling by NNs have been proposed. As a fusion of these two approaches, we propose a NN extension of LC with a variational autoencoder that performs the variational Bayesian estimation of a state-space model and dimensionality reduction by autoencoding. Despite being a NN model that performs single-stage estimation of parameters, our model has excellent interpretability and the ability to forecast with confidence intervals, as with the BSS models, without using Markov chain Monte Carlo methods.

Adapting the listening time for micro-electrode recordings in deep brain stimulation interventions.

Deep brain stimulation (DBS) is a common treatment for a variety of neurological disorders which involves the precise placement of electrodes at particular subcortical locations such as the subthalamic nucleus. This placement is often guided by auditory analysis of micro-electrode recordings (MERs) which informs the clinical team as to the anatomic region in which the electrode is currently positioned. Recent automation attempts have lacked flexibility in terms of the amount of signal recorded, not allowing them to collect more signal when higher certainty is needed or less when the anatomy is unambiguous. We have addressed this problem by evaluating a simple algorithm that allows for MER signal collection to terminate once the underlying model has sufficient confidence. We have parameterized this approach and explored its performance using three underlying models composed of one neural network and two Bayesian extensions of said network. We have shown that one particular configuration, a Bayesian model of the underlying network's certainty, outperforms the others and is relatively insensitive to parameterization. Further investigation shows that this model also allows for signals to be classified earlier without increasing the error rate. We have presented a simple algorithm that records the confidence of an underlying neural network, thus allowing for MER data collection to be terminated early when sufficient confidence is reached. This has the potential to improve the efficiency of DBS electrode implantation by reducing the time required to identify anatomical structures using MERs.

If Mathematical Psychology Did Not Exist We Might Need to Invent It: A Comment on Theory Building in Psychology

It is commonplace, when discussing the subject of psychological theory, to write articles from the assumption that psychology differs from the physical sciences in that we have no theories that would support cumulative, incremental science. In this brief article I discuss one counterexample: Shepard’s law of generalization and the various Bayesian extensions that it inspired over the past 3 decades. Using Shepard’s law as a running example, I argue that psychological theory building is not a statistical problem, mathematical formalism is beneficial to theory, measurement and theory have a complex relationship, rewriting old theory can yield new insights, and theory growth can drive empirical work. Although I generally suggest that the tools of mathematical psychology are valuable to psychological theorists, I also comment on some limitations to this approach.

Bayesian extensions on Lasso and adaptive Lasso Tobit regressions

Since lasso method launched, a lot of applications and extensions were run on it which made it to become deeply widely used in various discipline. In this paper, we proposed the Scale Mixture of Normals mixing with Rayleigh (SMNR) distribution on their variances to represent the double exponential distribution. Hierarchical model formula have derived with Gibbs sampler for SMNR. The proposed models; Bayesian Tobit Adaptive Lasso (BTAL) and Bayesian Tobit Lasso (BTL) models are illustrated using simulation example and a real data example through the prediction accuracy using the estimated relative efficiency with different sample. This is the first work that discussed regularization regression models under SMNR.

A Bayesian framework to quantify survival uncertainty

Abstract Background Existing survival models do not model uncertainty in personalized (per-patient) prediction of survival, which is useful for treatment planning of cancers. Moreover, their restrictive modeling assumptions limit their accuracy of personalized survival estimation. For example, the Cox proportional hazard (CPH) model assumes a constant effect of each input feature on survival rate over time along with a linear combination of features. The multi-task logistic regression (MTLR) allows a smooth time-varying impact of each feature without relaxing the linearity assumption. Recently, neural-MTLR (N-MTLR) was proposed to relax the linearity assumption as well. We propose Bayesian (B-MTLR) and Bayesian neural network (BN-MTLR) frameworks for predicting survival that are more accurate than the previous methods and compute uncertainty for each personalized prediction. We demonstrate that including more granular information, such as PAM50 gene expressions instead of the PAM50 labels, both increases accuracy and decreases uncertainty in personalized survival prediction. Methods Using four-fold cross validation on the TCGA-BRCA we compared the accuracy of personalized prediction (C-index and IBS) for CPH, MTLR, N-MTLR, B-MTLR, and BN-MTLR using PAM50 labels and clinical variables. We also compared BN-MTLR models trained on PAM50 gene-set and clinical variables for decrease in uncertainty due to more granular inputs. Results The table below shows that for PAM50 labels and clinical variables the C-index is higher and the IBS is lower for B-MTLR and BN-MTLR compared to the existing techniques. Additionally, BN-MTLR gave a mean uncertainty of 0.12 using PAM50 gene expression as opposed to 0.22 using PAM50 labels across all patients. Table: 112P . Comparison of models Methods C-index mean (std. dev.) IBS mean (std. dev.) CPH 0.67 (0.10) 0.20 (0.07) MTLR 0.68 (0.06) 0.21 (0.06) N-MTLR 0.68 (0.02) 0.16 (0.04) B-MTLR 0.69 (0.05) 0.14 (0.02) BN-MTLR 0.70 (0.05) 0.10 (0.01) Conclusions The proposed Bayesian extensions of survival models give better mean personalized accuracy and allow computation of personalized uncertainty scores, which will pave the way for more informative models for treatment planning of cancers. Legal entity responsible for the study The authors. Funding Has not received any funding. Disclosure All authors have declared no conflicts of interest.

A Tutorial on Canonical Correlation Methods

Canonical correlation analysis is a family of multivariate statistical methods for the analysis of paired sets of variables. Since its proposition, canonical correlation analysis has, for instance, been extended to extract relations between two sets of variables when the sample size is insufficient in relation to the data dimensionality, when the relations have been considered to be non-linear, and when the dimensionality is too large for human interpretation. This tutorial explains the theory of canonical correlation analysis, including its regularised, kernel, and sparse variants. Additionally, the deep and Bayesian CCA extensions are briefly reviewed. Together with the numerical examples, this overview provides a coherent compendium on the applicability of the variants of canonical correlation analysis. By bringing together techniques for solving the optimisation problems, evaluating the statistical significance and generalisability of the canonical correlation model, and interpreting the relations, we hope that this article can serve as a hands-on tool for applying canonical correlation methods in data analysis.

TreeBUGS: An R package for hierarchical multinomial-processing-tree modeling

Multinomial processing tree (MPT) models are a class of measurement models that account for categorical data by assuming a finite number of underlying cognitive processes. Traditionally, data are aggregated across participants and analyzed under the assumption of independently and identically distributed observations. Hierarchical Bayesian extensions of MPT models explicitly account for participant heterogeneity by assuming that the individual parameters follow a continuous hierarchical distribution. We provide an accessible introduction to hierarchical MPT modeling and present the user-friendly and comprehensive R package TreeBUGS, which implements the two most important hierarchical MPT approaches for participant heterogeneity—the beta-MPT approach (Smith & Batchelder, Journal of Mathematical Psychology 54:167-183, 2010) and the latent-trait MPT approach (Klauer, Psychometrika 75:70-98, 2010). TreeBUGS reads standard MPT model files and obtains Markov-chain Monte Carlo samples that approximate the posterior distribution. The functionality and output are tailored to the specific needs of MPT modelers and provide tests for the homogeneity of items and participants, individual and group parameter estimates, fit statistics, and within- and between-subjects comparisons, as well as goodness-of-fit and summary plots. We also propose and implement novel statistical extensions to include continuous and discrete predictors (as either fixed or random effects) in the latent-trait MPT model.

The Bayesian Ontology Language $$\mathcal {BEL}$$ BEL

We introduce the new probabilistic description logic (DL) $$\mathcal {BEL} $$BEL, which extends the light-weight DL $$\mathcal {EL}$$EL with the possibility of expressing uncertainty about the validity of some knowledge. Contrary to other probabilistic DLs, $$\mathcal {BEL}$$BEL is designed to represent classical knowledge that depends on an uncertain context; that is, some of the knowledge may hold or not depending on the current situation. The probability distribution of these contexts is expressed by a Bayesian network (BN). We study different reasoning problems in $$\mathcal {BEL}$$BEL, providing tight complexity bounds for all of them. One particularly interesting property of our framework is that reasoning can be decoupled between the logical (i.e., $$\mathcal {EL}$$EL), and the probabilistic (i.e., the BN) components. We later generalize all the notions presented to introduce Bayesian extensions of arbitrary ontology languages. Using the decoupling property, we are able to provide tight complexity bounds for reasoning in the Bayesian extensions of many other DLs. We provide a detailed analysis of our formalism w.r.t. the assumptions made and compare it with the existing approaches.

Bayesian tests for composite alternative hypotheses in cross-tabulated data

We present a methodology for constructing significance tests for “difficult” composite alternative hypotheses that have no natural test statistic. We apply our methodology to construct exact tests for cross-tabulated data, and our motivating example is constructing a test for discovering Simpson’s Paradox. Our tests are Bayesian extensions of the likelihood ratio test; they are optimal with respect to the prior distribution and are also closely related to Bayes factors and Bayesian FDR controlling testing procedures.

Learning Quadratic Receptive Fields from Neural Responses to Natural Stimuli

Models of neural responses to stimuli with complex spatiotemporal correlation structure often assume that neurons are selective for only a small number of linear projections of a potentially high-dimensional input. In this review, we explore recent modeling approaches where the neural response depends on the quadratic form of the input rather than on its linear projection, that is, the neuron is sensitive to the local covariance structure of the signal preceding the spike. To infer this quadratic dependence in the presence of arbitrary (e.g., naturalistic) stimulus distribution, we review several inference methods, focusing in particular on two information theory-based approaches (maximization of stimulus energy and of noise entropy) and two likelihood-based approaches (Bayesian spike-triggered covariance and extensions of generalized linear models). We analyze the formal relationship between the likelihood-based and information-based approaches to demonstrate how they lead to consistent inference. We demonstrate the practical feasibility of these procedures by using model neurons responding to a flickering variance stimulus.

Estimating infant mortality in Colombia: some overdispersion modelling approaches

It is common to fit generalized linear models with binomial and Poisson responses, where the data show a variability that is greater than the theoretical variability assumed by the model. This phenomenon, known as overdispersion, may spoil inferences about the model by considering significant parameters associated with variables that have no significant effect on the dependent variable. This paper explains some methods to detect overdispersion and presents and evaluates three well-known methodologies that have shown their usefulness in correcting this problem, using random mean models, quasi-likelihood methods and a double exponential family. In addition, it proposes some new Bayesian model extensions that have proved their usefulness in correcting the overdispersion problem. Finally, using the information provided by the National Demographic and Health Survey 2005, the departmental factors that have an influence on the mortality of children under 5 years and female postnatal period screening are determined. Based on the results, extensions that generalize some of the aforementioned models are also proposed, and their use is motivated by the data set under study. The results conclude that the proposed overdispersion models provide a better statistical fit of the data.

Bayesian Spatio-Dynamic Modeling in Cell Motility Studies: Learning Nonlinear Taxic Fields Guiding the Immune Response

We develop and analyze models of the spatio-temporal organization of lymphocytes in the lymph nodes and spleen. The spatial dynamics of these immune system white blood cells are influenced by biochemical fields and represent key components of the overall immune response to vaccines and infections. A primary goal is to learn about the structure of these fields that fundamentally shape the immune response. We define dynamic models of single-cell motion involving nonparametric representations of scalar potential fields underlying the directional biochemical fields that guide cellular motion. Bayesian hierarchical extensions define multicellular models for aggregating models and data on colonies of cells. Analysis via customized Markov chain Monte Carlo methods leads to Bayesian inference on cell-specific and population parameters together with the underlying spatial fields. Our case study explores data from multiphoton intravital microscopy in lymph nodes of mice, and we use a number of visualization tools to summarize and compare posterior inferences on the three-dimensional taxic fields.

Survey of Period Variations of Superhumps in SU UMa-Type Dwarf Novae. II The Second Year (2009–2010)

Abstract Continued from Kato et al. (2009, PASJ, 61, S395), we collected the times of superhump maxima for 68 SU UMa-type dwarf novae, mainly observed during the 2009–2010 season. The newly obtained data confirmed the basic findings reported in Kato et al. (ibid.): the presence of stages A–C and the predominance of positive period derivatives during stage B in systems with superhump periods shorter than 0.07 d. There was a systematic difference in the period derivatives for the systems with superhump periods longer than 0.075 d between this study and Kato et al. (ibid.). We suggest that this difference was possibly caused by a relative lack of frequently outbursting SU UMa-type dwarf novae in this period regime in the present study. We recorded a strong beat phenomenon during the 2009 superoutburst of IY UMa. A close correlation between the beat period and the superhump period suggests that the changing angular velocity of the apsidal motion of the elliptical disk is responsible for the variation of the superhump periods. We also described three new WZ Sge-type objects with established early superhumps and one with likely early superhumps. We suggest that two systems, VX For and EL UMa, are WZ Sge-type dwarf novae with multiple rebrightenings. The $O-C$ variation in OT J213806.6$+$261957 suggests that the frequent absence of rebrightenings in very short-$P_{\rm orb}$ objects can be the result of a sustained superoutburst plateau at the epoch when usual SU UMa-type dwarf novae return to quiescence, preceding a rebrightening. We also present a formulation for a variety of Bayesian extensions to traditional period analyses.

Predicting EQ-5D Utility Scores from the Seattle Angina Questionnaire in Coronary Artery Disease

The Seattle Angina Questionnaire (SAQ), a descriptive quality of life instrument, is often used in coronary artery disease studies. In its current form, however, it cannot be used in economic evaluations. The investigators sought to create a mapping algorithm that would allow translation of SAQ scores into EQ-5D utility scores. Data from the Alberta Provincial Project for Outcome Assessment in Coronary Heart Disease (APPROACH) database were used to examine the relationship between scores in each of the 5 domains of the SAQ (physical limitation, anginal stability, anginal frequency, treatment satisfaction, and disease perception) and the EQ-5D utility score. The cohort was divided into 80% derivation and 20% validation sets. Mapping algorithms were developed using simple linear regression and Tobit models. To account for the skewed distribution of the EQ-5D scores and the presence of heteroscedasticity, Bayesian extensions were applied to each model by specifying a nonconstant variance for the error term. Model performance was assessed by comparing predicted and observed mean EQ-5D scores in the validation set, and the unadjusted R2. The cohort consisted of 1992 patients. The simple linear regression model had the best predictive performance, with an R2 of 0.38. The nonconstant variance term did not improve overall performance for any of the models. The linear regression model accurately estimated the mean EQ-5D score in the validation set (predicted score 0.81 v. observed score 0.81). Mean EQ-5D utility weights can be accurately estimated from the SAQ using a simple linear regression mapping algorithm.

Bayesian extensions to Diebold-Li term structure model

This paper proposes a statistical model to adjust, interpolate, and forecast the term structure of interest rates. The model is based on the extensions for the term structure model of interest rates proposed by Diebold and Li (2006), through a Bayesian estimation using Markov Chain Monte Carlo (MCMC). The proposed extensions involve the use of a more flexible parametric form for the yield curve, allowing all the parameters to vary in time using a structure of latent factors, and the addition of a stochastic volatility structure to control the presence of conditional heteroskedasticity observed in the interest rates.The Bayesian estimation enables the exact distribution of the estimators in finite samples, and as a by-product, the estimation enables obtaining the distribution of forecasts of the term structure of interest rates. Unlike some econometric models of term structure, the methodology developed does not require a pre-interpolation of the yield curve. The model is fitted to the daily data of the term structure of interest rates implicit in SWAP DI-PRÉ contracts traded in the Mercantile and Futures Exchange (BM&F) in Brazil. The results are compared with the other models in terms of fitting and forecasts.

Stochastic dynamics of stimulus encoding in schizophrenia: Theory, testing, and application

Cognitive processing among schizophrenia participants, entailing encoding of presenting stimulation into a format facilitating collateral processes (e.g., memory search), is examined in light of stochastic mathematical models of performance. Results implicate additional encoding operations (encoding subprocesses) as the source of schizophrenia encoding-process elongation. Convergent evidence for this inference, including that from auxiliary neuro-connectionist simulations, are brought forth. Developments from initial, fixed-parameter accounts include random-parameter mixtures, and their Bayesian extensions, formally mediating group-level results to assessment of individual performance. Outgrowths bear on model-selection methodology, according to coherence of group-level and individual-level model functioning (in part addressing the issue of “small-trial-sample model testing”); longitudinal monitoring of encoding-specific treatment response; evaluation of treatment-regimen efficacy with respect to encoding efficiency; and specification of times of measurement interest, in fMRI. The symptom significance of encoding elongation, strongly hinted at by model developments, along with a model-endowed window on exacerbating effects of stress, are drawn out.

HIERARCHICAL BAYESIAN METHODS TO MODEL HETEROGENEITY IN COW- AND HERD-LEVEL RELATIONSHIPS BETWEEN MILK PRODUCTION AND REPRODUCTION IN DAIRY COWS

Two of the most important broad classifications of phenotypes for successful dairy production are milk yield and fertility. The nature of the relationship between milk production and reproductive performance of dairy cows is uncertain due to conflicting results reported in many studies. A common deficiency in many such studies is an underappreciation of the dual dimension of the production-reproduction relationship, as defined by herd (random or u) level and cow (residual or e) level sources of (co)variation. Our overall hypothesis is that the e- and u- level relationships between milk production and reproduction in dairy cows are heterogeneous and depend upon various herd-related and management factors. Our objective is to develop hierarchical Bayesian extensions that capture heterogeneity in the relationships between traits by mixed effects modeling of u level and e level covariances between traits of interest. We specify a bivariate Bayesian model to jointly model two continuous traits and we apply a square-root free Cholesky decomposition to the variance-covariance matrices of the residuals (cow-level) and random effects (herd-level). As a result, the e- and u-level covariances among the traits are reparameterized into unconstrained and easily interpretable e- and u- regression parameters, respectively. These regression parameters specify the cow- and herd-level relationships, respectively, between the traits and can be easily modeled as functions of relevant fixed and random effects, thereby providing a mixed model extension of Pourahmadi’s method. We validate our method using a simulation study and apply it to data on 305-day milk yield and calving interval of Michigan dairy cows.

Bayesian approximations and extensions: Optimal decisions for small brains and possibly big ones too

We compared the performance of Bayesian learning strategies and approximations to such strategies, which are far less computationally demanding, in a setting requiring individuals to make binary decisions based on experience. Extending Bayesian updating schemes, we compared the different strategies while allowing for various implementations of memory and knowledge about the environment. The dynamics of the observable variables was modeled through basic probability distributions and convolution. This theoretical framework was applied to the problem of male fruit flies who have to decide which females they should court. Computer simulations indicated that, for most parameter values, approximations to the Bayesian strategy performed as well as the full Bayesian one. The linear approximation, reminiscent of the linear operator, was notably successful, and, without innate knowledge, the only successful learning strategy. Besides being less demanding in computation and thus realistic for small brains, the linear approximation was also successful at limited memory, which would translate into robustness in rapidly changing environments. Knowledge about the environment boosted the performance of the various learning strategies with maximal performance at large utilization of memory. Only for limited memory capacities, intermediate knowledge was most successful. We conclude that many animals may rely on algorithms that involve approximations rather than full Bayesian calculations because such approximations achieve high levels of performance with only a fraction of the computational requirements, in particular for extensions of Bayesian updating schemes, which can represent universal and realistic environments.

Bayesian bootstrap for proportional hazards models

We propose two Bayesian bootstrap extensions, the binomial and Poisson forms, for proportional hazards models. The binomial form Bayesian bootstrap is the limit of the posterior distribution with a beta process prior as the amount of the prior information vanishes, and thus can be considered as a default nonparametric Bayesian analysis. It is also the same as Lo's Bayesian bootstrap for censored data when covariates are absent. The Poisson form Bayesian bootstrap is equivalent to the Bayesian analysis with Cox's profile likelihood. When the baseline distribution is discrete, thus when the data set has many ties, simulation study suggests that the binomial form Bayesian bootstrap performs better than standard frequentist procedures in the frequentist sense. An advantage of the proposed Bayesian bootstrap procedures over the standard Bayesian analysis is conceptual and computational simplicity. Finally, it is shown that both Bayesian bootstrap posteriors are asymptotically equivalent to the sampling distribution of the maximum likelihood estimator.