Bayesian Latent Variable Model Research Articles

A Bayesian latent variable model for the optimal identification of disease incidence rates given information constraints

Abstract We present an original approach for measuring infections as a latent variable and making use of serological and expert surveys to provide ground truth identification during the early pandemic period. Compared to existing approaches, our model relies more on empirical information than strong structural forms, permitting inference with relatively few assumptions of cumulative infections. We also incorporate a range of political, economic, and social covariates to richly parameterize the relationship between epidemic spread and human behaviour. To show the utility of the model, we provide robust estimates of total infections that account for biases in COVID-19 cases and tests counts in the U.S. from March to July of 2020, a period of time when accurate data about the nature of the SARS-CoV-2 virus was of limited availability. In addition, we can show how sociopolitical factors like the Black Lives Matter protests and support for President Donald Trump are associated with the spread of the virus via changes in fear of the virus and cell phone mobility. A reproducible version of this article is available as an Rmarkdown file at https://github.com/CoronaNetDataScience/covid_model.

A Bayesian Spatial Model to Correct Under-Reporting in Urban Crowdsourcing

Decision-makers often observe the occurrence of events through a reporting process. City governments, for example, rely on resident reports to find and then resolve urban infrastructural problems such as fallen street trees, flooded basements, or rat infestations. Without additional assumptions, there is no way to distinguish events that occur but are not reported from events that truly did not occur--a fundamental problem in settings with positive-unlabeled data. Because disparities in reporting rates correlate with resident demographics, addressing incidents only on the basis of reports leads to systematic neglect in neighborhoods that are less likely to report events. We show how to overcome this challenge by leveraging the fact that events are spatially correlated. Our framework uses a Bayesian spatial latent variable model to infer event occurrence probabilities and applies it to storm-induced flooding reports in New York City, further pooling results across multiple storms. We show that a model accounting for under-reporting and spatial correlation predicts future reports more accurately than other models, and further induces a more equitable set of inspections: its allocations better reflect the population and provide equitable service to non-white, less traditionally educated, and lower-income residents. This finding reflects heterogeneous reporting behavior learned by the model: reporting rates are higher in Census tracts with higher populations, proportions of white residents, and proportions of owner-occupied households. Our work lays the groundwork for more equitable proactive government services, even with disparate reporting behavior.

Rarity: discovering rare cell populations from single-cell imaging data.

Cell type identification plays an important role in the analysis and interpretation of single-cell data and can be carried out via supervised or unsupervised clustering approaches. Supervised methods are best suited where we can list all cell types and their respective marker genes a priori, while unsupervised clustering algorithms look for groups of cells with similar expression properties. This property permits the identification of both known and unknown cell populations, making unsupervised methods suitable for discovery. Success is dependent on the relative strength of the expression signature of each group as well as the number of cells. Rare cell types therefore present a particular challenge that is magnified when they are defined by differentially expressing a small number of genes. Typical unsupervised approaches fail to identify such rare subpopulations, and these cells tend to be absorbed into more prevalent cell types. In order to balance these competing demands, we have developed a novel statistical framework for unsupervised clustering, named Rarity, that enables the discovery process for rare cell types to be more robust, consistent, and interpretable. We achieve this by devising a novel clustering method based on a Bayesian latent variable model in which we assign cells to inferred latent binary on/off expression profiles. This lets us achieve increased sensitivity to rare cell populations while also allowing us to control and interpret potential false positive discoveries. We systematically study the challenges associated with rare cell type identification and demonstrate the utility of Rarity on various IMC datasets. Implementation of Rarity together with examples is available from the Github repository (https://github.com/kasparmartens/rarity).

Opaque prior distributions in Bayesian latent variable models

We review common situations in Bayesian latent variable models where the prior distribution that a researcher specifies differs from the prior distribution used during estimation. These situations can arise from the positive definite requirement on correlation matrices, from sign indeterminacy of factor loadings, and from order constraints on threshold parameters. The issue is especially problematic for reproducibility and for model checks that involve prior distributions, including prior predictive assessment and Bayes factors. In these cases, one might be assessing the wrong model, casting doubt on the relevance of the results. The most straightforward solution to the issue sometimes involves use of informative prior distributions. We explore other solutions and make recommendations for practice.

Bayesian size-and-shape regression modelling

Building on Dryden et al. (2021), this note presents the Bayesian estimation of a regression model for size-and-shape response variables with Gaussian landmarks. Our proposal fits into the framework of Bayesian latent variable models and, potentially, allows for a highly flexible modelling framework.

Rain Pattern Deeply Reshaped Total Phosphorus Load Pattern in Watershed: A Case Study from Northern China

Excessive phosphorus in aquatic systems poses a threat to ecosystem stability and human health. Precipitation has a profound influence on the phosphorus biogeochemical process; however, it has been inadequately considered at the watershed scale. In this study, the Bayesian latent variable regression model was utilized to quantify the impact of rainfall on the concentration of total phosphorus using daily monitoring data from 2019 to 2021. The results revealed a piecewise linear relationship between total phosphorus concentration and precipitation. It was further inferred that the breakpoint (The total rainfall during a single rainfall event equal to 39.4 ± 0.45 mm) represented the tipping point for the transformation of the primary river runoff generation mechanism. Subsequently, the excess phosphorus load caused by rainfall events was estimated in the Shahe basin by combining the regional nutrient management approach with the results of the Bayesian latent variable regression model. The results indicated that rainfall events were one of the most significant sources of TP loads from 2006 to 2017, accounting for 28.2% of the total. Non-artificial land, including farmland, forests, and grasslands, serves as the primary source of the excess phosphorus load resulting from rainfall events. This study provides insights into how to quantify the phosphorus load resulting from rainfall events at the basin scale, which is valuable for phosphorus management. Environmental managers should prioritize the regulation of phosphorus in non-artificial land moving forward. Implementing hierarchical management based on calibrated curve numbers and soil phosphorus content could prove to be an efficient approach for regulating phosphorus in the watershed.

Bayesian Latent Variable Model of Mixed Correlated Rank and Beta-Binomial Responses with Missing Data for the International Statistical Literacy Project Poster Competition

Bayesian Latent Variable Model of Mixed Correlated Rank and Beta-Binomial Responses with Missing Data for the International Statistical Literacy Project Poster Competition

Real-time estimation and forecasting of COVID-19 cases and hospitalizations in Wisconsin HERC regions for public health decision making processes

BackgroundThe spread of the COVID-19 (SARS-CoV-2) and the surging number of cases across the United States have resulted in full hospitals and exhausted health care workers. Limited availability and questionable reliability of the data make outbreak prediction and resource planning difficult. Any estimates or forecasts are subject to high uncertainty and low accuracy to measure such components. The aim of this study is to apply, automate, and assess a Bayesian time series model for the real-time estimation and forecasting of COVID-19 cases and number of hospitalizations in Wisconsin healthcare emergency readiness coalition (HERC) regions.MethodsThis study makes use of the publicly available Wisconsin COVID-19 historical data by county. Cases and effective time-varying reproduction number R_t by the HERC region over time are estimated using Bayesian latent variable models. Hospitalizations are estimated by the HERC region over time using a Bayesian regression model. Cases, effective Rt, and hospitalizations are forecasted over a 1-day, 3-day, and 7-day time horizon using the last 28 days of data, and the 20%, 50%, and 90% Bayesian credible intervals of the forecasts are calculated. The frequentist coverage probability is compared to the Bayesian credible level to evaluate performance.ResultsFor cases and effective R_t, all three time horizons outperform the three credible levels of the forecast. For hospitalizations, all three time horizons outperform the 20% and 50% credible intervals of the forecast. On the contrary, the 1-day and 3-day periods underperform the 90% credible intervals. Questions about uncertainty quantification should be re-calculated using the frequentist coverage probability of the Bayesian credible interval based on observed data for all three metrics.ConclusionsWe present an approach to automate the real-time estimation and forecasting of cases and hospitalizations and corresponding uncertainty using publicly available data. The models were able to infer short-term trends consistent with reported values at the HERC region level. Additionally, the models were able to accurately forecast and estimate the uncertainty of the measurements. This study can help identify the most affected regions and major outbreaks in the near future. The workflow can be adapted to other geographic regions, states, and even countries where decision-making processes are supported in real-time by the proposed modeling system.

Bayesian Latent Variable Co-kriging Model in Remote Sensing for Quality Flagged Observations

Bayesian Latent Variable Co-kriging Model in Remote Sensing for Quality Flagged Observations

Multi-task longitudinal forecasting with missing values on Alzheimer’s disease

Background and Objective:Machine learning techniques typically used in dementia assessment are not able to learn multiple tasks jointly and deal with time-dependent heterogeneous data containing missing values. In this paper, we reformulate SSHIBA, a recently introduced Bayesian multi-view latent variable model, for jointly learning diagnosis, ventricle volume, and ADAS score in dementia on longitudinal data with missing values.Methods:We propose a novel Bayesian Variational inference framework capable of simultaneously imputing missing values and combining information from several views. This way, we can combine different data views from different time-points in a common latent space and learn the relationships between each time-point, using the semi-supervised formulation to fully exploit the temporal structure of the data and handle missing values. In turn, the model can combine all the available information to simultaneously model and predict multiple output variables.Results:We applied the proposed model to jointly predict diagnosis, ventricle volume, and ADAS score in dementia. The comparison of imputation strategies demonstrated the superior performance of the semi-supervised formulation of the model, improving the best baseline methods. Moreover, the performance in simultaneous prediction of diagnosis, ventricle volume, and ADAS score led to an improved prediction performance over the best baseline method.Conclusions:The results demonstrate that the proposed SSHIBA framework can learn an excellent imputation of the missing values and outperforming the baselines while simultaneously predicting three different tasks.

Adaptive surrogate models with partially observed information

Surrogate models have been developed to replace expensive physical models and reduce the computational cost in various engineering applications, such as reliability analysis and uncertainty quantification. Gaussian process (GP) model exhibits superior performance among surrogate models with a distinguishing feature of estimating the uncertainty. However, fully observed datasets are generally required to establish a GP model, which is often scarce and expensive to obtain in complex engineering systems. Partially overserved information is often available and relatively plentiful in the collected datasets, which often contain data from different sources that have multi-fidelity or dimensionality and missing values. Therefore, correctly accounting for the partially observed information is important in order to take advantage of all available information and increase the prediction performance of the surrogate model to be developed. This paper presents a new method for modeling system performance with partially observed information, which integrates the Bayesian Gaussian process latent variable model (BGPLVM) with adaptive sampling to iteratively select new partially observable training sample points to improve the modeling efficiency. A novel adaptive sampling approach considering the missing frame and information cost of the partially observed information is proposed to iteratively select new training sample points and refine the model. To the best of the authors' knowledge, this is the first work designing adaptive sampling and adaptive surrogate modeling approaches for a dataset containing missing values. The numerical experiments demonstrated that the adaptive surrogate modeling method can effectively use all available information including both fully observed and partially observed data points. The developed methodology provides an accurate and cost-effective solution to take advantage of extra partially observed information in developing surrogate models.

A Differentially Private Probabilistic Framework for Modeling the Variability Across Federated Datasets of Heterogeneous Multi-View Observations

We propose a novel federated learning paradigm to model data variability among heterogeneous clients in multi-centric studies. Our method is expressed through a hierarchical Bayesian latent variable model, where client-specific parameters are assumed to be realization from a global distribution at the master level, which is in turn estimated to account for data bias and variability across clients. We show that our framework can be effectively optimized through expectation maximization (EM) over latent master's distribution and clients' parameters. We also introduce formal differential privacy (DP) guarantees compatibly with our EM optimization scheme. We tested our method on the analysis of multi-modal medical imaging data and clinical scores from distributed clinical datasets of patients affected by Alzheimer's disease. We demonstrate that our method is robust when data is distributed either in iid and non-iid manners, even when local parameters perturbation is included to provide DP guarantees. Our approach allows to quantify the variability of data, views and centers, while guaranteeing high-quality data reconstruction as compared to the state-of-the-art autoencoding models and federated learning schemes.<br>The code is available at <a href='https://gitlab.inria.fr/epione/federated-multi-views-ppca'>https://gitlab.inria.fr/epione/federated-multi-views-ppca</a>

Real-Time Estimation and Forecasting of COVID-19 Cases and Hospitalizations in Wisconsin HERC Regions for Public Health Decision Making Processes

PurposeThe spread of the COVID-19 and surging number of cases have resulted in overtaxed healthcare systems. However, limited availability and questionable reliability of the data make outbreak prediction and resource planning difficult. Moreover, any estimates or forecasts are subject to high uncertainty and low accuracy when measuring such components. The aim of the study is to apply, automate, and assess a workflow for the real-time estimation and forecasting of COVID-19 cases and hospitalizations in Wisconsin HERC regions.Methods & MaterialsThe reported cases are corrected for under-reporting and adjusted for test positivity by date of report. The corrected cases are estimated by date of infection, forecasted into the future, and transformed to date of report by region over time using a Bayesian latent variable model. The cases are corrected for hospitalization delay using log-normal distribution, and hospitalizations are estimated by county over time using Bayesian regression model. Models will be automated for real-time estimation and forecasting via RStudio Connect and made available as an R package.ResultsBoth models were fitted weekly and forecasted over a 1-day or 3-day period during the peak of the epidemic from September 20, 2020 to December 6, 2020. For cases, both scenarios outperformed the credible level of the forecast where the 3-day period (20% CrI: 0.468, 50% CrI: 0.810, 90% CrI: 1.000) performed slightly better than the 1-day period (20% CrI: 0.462, 50% CrI: 0.785, 90% CrI: 1.000). Similarly, for hospitalizations, both periods outperformed the credible level of the forecast where the 3-day period (20% CrI: 0.368, 50% CrI: 0.667, 90% CrI: 0.987) performed slightly better than the 1-day period (20% CrI: 0.358, 50% CrI: 0.653, 90% CrI: 0.987).ConclusionWe present an approach to estimate and forecast cases and hospitalizations and the corresponding uncertainty using publicly available data. The models were able to infer short-term trends consistent with reported values at the HERC region level. Models were able to accurately forecast and estimate the uncertainty of the measurements. This study can help to elucidate which regions are most affected and which regions will encounter outbreaks as well as support decision making processes.

Investigating Explanatory Factors of Machine Learning Models for Plant Classification.

Recent progress in machine learning and deep learning has enabled the implementation of plant and crop detection using systematic inspection of the leaf shapes and other morphological characters for identification systems for precision farming. However, the models used for this approach tend to become black-box models, in the sense that it is difficult to trace characters that are the base for the classification. The interpretability is therefore limited and the explanatory factors may not be based on reasonable visible characters. We investigate the explanatory factors of recent machine learning and deep learning models for plant classification tasks. Based on a Daucus carota and a Beta vulgaris image data set, we implement plant classification models and compare those models by their predictive performance as well as explainability. For comparison we implemented a feed forward convolutional neuronal network as a default model. To evaluate the performance, we trained an unsupervised Bayesian Gaussian process latent variable model as well as a convolutional autoencoder for feature extraction and rely on a support vector machine for classification. The explanatory factors of all models were extracted and analyzed. The experiments show, that feed forward convolutional neuronal networks ( and mean accuracy) outperforms the Bayesian Gaussian process latent variable pipeline ( and mean accuracy) as well as the convolutional autoenceoder pipeline ( and mean accuracy) based approaches in terms of classification accuracy, even though not significant for Beta vulgaris images. Additionally, we found that the neuronal network used biological uninterpretable image regions for the plant classification task. In contrast to that, the unsupervised learning models rely on explainable visual characters. We conclude that supervised convolutional neuronal networks must be used carefully to ensure biological interpretability. We recommend unsupervised machine learning, careful feature investigation, and statistical feature analysis for biological applications.

A note on identification constraints and information criteria in Bayesian latent variable models.

It is well known that, in traditional SEM applications, a scale must be set for each latent variable: typically, either the latent variance or a factor loading is fixed to one. While this has no impact on the fit metrics in ML estimation, it can potentially lead to varying Bayesian model comparison metrics due to the use of different prior distributions under each parameterization. This is a problem, because a researcher could artificially improve one's preferred model simply by changing the identification constraint. Using a single-factor CFA as motivation for study, we first show that Bayesian model comparison metrics can systematically change depending on constraints used. We then study principled methods for setting the scale of the latent variable that stabilize the model comparison metrics. These methods involve (i) the placement of priors on ratios of factor loadings, as opposed to individual loadings; and (ii) use of effect coding. We illustrate the methods via simulation and application.

A probabilistic model with spike-and-slab regularization for inferential fault detection and isolation of industrial processes

This article develops a Bayesian latent variable model for inferential fault detection and isolation using a spike-and-slab regularization technique. Different from conventional probabilistic models like probabilistic principal component analysis (PPCA), a mixture prior with spike-and-slab component is introduced to indicate whether the latent variables are sensitive to the abnormal state. The relevant information of latent variables is selected by the probability of assigning the effect to the slab component, while the influence of the non-informative ones is eliminated by the spike component. Furthermore, a Bayesian inference scheme based on the expectation-maximization framework is developed for the parameter learning procedure of the model. The fault detection and diagnosis method based on the inferential model is subsequently developed. The performance of the proposed method is illustrated by the benchmark Tennessee Eastman process and an application to an industrial methanol distillation process.

What Countries Select More Experienced Leaders? The PolEx Measure of Political Experience

AbstractHow can one assess which countries select more experienced leaders for the highest office? There is wide variation in prior career paths of national leaders within, and even more so between, regime types. It is therefore challenging to obtain a truly comparative measure of political experience; empirical studies have to rely on proxies instead. This article proposes PolEx, a measure of political experience that abstracts away from the details of career paths and generalizes based on the duration, quality and breadth of an individual's experience in politics. The analysis draws on a novel data set of around 2,000 leaders from 1950 to 2017 and uses a Bayesian latent variable model to estimate PolEx. The article illustrates how the new measure can be used comparatively to assess whether democracies select more experienced leaders. The authors find that while on average they do, the difference with non-democracies has declined dramatically since the early 2000s. Future research may leverage PolEx to investigate the role of prior political experience in, for example, policy making and crisis management.

Measuring rater bias in diagnostic tests with ordinal ratings.

Diagnostic tests are frequently reliant upon the interpretation of images by skilled raters. In many clinical settings, however, the variability observed between experts' ratings plays a detrimental role in the degree of confidence in these interpretations, leading to uncertainty in the diagnostic process. For example, in breast cancer testing, radiologists interpret mammographic images, while breast biopsy results are examined by pathologists. Each of these procedures involves elements of subjectivity. We propose here a flexible two-stage Bayesian latent variable model to investigate how the skills of individual raters impact the diagnostic accuracy of image-related testing in large-scale medical testing studies. A strength of the proposed model is that the true disease status of a patient within a reasonable time frame may or may not be known. In these studies, many raters each contribute classifications on a large sample of patients using a defined ordinal grading scale, leading to a complex correlation structure between ratings. Our modeling approach considers the different sources of variability contributed by experts and patients while accounting for correlations present between ratings and patients, in contrast to currently available methods. We propose a novel measure of a rater's ability (magnifier) that, in contrast to conventional measures of sensitivity and specificity, is robust to the underlying prevalence of disease in the population, providing an alternative measure of diagnostic accuracy across patient populations. Extensive simulation studies demonstrate lower bias in estimation of parameters and measures of accuracy, and illustrate outperformance of the proposed model when compared with existing models. Receiver operator characteristic curves are derived to assess the diagnostic accuracy of individual experts and their overall performance. Our proposed modeling approach is applied to a large breast imaging study for known disease status and a uterine cancer dataset for unknown disease status.

Estimation and visualization of process states using latent variable models based on Gaussian process.

The estimation and visualization of process states are important for process control in chemical and industrial plants. Since industrial processes are related to Gaussian distributions theoretically, this study focused on Gaussian process latent variable models. Process state estimation and visualization methods are proposed using two latent variables based on the Bayesian Gaussian process latent variable model (BGPLVM), infinite warped mixture model (iWMM), and Gaussian process dynamical models (GPDM). The Tennessee Eastman process dataset was analyzed and it was confirmed that the performance of estimating the process states was highest in the order of GPDM, iWMM, and BGPLVM. Moreover, time-delayed process variables were added to the process variables to consider the process dynamics, which further improved the performance of estimating the process states. Particularly in the case of GPDM, only two latent variables could estimate the process states, with approximately 100% accuracy for four process states. Additionally, even 10 process states could be estimated with approximately 90% accuracy, and it was confirmed that the process state estimation and process state visualization could be achieved simultaneously.

A Bayesian Latent Variable Selection Model for Nonignorable Missingness

Missing data are exceedingly common across a variety of disciplines, such as educational, social, and behavioral science areas. Missing not at random (MNAR) mechanism where missingness is related to unobserved data is widespread in real data and has detrimental consequence. However, the existing MNAR-based methods have potential problems such as leaving the data incomplete and failing to accommodate incomplete covariates with interactions, non-linear terms, and random slopes. We propose a Bayesian latent variable imputation approach to impute missing data due to MNAR (and other missingness mechanisms) and estimate the model of substantive interest simultaneously. In addition, even when the incomplete covariates involves interactions, non-linear terms, and random slopes, the proposed method can handle missingness appropriately. Computer simulation results suggested that the proposed Bayesian latent variable selection model (BLVSM) was quite effective when the outcome and/or covariates were MNAR. Except when the sample size was small, estimates from the proposed BLVSM tracked closely with those from the complete data analysis. With a small sample size, when the outcome was less predictable from the covariates, the missingness proportions of the covariates and the outcome were larger, and the missingness selection processes of the covariates and the outcome were more MNAR and MAR, the performance of BLVSM was less satisfactory. When the sample size was large, BLVSM always performed well. In contrast, the method with an MAR assumption provided biased estimates and undercoverage confidence intervals when the missingness was MNAR. The robustness and the implementation of BLVSM in real data were also illustrated. The proposed method is available in the Blimp software application, and the paper includes a data analysis example illustrating its use.