Bayesian Selection Research Articles

Hierarchical Bayesian Data Selection

There are many issues that can cause problems when attempting to infer model parameters from data. Data and models are both imperfect, and as such there are multiple scenarios in which standard methods of inference will lead to misleading conclusions; corrupted data, models which are only representative of subsets of the data, or multiple regions in which the model is best fit using different parameters. Methods exist for the exclusion of some anomalous types of data, but in practice, data cleaning is often undertaken by hand before attempting to fit models to data. In this work, we will employ hierarchical Bayesian data selection; the simultaneous inference of both model parameters, and parameters which represent our belief that each observation within the data should be included in the inference. The aim, within a Bayesian setting, is to find the regions of observation space for which the model can well-represent the data, and to find the corresponding model parameters for those regions. A number of approaches will be explored, and applied to test problems in linear regression, and to the problem of fitting an ODE model, approximated by a finite difference method. The approaches are simple to implement, can aid mixing of Markov chains, and are broadly applicable to many inferential problems.

Continuous Flow Chemistry and Bayesian Optimization for Polymer-Functionalized Carbon Nanotube-Based Chemiresistive Methane Sensors.

We report the preparation of poly(ionic) polymer-wrapped single-walled carbon nanotube dispersions for chemiresistive methane (CH4) sensors with improved humidity tolerance. Single-walled CNTs (SWCNTs) were noncovalently functionalized by poly(4-vinylpyridine) (P4VP) with varied amounts of a poly(ethylene glycol) (PEG) moiety bearing a Br and terminal azide group (Br-R1). The quaternization of P4VP with Br-R1 was performed using continuous flow chemistry and Bayesian optimization-guided reaction selection. Polymers (PyBrR1) with different degrees of functionalization were used to disperse SWCNTs and subsequently incorporated into sensors containing a platinum complex as an aerobic oxidative catalyst with a polyoxometalate (POM) redox mediator to facilitate room-temperature CH4 sensing. As the degree of quaternization in the PyBrR1-CNT composites increased, improvements in response magnitude were observed, with nominally 10% quaternized PyBrR1 giving the largest response. Incorporation of PEG improved sensor stability at relative humidities between 57-90% versus sensors fabricated from CNT dispersions with unfunctionalized P4VP. Devices fabricated with these dispersions outperformed those prepared in situ under dry conditions, and exhibited greater stability at elevated humidities. The influence of Keggin-type POM character was also evaluated to identify alternative POMs for enhanced sensor performance at high humidity. In an effort to identify areas for further improvement in algorithm performance for polymer functionalization, a kinetically informed machine learning model was explored as a route to predict reactivity of pyridine units and alkyl bromides under flow conditions.

Bayesian Selection of Relaxed-clock Models: Distinguishing Between Independent and Autocorrelated Rates.

In Bayesian molecular-clock dating of species divergences, rate models are used to construct the prior on the molecular evolutionary rates for branches in the phylogeny, with independent and autocorrelated rate models being commonly used. The two classes of models, however, can result in markedly different divergence time estimates for the same dataset, and thus selecting the best rate model appears important for obtaining reliable in- ferences of divergence times. However, the properties of Bayesian rate model selection are not well understood, in particular when the number of sequence partitions analysed increases and when age calibrations (such as fossil calibrations) are misspecified. Further- more, Bayesian rate model selection is computationally expensive as it requires calculation of marginal likelihoods by MCMC sampling, and therefore methods that can speed up the model selection procedure without compromising its accuracy are desirable. In this study, we use a combination of computer simulations and real data analysis to investigate the sta- tistical behaviour of Bayesian rate model selection and we also explore approximations of the likelihood to improve computational efficiency in large phylogenomic datasets. Our simulations demonstrate that the posterior probability for the correct rate model converges to one as more molecular sequence partitions are analysed and when no calibrations are used, as expected due to asymptotic Bayesian model selection theory. Furthermore, we also show the model selection procedure is robust to slight misspecification of calibrations, and reliable inference of the correct rate model is possible in this case. However, we show that when calibrations are seriously misspecified, calculated model probabilities are com- pletely wrong and may converge to one for the wrong rate model. Finally, we demonstrate that approximating the phylogenetic likelihood under an arcsine branch-length transform can dramatically reduce the computational cost of rate model selection without compro- mising accuracy. We test the approximate procedure on two large phylogenies of primates (372 species) and flowering plants (644 species), replicating results obtained on smaller datasets using exact likelihood. Our findings and methodology can assist users in selecting the optimal rate model for estimating times and rates along the Tree of Life.

Bi-objective ranking and selection using stochastic kriging

We consider bi-objective ranking and selection problems, where the goal is to correctly identify the Pareto-optimal solutions among a finite set of candidates for which the objective function values have to be estimated from noisy evaluations. When identifying these solutions, the noise perturbing the observed performance may lead to two types of errors: solutions that are truly Pareto-optimal may appear to be dominated, and solutions that are truly dominated may appear to be Pareto-optimal. We propose a novel Bayesian bi-objective ranking and selection method that sequentially allocates extra samples to competitive solutions, in view of reducing the misclassification errors when identifying the solutions with the best expected performance. The approach uses stochastic kriging to build reliable predictive distributions of the objectives, and exploits this information to decide how to resample. The experiments are designed to evaluate the algorithm on several artificial and practical test problems. The proposed approach is observed to consistently outperform its competitors (a well-known state-of-the-art algorithm and the standard equal allocation method), which may also benefit from the use of stochastic kriging information.

Transitive inference as probabilistic preference learning.

Transitive inference (TI) is a cognitive task that assesses an organism's ability to infer novel relations between items based on previously acquired knowledge. TI is known for exhibiting various behavioral and neural signatures, such as the serial position effect (SPE), symbolic distance effect (SDE), and the brain's capacity to maintain and merge separate ranking models. We propose a novel framework that casts TI as a probabilistic preference learning task, using one-parameter Mallows models. We present a series of simulations that highlight the effectiveness of our novel approach. We show that the Mallows ranking model natively reproduces SDE and SPE. Furthermore, extending the model using Bayesian selection showcases its capacity to generate and merge ranking hypotheses as pairs with connecting symbols. Finally, we employ neural networks to replicate Mallows models, demonstrating how this framework aligns with observed prefrontal neural activity during TI. Our innovative approach sheds new light on the nature of TI, emphasizing the potential of probabilistic preference learning for unraveling its underlying neural mechanisms.

Information FOMO: The Unhealthy Fear of Missing Out on Information-A Method for Removing Misleading Data for Healthier Models.

Misleading or unnecessary data can have out-sized impacts on the health or accuracy of Machine Learning (ML) models. We present a Bayesian sequential selection method, akin to Bayesian experimental design, that identifies critically important information within a dataset while ignoring data that are either misleading or bring unnecessary complexity to the surrogate model of choice. Our method improves sample-wise error convergence and eliminates instances where more data lead to worse performance and instabilities of the surrogate model, often termed sample-wise "double descent". We find these instabilities are a result of the complexity of the underlying map and are linked to extreme events and heavy tails. Our approach has two key features. First, the selection algorithm dynamically couples the chosen model and data. Data is chosen based on its merits towards improving the selected model, rather than being compared strictly against other data. Second, a natural convergence of the method removes the need for dividing the data into training, testing, and validation sets. Instead, the selection metric inherently assesses testing and validation error through global statistics of the model. This ensures that key information is never wasted in testing or validation. The method is applied using both Gaussian process regression and deep neural network surrogate models.

Using removable sensors in structural health monitoring: A Bayesian methodology for attachment-to-attachment uncertainty quantification

Structural responses are vulnerable to perturbations when subjected to variable environmental and operational conditions, thereby contributing to the slow uptake of Structural Health Monitoring (SHM) beyond academic research. More importantly, SHM systems using removable sensors face additional sensing variability due to the inherent uncertainty in sensor reattachment compared to permanently attached sensors. In this study, a novel probabilistic approach is developed for output-only damage detection utilizing ultrasonic guided waves. Experimental uncertainties and sensor attachment factors are simultaneously considered for the first time in a Bayesian context. The uncertainty introduced by reattached sensors is incorporated into the likelihood function in a comprehensive way. Bayesian model class selection using an adaptive sampling scheme is adopted to evaluate the evidence of discrete damage classes. A reference case to an aluminum plate with progressive notches is utilized to demonstrate the significant effects of attachment uncertainty on guided wave signals. The optimal data normalization model for guided wave processing under different attachment states is evaluated using the Mahalanobis distance. The notch length is identified accurately along with the uncertainty, even though the latent adhesive parameters cannot be estimated directly.

Integrative Modeling of Protein-Polypeptide Complexes by Bayesian Model Selection using AlphaFold and NMR Chemical Shift Perturbation Data.

Protein-polypeptide interactions, including those involving intrinsically-disordered peptides and intrinsically-disordered regions of protein binding partners, are crucial for many biological functions. However, experimental structure determination of protein-peptide complexes can be challenging. Computational methods, while promising, generally require experimental data for validation and refinement. Here we present CSP_Rank, an integrated modeling approach to determine the structures of protein-peptide complexes. This method combines AlphaFold2 (AF2) enhanced sampling methods with a Bayesian conformational selection process based on experimental Nuclear Magnetic Resonance (NMR) Chemical Shift Perturbation (CSP) data and AF2 confidence metrics. Using a curated dataset of 108 protein-peptide complexes from the Biological Magnetic Resonance Data Bank (BMRB), we observe that while AF2 typically yields models with excellent consistency with experimental CSP data, applying enhanced sampling followed by data-guided conformational selection routinely results in ensembles of structures with improved agreement with NMR observables. For two systems, we cross-validate the CSP-selected models using independently acquired nuclear Overhauser effect (NOE) NMR data and demonstrate how CSP and NMR can be combined using our Bayesian framework for model selection. CSP_Rank is a novel method for integrative modeling of protein-peptide complexes and has broad implications for studies of protein-peptide interactions and aiding in understanding their biological functions.

Incorporating Different Sources of Information for Bayesian Optimal Portfolio Selection

This article introduces Bayesian inference procedures for tangency portfolios, with a primary focus on deriving a new conjugate prior for portfolio weights. This approach not only enables direct inference about the weights but also seamlessly integrates additional information into the prior specification. Specifically, it automatically incorporates high-frequency returns and a market condition metric (MCM), exemplified by the CBOE Volatility Index (VIX) and Economic Policy Uncertainty Index (EPU), significantly enhancing the decision-making process for optimal portfolio construction. While the Jeffreys’ prior is also acknowledged, emphasis is placed on the advantages and practical applications of the conjugate prior. An extensive empirical study reveals that our method, leveraging this conjugate prior, consistently outperforms existing trading strategies in the majority of examined cases.

Genomic Association and Prediction Study for Yield Traits in a Sugarcane (Saccharum spp. Hybrids) Mapping Population ‘LCP 85‐384’

ABSTRACTSugarcane (Saccharum spp. hybrids) are complex polyploid and aneuploid interspecific hybrids with 110–130 chromosomes. A traditional sugarcane breeding cycle takes 13–15 years and involves multiple years and locations testing of yield. To identify molecular markers associated with yield‐related traits, the LCP 85‐384 cultivar and its mapping population of 263 self‐progenies were planted in two randomly replicated field plots. The mapping population was genotyped with amplified fragment length polymorphism (AFLP), simple sequence repeats (SSR) and target region amplification polymorphism (TRAP) markers. Data on plant height, stalk number, stalk diameter and stalk weight were collected. A large variation was observed for each trait. A genome‐wide association study (GWAS) was conducted using mixed linear model (MLM), generalized linear model (GLM) and single marker regression (SMR) programmes of TASSEL 5 and FarmCPU of GAPIT 3. A total of 64 yield trait‐associated alleles were identified, including 11 for stalk number, 36 for stalk weight, 21 for stalk diameter and 5 for plant height. Of the 64 alleles, seven were linked to two traits and one to three traits. Genomic prediction (GP) was also performed by cross‐prediction with five models, namely, ridge regression best linear unbiased prediction (rrBLUP), Bayesian ridge regression (BRR), Bayesian A (BA), Bayesian B (BB), and Bayesian least absolute shrinkage and selection operator (BL). Prediction accuracy (r value) reached 0.40 for plant height, 0.36 for stalk number, 0.44 for stalk diameter and 0.54 for stalk weight with the standard errors from 0.009 to 0.012. Once verified, these markers will be a valuable tool to aid in the selection of yield‐related traits in sugarcane improvement programmes.

Genomic analyses identify 15 susceptibility loci and reveal HDAC2, SOX2-OT, and IGF2BP2 in a naturally-occurring canine model of gastric cancer.

Gastric cancer (GC) is the fifth most common human cancer worldwide, but the genetic etiology is largely unknown. We performed a Bayesian genome-wide association study and selection analyses in a naturally-occurring canine model of GC, the Belgian Tervuren and Sheepdog breeds, to elucidate underlying genetic risk factors. We identified 15 loci with over 90% predictive accuracy for the GC phenotype. Variant filtering revealed germline putative regulatory variants for the EPAS1 (HIF2A) and PTEN genes and a coding variant in CD101. Although closely related to Tervuren and Sheepdogs, Belgian Malinois rarely develop GC. Across-breed analyses uncovered protective haplotypes under selection in Malinois at SOX2-OT and IGF2BP2. Among Tervuren and Sheepdogs, HDAC2 putative regulatory variants were present at comparatively high frequency and were associated with GC. Here, we describe a complex genetic architecture governing GC in a dog model, including genes such as PDZRN3, that have not been associated with human GC.

Machine learning-based atrial fibrillation detection and onset prediction using QT-dynamicity

Objective. This study examines the value of ventricular repolarization using QT dynamicity for two different types of atrial fibrillation (AF) prediction. Approach. We studied the importance of QT-dynamicity (1) in the detection and (2) the onset prediction (i.e. forecasting) of paroxysmal AF episodes using gradient-boosted decision trees (GBDT), an interpretable machine learning technique. We labeled 176 paroxysmal AF onsets from 88 patients in our unselected Holter recordings database containing paroxysmal AF episodes. Raw ECG signals were delineated using a wavelet-based signal processing technique. A total of 44 ECG features related to interval and wave durations and amplitude were selected and the GBDT model was trained with a Bayesian hyperparameters selection for various windows. The dataset was split into two parts at the patient level, meaning that the recordings from each patient were only present in either the train or test set, but not both. We used 80% on the database for the training and the remaining 20% for the test of the trained model. The model was evaluated using 5-fold cross-validation. Main results. The mean age of the patients was 75.9 ± 11.9 (range 50–99), the number of episodes per patient was 2.3 ± 2.2 (range 1–11), and CHA2DS2-VASc score was 2.9 ± 1.7 (range 1–9). For the detection of AF, we obtained an area under the receiver operating curve (AUROC) of 0.99 (CI 95% 0.98–0.99) and an accuracy of 95% using a 30 s window. Features related to RR intervals were the most influential, followed by those on QT intervals. For the AF onset forecast, we obtained an AUROC of 0.739 (0.712–0.766) and an accuracy of 74% using a 120s window. R wave amplitude and QT dynamicity as assessed by Spearman’s correlation of the QT-RR slope were the best predictors. Significance. The QT dynamicity can be used to accurately predict the onset of AF episodes. Ventricular repolarization, as assessed by QT dynamicity, adds information that allows for better short time prediction of AF onset, compared to relying only on RR intervals and heart rate variability. Communication between the ventricles and atria is mediated by the autonomic nervous system (ANS). The variations in intraventricular conduction and ventricular repolarization changes resulting from the influence of the ANS play a role in the initiation of AF.

Subsample Selection Methods in the Lake Management

AbstractThe problem of subsample selection among an enormous number of combinations arises when some covariates are available for all units, but the response can be measured only for a subset of them. When estimating a Bayesian prediction model, optimized selections can be more efficient than random sampling. The work is motivated by environmental management of aquatic systems. We consider data on 4360 Finnish lakes and aim to find an approximately optimal subsample of lakes in the sense of Bayesian D-optimality. We study Bayesian two-stage selection where the choice of lakes to be measured at the second stage depends on the measurements carried out at the first stage. The results indicate that the two-stage approach has a modest advantage compared to the single-stage approach.

Bayesian effect selection in structured additive quantile regression

Bayesian structured additive quantile regression is an established tool for regressing outcomes with unknown distributions on a set of explanatory variables and/or when interest lies with effects on the more extreme values of the outcome. Even though variable selection for quantile regression exists, its scope is limited. We propose the use of the Normal Beta Prime Spike and Slab (NBPSS) prior in Bayesian quantile regression to aid the researcher in not only variable but also effect selection. We compare the Bayesian NBPSS approach to statistical boosting for quantile regression, a current standard in automated variable selection in quantile regression, in a simulation study with varying degrees of model complexity and illustrate both methods on an example of childhood malnutrition in Nigeria. The NBPSS prior shows good performance in variable and effect selection as well as prediction compared to boosting and can thus be recommended as an additional tool for quantile regression model building.

Bayesian model class selection of nonlinear constitutive relationships for piezoelectric energy harvesters with small set of observations

The demand for efficient energy harvesting from vibratory environments has notably increased with the proliferation of piezoelectric devices. However, mismatches between model predictions and experimental observations underpin a critical challenge: accurately modeling piezoelectric energy harvesters (PEHs) while accounting for the inherent uncertainties in their electromechanical properties. A comprehensive Bayesian inference framework is posited for identifying electromechanical properties and facilitating model class selection of nonlinear PEHs. This approach addresses uncertainties by integrating prior knowledge and experimental data to update model parameters, offering the potential to balance model complexity and predictive precision systematically. The influence of different excitation amplitudes and the number of observations employed in the framework is studied. The results demonstrated the ability to capture both linear and nonlinear behaviors, while the model class selection effectively determined the simplest constitutive models sufficient for accurate predictions, validated against varying excitation intensities. By elucidating the trade-off between model simplicity and accuracy, this method not only refines the understanding of PEH behavior under diverse operating conditions but also equips designers with a robust predictive tool. Consequently, this methodology serves to optimize the deployment of PEHs in real-world applications, such as structural health monitoring, ensuring reliable energy supply with quantifiable confidence levels.

Correction to: Bayesian high-dimensional covariate selection in non-linear mixed-effects models using the SAEM algorithm

Correction to: Bayesian high-dimensional covariate selection in non-linear mixed-effects models using the SAEM algorithm

The impact of Bayesian optimization on feature selection

Feature selection is an indispensable step for the analysis of high-dimensional molecular data. Despite its importance, consensus is lacking on how to choose the most appropriate feature selection methods, especially when the performance of the feature selection methods itself depends on hyper-parameters. Bayesian optimization has demonstrated its advantages in automatically configuring the settings of hyper-parameters for various models. However, it remains unclear whether Bayesian optimization can benefit feature selection methods. In this research, we conducted extensive simulation studies to compare the performance of various feature selection methods, with a particular focus on the impact of Bayesian optimization on those where hyper-parameters tuning is needed. We further utilized the gene expression data obtained from the Alzheimer's Disease Neuroimaging Initiative to predict various brain imaging-related phenotypes, where various feature selection methods were employed to mine the data. We found through simulation studies that feature selection methods with hyper-parameters tuned using Bayesian optimization often yield better recall rates, and the analysis of transcriptomic data further revealed that Bayesian optimization-guided feature selection can improve the accuracy of disease risk prediction models. In conclusion, Bayesian optimization can facilitate feature selection methods when hyper-parameter tuning is needed and has the potential to substantially benefit downstream tasks.

Influence of study characteristics, methodological rigour and publication bias on efficacy of pharmacotherapy in obsessive-compulsive disorder: a systematic review and meta-analysis of randomised, placebo-controlled trials

QuestionWe examined the effect of study characteristics, risk of bias and publication bias on the efficacy of pharmacotherapy in randomised controlled trials (RCTs) for obsessive-compulsive disorder (OCD).Study selection and analysisWe...

Analytical Bayesian Copula‐Based Uncertainty Quantification (A‐BASIC‐UQ) Using Data with Missing Values in Structural Health Monitoring

The presence of missing values is common in real‐world datasets, so modeling and uncertainty quantification (UQ) of incomplete datasets have gained increasing attention in various research areas, including structural health monitoring (SHM). However, modeling and UQ utilizing incomplete datasets are nontrivial tasks. On the other hand, prediction based on a set of incomplete measured input variables is also an important task, but most existing methods, which are discriminative models, do not possess this capability. Aiming to tackle these two challenges, we propose the two‐stage analytical Bayesian copula‐based uncertainty quantification (A‐BASIC‐UQ) using incomplete SHM data. In the modeling stage, the copula‐based multivariate joint probability density function (PDF) is modeled directly according to an incomplete dataset without imputation or disposal of any data points. For the univariate marginal PDF, using the measured (nonmissing) values of the corresponding random variable (RV), Bayesian model class selection is conducted to select the most suitable model class. For the Gaussian copula PDF, using the bivariate complete data points of entry‐by‐entry pairwise data, the optimal parameter vector is obtained from the estimation of the Pearson correlation coefficient. In the prediction stage, the analytical expressions of the predictive PDF, the predicted value and the credible region of the output variables are derived according to a set of incomplete measured input variables. The analytical expression of the predictive PDF is obtained based on the analytical operations on the auxiliary RVs and that of the predicted value and the credible region are obtained based on the analysis of multivariate Gaussian distribution. Therefore, the proposed method does not require numerical integration nor Monte Carlo simulation and does not suffer from computational burden even when there are many variables (say 4 or above). Examples using simulated data and real SHM data are presented to illustrate the capability of the proposed A‐BASIC‐UQ.

Unveiling the Interplay Between Depressive Symptoms' Alleviation and Quality of Life Improvement in Major Depressive Disorder: A Network Analysis Based on Longitudinal Data.

Understanding the dynamic relationship between depressive symptoms and quality of life (QOL) is essential in improving long-term outcomes for patients with Major Depressive Disorder (MDD). While previous studies often relied on cross-sectional data, there is a pressing need for stronger evidence based on longitudinal data to better inform the development of effective clinical interventions. By focusing on key depressive symptoms, such interventions have the potential to ultimately enhance QOL in individuals with MDD. This multi-center prospective study, conducted between 2016 and 2020, enrolled outpatients and inpatients diagnosed with MDD across twelve psychiatric hospitals in China. Longitudinal data on Patient Health Questionnaire - 9 (PHQ-9) and Quality of Life Enjoyment and Satisfaction Questionnaire-Short Form (Q-LES-Q-SF) was analyzed using an Extended Bayesian Information Criterion (EBIC) graphical least absolute shrinkage and selection operator (gLASSO) network model to explore the connections between depressive symptom changes and QOL changes. Flow network was applied to investigate relationships between individual symptom changes and overall QOL score change, as well as daily functional independence. This study included 818 participants with complete data after 8-week antidepressant treatment. Apart from the overlapping items from PHQ-9 and Q-LES-Q-SF, the three edges between "mood" (delta-QLES2) and "anhedonia" (delta-DEP1), between "physical health" (delta-QLES1) and "sleep problems" (delta-DEP3), and between "physical health" (delta-QLES1) and "sad mood" (delta-DEP2) were the most strong bridges between the cluster of depressive symptoms alleviation and the cluster of QOL change. "Anhedonia" (delta-DEP1), "sad mood" (delta-DEP2) and "loss of energy" (delta-DEP4) had the highest bridge strength between the alleviations of depressive symptoms and the total score change of Q-LES-Q-SF. Anhedonia had the greatest connection with participants' satisfaction with function in daily life. This study highlighted the potential for developing highly effective interventions by targeting on central symptoms, thereby to ultimately improve QOL for patients with MDD.