Bayesian Machine Learning Research Articles

Seismic forecasting by gapped wavelet transform for the Río de la Plata craton and adjacent continental platform

Historical and instrumental seismicity from 1845 to the present was compiled for the Pampean regions of Argentina and Uruguay and the adjacent continental platform. These data were analyzed using the Gapped Wavelet Transform (GWT) with the aim of investigating recurrence patterns in the time series.The GWT analysis reveals the existence of periods of seismic activity and periods of seismic silence within the considered area. A 22-year period was determined as the main pattern of seismic activity, during which earthquakes are registered only in 11 years. Based on these results, a probabilistic earthquake forecasting model was created using the Bayesian Machine Learning method, which suggests that the current seismically active period spans from 2011 to 2023 ± 1, with a cessation until 2034 ± 1, while the next seismically active period would begin in 2034 ± 2 ending in 2046 ± 2.These periods or recurrence patterns are interpreted as intervals during which stress accumulates throughout intracrustal faults in the studied area, being then released through various events within the established temporal interval. The accumulation and release mechanisms are still a subject of debate. However, the earthquakes distribution suggests a sublatitudinal arrangement that tends to confirm some correspondence with the controlling faulting of the Quilmes Trough and others transtensional regional lineaments.

A comparative study on state-of-charge estimation for lithium-rich manganese-based battery based on Bayesian filtering and machine learning methods

In this paper, a comparative study on the state of charge (SOC) estimation of the lithium-rich manganese-based battery (LRMB) has been conducted by systematically considering the equivalent circuit model (ECM), aging state and algorithms. The results show that the first-order RC model combined with the Extended Kalman filter (EKF) is more suitable for the SOC estimation of the LRMB when the battery decays less severely. The maximum value of root mean square error of SOC estimation error no exceeds 1.6 % under various dynamic operating conditions, which can well meet the practical application. Besides, the first-order model based EKF algorithm occupies least computing resource and simultaneously possesses the best convergence velocity against the initial SOC value deviation. When the LRMB suffers a severe aging, the SOC estimation performance decreases significantly with the max error exceeding 6 %, to overcome this, a new algorithm combined deep learning and Extend Kalman filter (EKF) is proposed, which exhibit high SOC estimation accuracy with the RMSE less than 0.16 %. In summary, a systematic study about the SOC estimation of LRMB has been carried out, which can provide great guidance for the future engineering application of LRMB.

Drift versus Shift: Decoupling Trends and Changepoint Analysis

We introduce a new approach for decoupling trends (drift) and changepoints (shifts) in time series. Our locally adaptive model-based approach for robustly decoupling combines Bayesian trend filtering and machine learning based regularization. An over-parameterized Bayesian dynamic linear model (DLM) is first applied to characterize drift. Then a weighted penalized likelihood estimator is paired with the estimated DLM posterior distribution to identify shifts. We show how Bayesian DLMs specified with so-called shrinkage priors can provide smooth estimates of underlying trends in the presence of complex noise components. However, their inability to shrink exactly to zero inhibits direct changepoint detection. In contrast, penalized likelihood methods are highly effective in locating changepoints. However, they require data with simple patterns in both signal and noise. The proposed decoupling approach combines the strengths of both, that is, the flexibility of Bayesian DLMs with the hard thresholding property of penalized likelihood estimators, to provide changepoint analysis in complex, modern settings. The proposed framework is outlier robust and can identify a variety of changes, including in mean and slope. It is also easily extended for analysis of parameter shifts in time-varying parameter models like dynamic regressions. We illustrate the flexibility and contrast the performance and robustness of our approach with several alternative methods across a wide range of simulations and application examples.

Improvement and generalization of ABCD method with Bayesian inference

To find New Physics or to refine our knowledge of the Standard Model at the LHC is an enterprise that involves many factors, such as the capabilities and the performance of the accelerator and detectors, the use and exploitation of the available information, the design of search strategies and observables, as well as the proposal of new models. We focus on the use of the information and pour our effort in re-thinking the usual data-driven ABCD method to improve it and to generalize it using Bayesian Machine Learning techniques and tools. We propose that a dataset consisting of a signal and many backgrounds is well described through a mixture model. Signal, backgrounds and their relative fractions in the sample can be well extracted by exploiting the prior knowledge and the dependence between the different observables at the event-by-event level with Bayesian tools. We show how, in contrast to the ABCD method, one can take advantage of understanding some properties of the different backgrounds and of having more than two independent observables to measure in each event. In addition, instead of regions defined through hard cuts, the Bayesian framework uses the information of continuous distribution to obtain soft-assignments of the events which are statistically more robust. To compare both methods we use a toy problem inspired by pp\to hh\to b\bar b b \bar bpp→hh→bb‾bb‾, selecting a reduced and simplified number of processes and analysing the flavor of the four jets and the invariant mass of the jet-pairs, modeled with simplified distributions. Taking advantage of all this information, and starting from a combination of biased and agnostic priors, leads us to a very good posterior once we use the Bayesian framework to exploit the data and the mutual information of the observables at the event-by-event level. We show how, in this simplified model, the Bayesian framework outperforms the ABCD method sensitivity in obtaining the signal fraction in scenarios with 1% and 0.5% true signal fractions in the dataset. We also show that the method is robust against the absence of signal. We discuss potential prospects for taking this Bayesian data-driven paradigm into more realistic scenarios.

Development of a machine learning-based model for the prediction and progression of diabetic kidney disease: A single centred retrospective study

BackgroundDiabetic kidney disease (DKD) is a diabetic microvascular complication often characterized by an unpredictable progression. Hence, early detection and recognition of patients vulnerable to progression is crucial. ObjectiveTo develop a prediction model to identify the stages of DKD and the factors contributing to progression to each stage using machine learning. MethodologyA retrospective study was conducted in a South Indian tertiary care hospital and collected the details of patients diagnosed with DKD from January 2017 to January 2022. Bayesian optimization-based machine learning techniques such as classification and regression were employed. The model was developed with the help of an optimization framework that effectively balances classification, prediction accuracy, and explainability. ResultsOf the 311 patients diagnosed with DKD, 227 were selected for the study. A system for predicting DKD has been created for a patient dataset utilizing a variety of machine-learning approaches. The eXtreme gradient (XG) Boost method excelled, achieving 88.75% accuracy, 88.57% precision, 91.4% sensitivity,100% specificity, and 89.49% F1-score. An interpretable data-driven method highlights significant features for early DKD diagnosis. The best explainable prediction model uses the XG Boost classifier, revealing serum uric acid, urea, phosphorous, red blood cells, calcium, and absolute eosinophil count as the major predictors influencing the progression of DKD. In the case of regression models, the gradient boost regressor performed the best, with an R2 score of 0.97. ConclusionMachine learning algorithms can effectively predict the stages of DKD and thus help physicians in providing patients with personalized care at the right time.

Estimation of water quality in Korattur Lake, Chennai, India, using Bayesian optimization and machine learning

Assessing water quality becomes imperative to facilitate informed decision-making concerning the availability and accessibility of water resources in Korattur Lake, Chennai, India, which has been adversely affected by human actions. Although numerous state-of-the-art studies have made significant advancements in water quality classification, conventional methods for training machine learning model parameters still require substantial human and material resources. Hence, this study employs stochastic gradient descent (SGD), adaptive boosting (AdaBoosting), Perceptron, and artificial neural network algorithms to classify water quality categories as these well-established methods, combined with Bayesian optimization for hyperparameter tuning, provide a robust framework to demonstrate significant performance enhancements in water quality classification. The input features for model training from 2010 to 2019 comprise water parameters such as pH, phosphate, total dissolved solids (TDS), turbidity, nitrate, iron, chlorides, sodium, and chemical oxygen demand (COD). Bayesian optimization is employed to dynamically tune the hyperparameters of different machine learning algorithms and select the optimal algorithms with the best performance. Comparing the performance of different algorithms, AdaBoosting exhibits the highest performance in water quality level classification, as indicated by its superior accuracy (100%), precision (100%), recall (100%), and F1 score (100%). The top four important factors for water quality level classification are COD (0.684), phosphate (0.119), iron (0.112), and TDS (0.084). Additionally, variations or changes in phosphate levels are likely to coincide with similar variations in TDS levels.

Bayesian Inference-Assisted Machine Learning for Near Real-Time Jamming Detection and Classification in 5G New Radio (NR)

Bayesian Inference-Assisted Machine Learning for Near Real-Time Jamming Detection and Classification in 5G New Radio (NR)

Print culture and economic constraints: A quantitative analysis of book prices in eighteenth-century Britain

Who could afford books in the late early-modern period? We explore how prices related to the demand for books in eighteenth-century Britain by analysing extensive bibliographic and socio-economic data based on Bayesian statistics and machine learning. Our results quantify in financial terms the difficulty of buying print products faced by most British households in the eighteenth century, and how this related to the varying levels of supply across price segments. We found no evidence of the well known claim that legislation would have led to lower prices. The inadequate supply and high cost of books make it likely that only higher-income households bought them regularly from the primary market.

Constraints on Prospective Deviations from the Cold Dark Matter Model Using a Gaussian Process

Recently, using Bayesian Machine Learning, a deviation from the cold dark matter model on cosmological scales has been put forward. Such a model might replace the proposed non-gravitational interaction between dark energy and dark matter, and help solve the H0 tension problem. The idea behind the learning procedure relies on a generated expansion rate, while the real expansion rate is just used to validate the learned results. In the present work, however, the emphasis is put on a Gaussian Process (GP), with the available H(z) data confirming the possible existence of the already learned deviation. Three cosmological scenarios are considered: a simple one, with an equation-of-state parameter for dark matter ωdm=ω0≠0, and two other models, with corresponding parameters ωdm=ω0+ω1z and ωdm=ω0+ω1z/(1+z). The constraints obtained on the free parameters ω0 and ω1 hint towards a dynamical nature of the deviation. The dark energy dynamics is also reconstructed, revealing interesting aspects connected with the H0 tension problem. It is concluded, however, that improved tools and more data are needed, to reach a better understanding of the reported deviation.

All‐nonlinear static‐dynamic neural networks versus Bayesian machine learning for data‐driven modelling of chemical processes

AbstractIn recent decades, the utilization of machine learning (ML) and artificial intelligence (AI) approaches have been explored for process modelling applications. However, different types of ML models may have contrasting advantages and disadvantages, which become critical during the optimal selection of a specific data‐driven model for a particular application as well as estimation of parameters during model training. This paper compares and contrasts two different types of data‐driven modelling approaches, namely the series/parallel all‐nonlinear static‐dynamic neural network models and models from a Bayesian ML approach. Both types of AI modelling approaches considered in this work have shown to significantly outperform several state‐of‐the‐art steady‐state and dynamic data‐driven modelling techniques for various performance measures, specifically, model sparsity, predictive capabilities, and computational expense. The performances of the proposed model structures and algorithms have been evaluated for two nonlinear dynamic chemical engineering systems—a plug‐flow reactor for vapour phase cracking of acetone for production of acetic anhydride and a pilot‐plant for post‐combustion CO2 capture using monoethanolamine as the solvent. For the validation data from the CO2 capture pilot plant, root mean squared error (RMSE) for flue gas outlet temperature, flowrate and CO2 concentration is 0.05%, 1.07%, and 5.0%, respectively, for the all‐nonlinear static‐dynamic neural networks and 0.1%, 1.75%, and 14.14%, respectively, for the Bayesian ML models. For the plug flow reactor data, the Bayesian ML models yield superior RMSE compared to the all‐nonlinear static‐dynamic neural networks when the measurement data are corrupted with Gaussian, auto‐correlated, or cross‐correlated noise.

Prognostication in Neurocritical Care.

This article synthesizes the current literature on prognostication in neurocritical care, identifies existing challenges, and proposes future research directions to reduce variability and enhance scientific and patient-centered approaches to neuroprognostication. Patients with severe acute brain injury often lack the capacity to make their own medical decisions, leaving surrogate decision makers responsible for life-or-death choices. These decisions heavily rely on clinicians' prognostication, which is still considered an art because of the previous lack of specific guidelines. Consequently, there is significant variability in neuroprognostication practices. This article examines various aspects of neuroprognostication. It explores the cognitive approach to prognostication, highlights the use of statistical modeling such as Bayesian models and machine learning, emphasizes the importance of clinician-family communication during prognostic disclosures, and proposes shared decision making for more patient-centered care. This article identifies ongoing challenges in the field and emphasizes the need for future research to ameliorate variability in neuroprognostication. By focusing on scientific methodologies and patient-centered approaches, this research aims to provide guidance and tools that may enhance neuroprognostication in neurocritical care.

Treed Gaussian processes for animal movement modeling.

Wildlife telemetry data may be used to answer a diverse range of questions relevant to wildlife ecology and management. One challenge to modeling telemetry data is that animal movement often varies greatly in pattern over time, and current continuous-time modeling approaches to handle such nonstationarity require bespoke and often complex models that may pose barriers to practitioner implementation. We demonstrate a novel application of treed Gaussian process (TGP) modeling, a Bayesian machine learning approach that automatically captures the nonstationarity and abrupt transitions present in animal movement. The machine learning formulation of TGPs enables modeling to be nearly automated, while their Bayesian formulation allows for the derivation of movement descriptors with associated uncertainty measures. We demonstrate the use of an existing R package to implement TGPs using the familiar Markov chain Monte Carlo algorithm. We then use estimated movement trajectories to derive movement descriptors that can be compared across individuals and populations. We applied the TGP model to a case study of lesser prairie-chickens (Tympanuchus pallidicinctus) to demonstrate the benefits of TGP modeling and compared distance traveled and residence times across lesser prairie-chicken individuals and populations. For broad usability, we outline all steps necessary for practitioners to specify relevant movement descriptors (e.g., turn angles, speed, contact points) and apply TGP modeling and trajectory comparison to their own telemetry datasets. Combining the predictive power of machine learning and the statistical inference of Bayesian methods to model movement trajectories allows for the estimation of statistically comparable movement descriptors from telemetry studies. Our use of an accessible R package allows practitioners to model trajectories and estimate movement descriptors, facilitating the use of telemetry data to answer applied management questions.

Bayesian network methodology and machine learning approach: an application on the impact of digital technologies on logistics service quality

PurposeThis paper aims to use Bayesian network (BN) methodology complemented by machine learning (ML) and what-if analysis to investigate the impact of digital technologies (DT) on logistics service quality (LSQ), employing the service quality (SERVQUAL) framework.Design/methodology/approachUsing a sample of 244 Italian firms, this study estimates the probability distributions associated with both DT and SERVQUAL logistics, as well as their interrelationships. Additionally, BN technique enables the application of ML techniques to uncover hidden relationships, as well as a series of what-if analyses to extract more knowledge.FindingsThe results show that the average probability of firms investing in DT for analytics (DTA) is higher than that of investing inDT for immersive experiences (DTIE). Furthermore, adopting both offers only a moderate likelihood of successfully implementing SERVQUAL logistics. Additionally, certain technologies may not directly influence some SERVQUAL dimensions. The application of ML reveals hidden relationships among technologies, enhancing the predictions of SERVQUAL logistics. Finally, what-if analyses provide further insights to guide decision-making processes aimed at enhancing SERVQUAL logistics dimensions through DTA and DTIE.Originality/valueThis research delves into the influence of DTIE and DTA on SERVQUAL logistics, thereby filling a gap in the existing literature in which no study has explored the intricate relationships between these technologies and SERVQUAL dimensions. Methodologically, we pioneer the integration of BN with ML techniques and what-if analysis, thus exploring innovative techniques to be used in logistics and supply-chain studies.

Hemoglobin value prediction with bayesian optimization assisted machine learning models

This study presents a framework for predicting hemoglobin (Hb) levels utilizing Bayesian optimization-assisted machine learning models, incorporating both time-domain and frequency-domain features derived from photoplethysmography (PPG) signals. Hemoglobin, a crucial protein for oxygen and carbon dioxide transport in the blood, has levels that indicate various health conditions, including anemia and diseases affecting red blood cell production. Traditional methods for measuring Hb levels are invasive, posing potential risks and discomfort. To address this, a dataset comprising PPG signals, along with demographic data (gender and age), was analyzed to predict Hb levels accurately. Our models employ support vector regression (SVR), artificial neural networks (ANNs), classification and regression trees (CART), and ensembles of trees (EoT) optimized through Bayesian optimization algorithm. The results demonstrated that incorporating age and gender as features significantly improved model performance, highlighting their importance in Hb level prediction. Among the tested models, ANN provided the best results, involving normalized raw signals, feature selection, and reduction methods. The model achieved a mean squared error (MSE) of 1.508, root mean squared error (RMSE) of 1.228, and R-squared (R²) of 0.226. This study's findings contribute to the growing body of research on non-invasive hemoglobin prediction, offering a potential tool for healthcare professionals and patients for convenient and risk-free Hb level monitoring.

Genomic prediction for sugarcane diseases including hybrid Bayesian-machine learning approaches.

Sugarcane smut and Pachymetra root rots are two serious diseases of sugarcane, with susceptible infected crops losing over 30% of yield. A heritable component to both diseases has been demonstrated, suggesting selection could improve disease resistance. Genomic selection could accelerate gains even further, enabling early selection of resistant seedlings for breeding and clonal propagation. In this study we evaluated four types of algorithms for genomic predictions of clonal performance for disease resistance. These algorithms were: Genomic best linear unbiased prediction (GBLUP), including extensions to model dominance and epistasis, Bayesian methods including BayesC and BayesR, Machine learning methods including random forest, multilayer perceptron (MLP), modified convolutional neural network (CNN) and attention networks designed to capture epistasis across the genome-wide markers. Simple hybrid methods, that first used BayesR/GWAS to identify a subset of 1000 markers with moderate to large marginal additive effects, then used attention networks to derive predictions from these effects and their interactions, were also developed and evaluated. The hypothesis for this approach was that using a subset of markers more likely to have an effect would enable better estimation of interaction effects than when there were an extremely large number of possible interactions, especially with our limited data set size. To evaluate the methods, we applied both random five-fold cross-validation and a structured PCA based cross-validation that separated 4702 sugarcane clones (that had disease phenotypes and genotyped for 26k genome wide SNP markers) by genomic relationship. The Bayesian methods (BayesR and BayesC) gave the highest accuracy of prediction, followed closely by hybrid methods with attention networks. The hybrid methods with attention networks gave the lowest variation in accuracy of prediction across validation folds (and lowest MSE), which may be a criteria worth considering in practical breeding programs. This suggests that hybrid methods incorporating the attention mechanism could be useful for genomic prediction of clonal performance, particularly where non-additive effects may be important.

Identification and Connectomic Profiling of Concussion Using Bayesian Machine Learning.

Accurate early diagnosis of concussion is useful to prevent sequelae and improve neurocognitive outcomes. Early after head impact, concussion diagnosis may be doubtful in persons whose neurological, neuroradiological, and/or neurocognitive examinations are equivocal. Such individuals can benefit from novel accurate assessments that complement clinical diagnostics. We introduce a Bayesian machine learning classifier to identify concussion through cortico-cortical connectome mapping from magnetic resonance imaging in persons with quasi-normal cognition and without neuroradiological findings. Classifier features are generated from connectivity matrices specifying the mean fractional anisotropy of white matter connections linking brain structures. Each connection's saliency to classification was quantified by training individual classifier instantiations using a single feature type. The classifier was tested on a discovery sample of 92 healthy controls (HCs; 26 females, age μ ± σ: 39.8 ± 15.5 years) and 471 adult mTBI patients (158 females, age μ ± σ: 38.4 ± 5.9 years). Results were replicated in an independent validation sample of 256 HCs (149 females, age μ ± σ: 55.3 ± 12.1 years) and 126 patients with concussion (46 females, age μ ± σ: 39.0 ± 17.7 years). Classifier accuracy exceeds 99% in both samples, suggesting robust generalizability to new samples. Notably, 13 bilateral cortico-cortical connection pairs predict diagnostic status with accuracy exceeding 99% in both discovery and validation samples. Many such connection pairs are between prefrontal cortex structures, fronto-limbic and fronto-subcortical structures, and occipito-temporal structures in the ventral ("what") visual stream. This and related connectivity form a highly salient network of brain connections that is particularly vulnerable to concussion. Because these connections are important in mediating cognitive control, memory, and attention, our findings explain the high frequency of cognitive disturbances after concussion. Our classifier was trained and validated on concussed participants with cognitive profiles very similar to those of HCs. This suggests that the classifier can complement current diagnostics by providing independent information in clinical contexts where patients have quasi-normal cognition but where concussion diagnosis stands to benefit from additional evidence.

Extrapolation of polaron properties to low phonon frequencies by Bayesian machine learning

Extrapolation of polaron properties to low phonon frequencies by Bayesian machine learning

Enhancing water quality prediction for fluctuating missing data scenarios: A dynamic Bayesian network-based processing system to monitor cyanobacteria proliferation

Tackling the impact of missing data in water management is crucial to ensure the reliability of scientific research that informs decision-making processes in public health. The goal of this study is to ascertain the root causes associated with cyanobacteria proliferation under major missing data scenarios. For this purpose, a dynamic missing data management methodology is proposed using Bayesian Machine Learning for accurate surface water quality prediction of a river from Limia basin (Spain). The methodology used entails a sequence of analytical steps, starting with data pre-processing, followed by the selection of a reliable dynamic Bayesian missing value prediction system, leading finally to a supervised analysis of the behavioral patterns exhibited by cyanobacteria. For that, a total of 2,118,844 data points were used, with 205,316 (9.69 %) missing values identified. The machine learning testing showed the iterative structural expectation maximization (SEM) as the best performing algorithm, above the dynamic imputation (DI) and entropy-based dynamic imputation methods (EBDI), enhancing in some cases the accuracy of imputations by approximately 50 % in R2, RMSE, NRMSE, and logarithmic loss values. These findings can impact how data on water quality is being processed and studied, thus, opening the door for more reliable water management strategies that better inform public health decisions.

From green-washing to innovation-washing: Environmental information intangibility and corporate green innovation in China

We use a sample of China's listed firms and employ a naïve Bayesian machine learning algorithm to reveal that environmental information intangibility superficially promotes green innovation. We demonstrate that this effect is channelled through the acquisition of institutional resources, including bank loans and government subsidies. The impact of environmental information intangibility on green innovation is most pronounced in state-owned enterprises, large firms, and politically connected firms. Furthermore, we confirm that environmental information intangibility does not lead to improvements in innovation efficiency or quality. This implies that green innovation may serve as a symbolic environmental activity. Our findings contribute to the research on environmental information intangibility, greenwashing behaviour, and their relationship to green innovation.

Uncertainty quantified discovery of chemical reaction systems via Bayesian scientific machine learning

The recently proposed Chemical Reaction Neural Network (CRNN) discovers chemical reaction pathways from time resolved species concentration data in a deterministic manner. Since the weights and biases of a CRNN are physically interpretable, the CRNN acts as a digital twin of a classical chemical reaction network. In this study, we employ a Bayesian inference analysis coupled with neural ordinary differential equations (ODEs) on this digital twin to discover chemical reaction pathways in a probabilistic manner. This allows for estimation of the uncertainty surrounding the learned reaction network. To achieve this, we propose an algorithm which combines neural ODEs with a preconditioned stochastic gradient langevin descent (pSGLD) Bayesian framework, and ultimately performs posterior sampling on the neural network weights. We demonstrate the successful implementation of this algorithm on several reaction systems by not only recovering the chemical reaction pathways but also estimating the uncertainty in our predictions. We compare the results of the pSGLD with that of the standard SGLD and show that this optimizer more efficiently and accurately estimates the posterior of the reaction network parameters. Additionally, we demonstrate how the embedding of scientific knowledge improves extrapolation accuracy by comparing results to purely data-driven machine learning methods. Together, this provides a new framework for robust, autonomous Bayesian inference on unknown or complex chemical and biological reaction systems.