Probabilistic Machine Learning Approach Research Articles

A Deep Probabilistic Machine Learning Approach to Ray Tracing Path Loss Prediction at 900 MHz

A Deep Probabilistic Machine Learning Approach to Ray Tracing Path Loss Prediction at 900 MHz

Probabilistic machine learning approach to reliability analysis of a bogie frame under dynamic loading

ABSTRACT This study aims to evaluate the fatigue failure probability of a bogie frame considering the variability of input parameters, including loading ( L ), endurance limit ( S e ), and fillet size ( r ), through a data-driven surrogate model. Mechanical tests were conducted to determine the mechanical properties of the material of the bogie frame while a combination of machine learning and FEA has been utilized to generate a dataset for the dynamic response of the bogie frame under main in-service fatigue loads. Nine machine learning-based surrogate models were constructed based on the actual response at a limited set of data points chosen by the Optimum space-filling algorithm, and their accuracy was investigated. It is found that the CatBoost model is the optimal algorithm to map the stochastic input parameters with the factor of safety as the output parameter and perform the reliability evaluation. Also, results reveal a fatigue reliability of 99.34% for the bogie frame under normal conditions, with a cumulative failure probability of less than 0.66% over a 30-year service life. Furthermore, the results show that the proposed machine learning-based approach is an efficient tool to evaluate the fatigue failure probability of the bogie frame with reasonable accuracy when a small set of training data is available. This study’s scope extends to providing comprehensive guidelines for employing machine learning methods for fatigue reliability analysis of complex vehicle structures in the presence of various stochastic variables.

How Differently Do Farms Respond to Agri-environmental Policies? A Probabilistic Machine-Learning Approach

How Differently Do Farms Respond to Agri-environmental Policies? A Probabilistic Machine-Learning Approach

Using probabilistic machine learning to better model temporal patterns in parameterizations: a case study with the Lorenz 96 model

Abstract. The modelling of small-scale processes is a major source of error in weather and climate models, hindering the accuracy of low-cost models which must approximate such processes through parameterization. Red noise is essential to many operational parameterization schemes, helping model temporal correlations. We show how to build on the successes of red noise by combining the known benefits of stochasticity with machine learning. This is done using a recurrent neural network within a probabilistic framework (L96-RNN). Our model is competitive and often superior to both a bespoke baseline and an existing probabilistic machine learning approach (GAN, generative adversarial network) when applied to the Lorenz 96 atmospheric simulation. This is due to its superior ability to model temporal patterns compared to standard first-order autoregressive schemes. It also generalizes to unseen scenarios. We evaluate it across a number of metrics from the literature and also discuss the benefits of using the probabilistic metric of hold-out likelihood.

Probabilistic machine learning approach for structural reliability analysis

In recent years, the surrogate modeling technique has been increasingly employed for the failure probability estimation of civil structures. The surrogate modeling technique acts as a black box that predicts the performance of expensive-to-evaluate problems. However, the uncertainty of failure probability estimation due to the discrepancy between the surrogate model and physical model has not been well studied. This study introduces a new surrogate modeling technique for failure probability estimation of civil structures that utilizes a Bayesian neural network. Specifically, a fully connected neural network is constructed to approximate the limit state function. The hyperparameters in the neural network are obtained within the Bayesian paradigm, and the probability distributions are estimated through Laplace approximation. When the Bayesian neural network method is used for reliability analysis, the failure curve and the corresponding uncertainty can be estimated. Numerical studies on 2d truss, a simple support beam under moving load, and dynamic analysis of subway station considering the soil–structure interaction are conducted to validate the efficiency of the approach. Results with Monte Carlo simulation and subset simulation are also presented. The results demonstrate the proposed method’s potential for improving prediction accuracy by factoring in the uncertainty embedded in the surrogate model. Additionally, it exhibits higher levels of efficiency than traditional methods.

Probabilistic machine learning approach to predict incompetent rock masses in TBM construction

Probabilistic machine learning approach to predict incompetent rock masses in TBM construction

Identifying At-Risk Students for Early Intervention—A Probabilistic Machine Learning Approach

The utilization of learning analytics to identify at-risk students for early intervention has exhibited promising results. However, most predictive models utilized to address this issue have been based on non-probabilistic machine learning models. In response, this study incorporated probabilistic machine learning for two reasons: (1) to facilitate the inclusion of domain knowledge, and (2) to enable the quantification of uncertainty in model parameters and predictions. The study developed a five-stage, probabilistic logistic regression model to identify at-risk students at different stages throughout the academic calendar. Rather than predicting a student’s final or exam mark, the model was focused on predicting the at-risk probabilities for subsequent assessments—specifically, the probability of a student failing an upcoming assessment. The model incorporated student engagement data from Moodle, as well as demographic and student performance data. The study’s findings indicate that the significance and certainty of student engagement and demographic variables decreased after incorporating student-performance variables, such as assignments and tests. The most effective week for identifying at-risk students was found to be week 6, when the accuracy was 92.81%. Furthermore, the average level of uncertainty exhibited by the models decreased by 60% from stage 3 to 5, indicating more reliable predictions at later than earlier stages. The study highlights the potential of a probabilistic machine learning model to aid instructors and practitioners in identifying at-risk students, and thereby to enhance academic outcomes.

Data-driven prediction of critical flutter velocity of long-span suspension bridges using a probabilistic machine learning approach

Among the consequences of wind-induced excitation on long-span cable-supported bridges, flutter instability is the most dangerous and can collapse bridge structures. Until now, the flutter velocity of long-span bridges can be computed after conducting an experimental wind tunnel test or by means of computational fluid dynamics (CFD). However, the approaches mentioned above are cost-prohibitive and time-consuming especially when there are many sampling designs to evaluate. Therefore, it is crucial to develop an alternative solution for evaluating the flutter performance of such structures while minimizing the associated cost and computation time without worsening the accuracy of the results. Scholars have proposed machine learning (ML) techniques to address this issue. Unfortunately, some ML techniques do not provide excellent generalization for small datasets and are prone to bias even when the accuracy is relatively high. Besides, conventional ML models do not consider the uncertainty in data. This study proposes a probabilistic machine learning approach to overcome the weaknesses of the conventional ML models. The dataset used to train the proposed model was generated from 73 sampling designs using a fully numerical approach that involved 2D URANS (Unsteady Reynolds-averaged Navier-Stokes) CFD simulations, quasi-steady flutter theory, finite element model, and multi-mode approach for critical flutter velocity computation. The deck cross-section's most influential parameters and the critical flutter velocity constitute the datasets used to build the proposed probabilistic machine learning model. Our finding substantiates that the probabilistic machine learning approach based on hierarchical Bayesian modeling (PML-HBM) overperformed the conventional machine learning models, including extreme gradient boosting regression (XGBR), random forest (RF), and the support vector regression(SVR). Therefore, PML-HBM can accurately predict the critical flutter velocity. This model is an interesting approach to circumvent the time and cost associated with wind tunnel tests and CFD simulations at the earlier design stage of long-span bridge aerodynamic study.

Investigating the Effect of Temperature History on Crystal Morphology of Thermoplastic Composites Using In Situ Polarized Light Microscopy and Probabilistic Machine Learning.

Processing parameters including temperature history affect the morphology of semicrystalline thermoplastic composites, and in turn their performance. In addition, the competition between spherulite growth in resin-rich areas, and transcrystallinity growth from fiber surfaces, determines the final morphology. In this study, growth of crystals in low volume fraction PEEK-carbon fiber composites was studied in situ, using a polarized microscope equipped with a heating and cooling controlled stage and a probabilistic machine learning approach, Gaussian Process Regression (GPR). GPR showed that for spherulites, growth kinetics follows the established Lauritzen-Hoffman equation, while transcrystallinity growth deviates from the theory. Combined GPR model and Lauritzen-Hoffman equation were used to deconvolute the underlying competition between diffusion and secondary nucleation at growth front of spherulites and transcrystalline regions.

Short-term rainfall forecasting using cumulative precipitation fields from station data: a probabilistic machine learning approach

Rainfall nowcasting supports emergency decision-making in hydrological, agricultural, and economical sectors. However, short-term prediction is challenging because meteorological variables are strongly interconnected and rapidly change during an event. Since machine learning do not require any previous physical assumption, this study investigates their ability to provide reliable and quick forecasts. This paper proposes a machine learning model for probabilistic rainfall nowcasting at 10 min intervals for short lead times - from 30 min up to 6 h. The model employs cumulative rainfall fields from station data as inputs for a feed forward neural network to predict rainfall interval and the corresponding probability of occurrence. Cumulative rainfall depths from station data were used to overcome the lack of temporal memory of the feed forward neural networks. In this way, using only the current rain field as input, the model exploited pattern recognition techniques combining both temporal - cumulative rainfall depth - and spatial - cumulative rainfall field – information. Based on 359 rain events observed in an area of 1619 km2 in Southern Italy, 95 machine learning models were independently trained for 19 recording stations and for each target lead-time (30, 60, 120, 180 and 360 min). Comprehensive nowcasts verification was performed to analyse the reliability of probabilistic nowcasts using both continuous (RMSE and RAE) and categorical (POD, FAR and CSI) indicators. The performance of the models was also compared with the results of Eulerian Persistence. All the models produced consistent nowcasts and learnt the complex relationship describing space–time rainfall evolution. As expected, predictive accuracy gradually decreased as the lead-time increase, according to physically based models. Results showed that the use of both temporal and spatial information enables the model to predict short-term rainfall using only the current measurements as input, resulting in a rapid, easily replicable and convenient nowcasting approach. The procedure is an effective way to predict multi-step rainfall fields and is suitable for operational early warning system.

A Machine Learning Framework for Detecting COVID-19 Infection Using Surface-Enhanced Raman Scattering.

In this study, we explored machine learning approaches for predictive diagnosis using surface-enhanced Raman scattering (SERS), applied to the detection of COVID-19 infection in biological samples. To do this, we utilized SERS data collected from 20 patients at the University of Maryland Baltimore School of Medicine. As a preprocessing step, the positive-negative labels are obtained using Polymerase Chain Reaction (PCR) testing. First, we compared the performance of linear and nonlinear dimensionality techniques for projecting the high-dimensional Raman spectra to a low-dimensional space where a smaller number of variables defines each sample. The appropriate number of reduced features used was obtained by comparing the mean accuracy from a 10-fold cross-validation. Finally, we employed Gaussian process (GP) classification, a probabilistic machine learning approach, to correctly predict the occurrence of a negative or positive sample as a function of the low-dimensional space variables. As opposed to providing rigid class labels, the GP classifier provides a probability (ranging from zero to one) that a given sample is positive or negative. In practice, the proposed framework can be used to provide high-throughput rapid testing, and a follow-up PCR can be used for confirmation in cases where the model’s uncertainty is unacceptably high.

A Probabilistic Machine Learning Approach for the Uncertainty Quantification of Electronic Circuits Based on Gaussian Process Regression

This article introduces a probabilistic machine learning framework for the uncertainty quantification (UQ) of electronic circuits based on the Gaussian process regression (GPR). As opposed to classical surrogate modeling techniques, GPR inherently provides information on the model uncertainty. The main contribution of this work is twofold. First, it describes how, in a UQ scenario, the model uncertainty can be combined with the uncertainty of the input design parameters to provide confidence bounds for the statistical estimates of the system outputs, such as moments and probability distributions. These confidence bounds allow assessing the accuracy of the predicted statistics. Second, in order to deal with dynamic multioutput systems, principal component analysis (PCA) is effectively employed to compress the time-dependent output variables into a smaller set of components, for which the training of individual GPR models becomes feasible. The uncertainty on the principal components is then propagated back to the original output variables. Several application examples, ranging from a trivial RLC circuit to real-life designs, are used to illustrate and validate the advocated approach.

Identifying At-Risk Students for Early Intervention – a Probabilistic Machine Learning Approach

Identifying At-Risk Students for Early Intervention – a Probabilistic Machine Learning Approach

Predicting battery end of life from solar off-grid system field data using machine learning

Hundreds of millions of people lack access to electricity. Decentralised solar-battery systems are key for addressing this whilst avoiding carbon emissions and air pollution, but are hindered by relatively high costs and rural locations that inhibit timely preventative maintenance. Accurate diagnosis of battery health and prediction of end of life from operational data improves user experience and reduces costs. But lack of controlled validation tests and variable data quality mean existing lab-based techniques fail to work. We apply a scaleable probabilistic machine learning approach to diagnose health in 1027 solar-connected lead-acid batteries, each running for 400-760 days, totalling 620 million data rows. We demonstrate 73% accurate prediction of end of life, eight weeks in advance, rising to 82% at the point of failure. This work highlights the opportunity to estimate health from existing measurements using `big data' techniques, without additional equipment, extending lifetime and improving performance in real-world applications.

A Probabilistic Machine Learning Approach to Detect Industrial Plant Faults

Fault detection in industrial plants is a hot research area as more and more sensor data are being collected throughout the industrial process. Automatic data-driven approaches are widely needed and seen as a promising area of investment. This paper proposes an effective machine learning algorithm to predict industrial plant faults based on classification methods such as penalized logistic regression, random forest and gradient boosted tree. A fault’s start time and end time are predicted sequentially in two steps by formulating the original prediction problems as classification problems. The algorithms described in this paper won first place in the Prognostics and Health Management Society 2015 Data Challenge.

Backstepping Based Super-Twisting Sliding Mode MPPT Control with Differential Flatness Oriented Observer Design for Photovoltaic System

The formulation of a maximum power point tracking (MPPT) control strategy plays a vital role in enhancing the inherent low conversion efficiency of a photovoltaic (PV) module. Keeping in view the nonlinear electrical characteristics of the PV module as well as the power electronic interface, in this paper, a hybrid nonlinear sensorless observer based robust backstepping super-twisting sliding mode control (BSTSMC) MPPT strategy is formulated to optimize the electric power extraction from a standalone PV array, connected to a resistive load through a non-inverting DC–DC buck-boost power converter. The reference peak power voltage is generated via the Gaussian process regression (GPR) based probabilistic machine learning approach that is adequately tracked by the proposed MPPT scheme. A generalized super-twisting algorithm (GSTA) based differential flatness approach (DFA) is used to retrieve all the missing system states. The Lyapunov stability theory is used for guaranteeing the stability of the proposed closed-loop MPPT technique. The Matlab/Simulink platform is used for simulation, testing and performance validation of the proposed MPPT strategy under different weather conditions. Its MPPT performance is further compared with the recently proposed benchmark backstepping based MPPT control strategy and the conventional MPPT strategies, namely, sliding mode control (SMC), proportional integral derivative (PID) control and the perturb-and-observe (P&O) algorithm. The proposed technique is found to have a superior tracking performance in terms of offering a fast dynamic response, finite-time convergence, minute chattering, higher tracking accuracy and having more robustness against plant parametric uncertainties, load disturbances and certain time-varying sinusoidal faults occurring in the system.

Learning Coarse-Grained Potentials for Binary Fluids.

For a multiple-fluid system, CG models capable of accurately predicting the interfacial properties as a function of curvature are still lacking. In this work, we propose a new probabilistic machine learning (ML) model for learning CG potentials for binary fluids. The water-hexane mixture is selected as a typical immiscible binary liquid-liquid system. We develop a new CG force field (FF) using the Shinoda-DeVane-Klein (SDK) FF framework and compute parameters in this CG FF using the proposed probabilistic ML method. It is shown that a standard response-surface approach does not provide a unique set of parameters, as it results in a loss function with multiple shallow minima. To address this challenge, we develop a probabilistic ML approach where we compute the probability density function (PDF) of parameters that minimize the loss function. The PDF has a well-defined peak corresponding to a unique set of parameters in the CG FF that reproduces the desired properties of a liquid-liquid interface. We compare the performance of the new CG FF with several existing FFs for the water-hexane mixture, including two atomistic and three CG FFs with respect to modeling the interface structure and thermodynamic properties. It is demonstrated that the new FF significantly improves the CG model prediction of both the interfacial tension and structure for the water-hexane mixture.

A Probabilistic Machine Learning Approach to Scheduling Parallel Loops with Bayesian Optimization

This paper proposes Bayesian optimization augmented factoring self-scheduling (BO FSS), a new parallel loop scheduling strategy. BO FSS is an automatic tuning variant of the factoring self-scheduling (FSS) algorithm and is based on Bayesian optimization (BO), a black-box optimization algorithm. Its core idea is to automatically tune the internal parameter of FSS by solving an optimization problem using BO. The tuning procedure only requires online execution time measurement of the target loop. In order to apply BO, we model the execution time using two Gaussian process (GP) probabilistic machine learning models. Notably, we propose a locality-aware GP model, which assumes that the temporal locality effect resembles an exponentially decreasing function. By accurately modeling the temporal locality effect, our locality-aware GP model accelerates the convergence of BO. We implemented BO FSS on the GCC implementation of the OpenMP standard and evaluated its performance against other scheduling algorithms. Also, to quantify our method's performance variation on different workloads, or workload-robustness in our terms, we measure the minimax regret. According to the minimax regret, BO FSS shows more consistent performance than other algorithms. Within the considered workloads, BO FSS improves the execution time of FSS by as much as 22% and 5% on average.

Information Theory Based Probabilistic Machine Learning And Wireline Inversion: Surat Basin Case Study

We present a probabilistic machine learning approach to determine lithologies for the wireline data from the Springbok Sandstone formation in the Surat Basin. Deterministic inversions of the data a...

Probabilistic machine learning approach to bridge fatigue failure analysis due to vehicular overloading

With the rapid development of freight transportation, truck overloading becomes very common and severe, posing a great threat to the safety of bridges, and it can even result in bridge failure. Traditional approaches investigating the overloading-induced fatigue damage on bridges, such as finite element analysis (FEA) and reliability analysis, have proven to be computationally expensive and model dependent. In this study, the prediction of fatigue failure probability of bridges due to traffic overloading was investigated by using the feedforward neural network and the Monte Carlo method. Our results show that based on a finite set of training data for the bridge under consideration, the proposed machine-learning-based approach can assist in providing an instantaneous assessment of the fatigue failure probability with high accuracy.