Learning Probabilistic Models Research Articles

Photometric Redshift Estimation of Quasars by a Cross-modal Contrast Learning Method

Estimating photometric redshifts (photo-z) of quasars is crucial for measuring cosmic distances and monitoring cosmic evolution. While numerous point estimation methods have successfully determined photo-z, they often struggle with the inherently ill-posed nature of the problem and frequently overlook significant morphological features in the probability density functions (pdfs) of photo-z, such as calibration and sharpness. To address these challenges, we introduce a cross-modal contrastive learning probabilistic model that employs adversarial training, contrastive loss functions, and a mixture density network to estimate the pdf of photo-z. This method facilitates the conversion between multiband photometric data attributes, such as magnitude and color, and photometric image features, while extracting features invariant across modalities. We utilize the continuous ranked probability score (CRPS) and the probability integral transform (PIT) as metrics to assess the quality of the pdf. Our approach demonstrates robust performance across various survey bands, image qualities, and redshift distributions. Specifically, in a comprehensive data set from the Sloan Digital Sky Survey and the Wide-field Infrared Survey Explorer (WISE) survey, our probabilistic model achieved a CRPS of 0.1187. Additionally, in a combined data set from SkyMapper and WISE, it reached a CRPS of 0.0035. Our probabilistic model also produced well-calibrated PIT histograms for both data sets, indicating nearly uniform distributions. We further tested our approach in classification tasks within the SkyMapper data set. Despite the absence of u, v, and g bands, it effectively distinguished between quasars, galaxies, and stars with an accuracy of 98.96%. This versatile method can be extended to other scenarios, such as analyzing extended sources like galaxies, across different surveys and varying redshift distributions.

Deep Learning‐Based Prediction of Final Infarct Core from CT Perfusion Data: A Comparison to the Clinical Standard

Background In the management of acute ischemic stroke, computed tomography perfusion (CTP) is used to define the ischemic core and penumbra to estimate tissue fate after reperfusion therapy. The core/penumbra dichotomy uses single‐value thresholds, which potentially discards valuable data and oversimplifies the complexity of core and penumbral estimation. Advancing the dichotomous CTP output to a probability model has several advantages such as more sophisticated modeling of pathophysiology, supporting reader interpretation, and assessing a greater range of available data to estimate tissue fate. Methods In this retrospective study, we developed a CTP probability model to move away from single perfusion thresholds to estimate tissue fate. All patients from the International Stroke Perfusion Imaging Registry database had baseline CTP and were included in the current study if they had a large vessel occlusion that recanalized fully after thrombectomy and had follow‐up diffusion‐weighted imaging. Data were split into training, validation, and testing groups. Training and validation cohorts were used to develop a deep learning model in project MONAI (Medical Open Network for Artificial Intelligence) and performance metrics were derived from the testing set. Results In total, 243 patients were included in the study. The Attention U‐Net was the best performing deep learning model, producing the best prediction of follow‐up infarct core on the test set (n = 48): mean diverse counterfactual explanations score = 0.430±0.213, mean area under the curve = 0.765±0.095; better than the single‐value thresholding with a diverse counterfactual explanations scoreof 0.247±0.167 (paired t ‐test, P <0.0001) and area under the curveof 0.604±0.074 ( P <0.0001). Conclusion The deep learning probabilistic CTP model outperformed the current clinical standard, providing a more accurate core estimate than single‐threshold‐based measures.

Probabilistic machine learning to improve generalisation of data-driven turbulence modelling

A probabilistic machine learning model is introduced to augment the k−ωSST turbulence model in order to improve the modelling of separated flows and the generalisability of learnt corrections. Increasingly, machine learning methods have been used to leverage experimental and high-fidelity simulation data, improving the accuracy of the Reynolds Averaged Navier–Stokes (RANS) turbulence models widely used in industry. A significant challenge for such methods is their ability to generalise to unseen geometries and flow conditions. Furthermore, heterogeneous datasets containing a mix of experimental and simulation data must be efficiently handled. In this work, field inversion and an ensemble of Gaussian Process Emulators (GPEs) is employed to address both of these challenges. The ensemble model is applied to a range of benchmark test cases, demonstrating improved turbulence modelling for cases involving separated flows with adverse pressure gradients, where RANS simulations are understood to be unreliable. Perhaps more significantly, the simulation reverted to the uncorrected model in regions of the flow exhibiting physics outside of the training data.

Choosing the right treatment - combining clinicians' expert knowledge with data-driven predictions.

This study proposes a Bayesian network model to aid mental health specialists making data-driven decisions on suitable treatments. The aim is to create a probabilistic machine learning model to assist psychologists in selecting the most suitable treatment for individuals for four potential mental disorders: Depression, Panic Disorder, Social Phobia, or Specific Phobia. This study utilized a dataset from 1,094 individuals in Denmark containing socio-demographic details and mental health information. A Bayesian network was initially employed in a purely data-driven approach and was later refined with expert knowledge, referred to as a hybrid model. The model outputted probabilities for each disorder, with the highest probability indicating the most suitable disorder for treatment. By incorporating expert knowledge, the model demonstrated enhanced performance compared to a strictly data-driven approach. Specifically, it achieved an AUC score of 0.85 vs 0.80 on the test data. Furthermore, we evaluated some cases where the predictions of the model did not match the actual treatment. The symptom questionnaires indicated that these participants likely had comorbid disorders, with the actual treatment being proposed by the model with the second highest probability. In 90.1% of cases, the hybrid model ranked the actual disorder treated as either the highest (67.3%) or second-highest (22.8%) on the test data. This emphasizes that instead of suggesting a single disorder to be treated, the model can offer the probabilities for multiple disorders. This allows individuals seeking treatment or their therapists to incorporate this information as an additional data-driven factor when collectively deciding on which treatment to prioritize.

Assessment of Soccer Teaching Ability Based on Deep Learning Algorithm

Soccer teaching involves imparting fundamental skills, tactics, and strategies related to the sport of soccer. Coaches and instructors focus on teaching players proper techniques for dribbling, passing, shooting, and defending, while also emphasizing teamwork, sportsmanship, and game awareness. Soccer teaching sessions typically include a combination of drills, scrimmages, and tactical discussions tailored to the age and skill level of the players. This paper proposed an effective soccer assessment technique for teaching ability with the Automated Probabilistic Deep Learning (APDL) model. The proposed APDL model comprises an automated model for the assessment of student performance. The proposed APDL model processes the input soccer images with the pre-processing of the features. With APDL model uses the probabilistic computation features for the computation of the variables in the soccer data. With the extraction of the features, the maximum likelihood is computed for the classification with the deep learning model features. The APDL model implements the classification-based deep learning model features with the examination of soccer teaching, instruction, development of skill, and players. Simulation results demonstrated that prediction with the APDL model estimates the probability of prediction as 0.91 with an estimated uncertainty value of 0.08. In the case of teaching and coaching assessment, the uncertainty is stated as 0.08 for both with the prediction assessment score of 0.92. The classification accuracy of the proposed APDL model is achieved as 0.95 with the precision value of 0.97. The findings of this research contribute to the advancement of coaching methodologies in soccer, providing coaches and educators with valuable insights into their teaching effectiveness and areas for improvement. Additionally, the automated nature of the APDL model offers scalability and efficiency in assessing coaching performance, paving the way for enhanced coaching practices and player development in soccer.

An artificial intelligence-enabled electrocardiogram algorithm for the prediction of left atrial low-voltage areas in persistent atrial fibrillation.

We aimed to construct an artificial intelligence-enabled electrocardiogram (ECG) algorithm that can accurately predict the presence of left atrial low-voltage areas (LVAs) in patients with persistent atrial fibrillation. The study included 587 patients with persistent atrial fibrillation who underwent catheter ablation procedures between March 2012 and December 2023 and 942 scanned images of 12-lead ECGs obtained before the ablation procedures were performed. Artificial intelligence-based algorithms were used to construct models for predicting the presence of LVAs. The DR-FLASH and APPLE clinical scores for LVA prediction were calculated. We used a receiver operating characteristic (ROC) curve, calibration curve, and decision curve analysis to evaluate model performance. The data obtained from the participants were split into training (n = 469), validation (n = 58), and test sets (n = 60). LVAs were detected in 53.7% of all participants. Using ECG alone, the deep learning algorithm achieved an area under the ROC curve (AUROC) of 0.752, outperforming both the DR-FLASH score (AUROC = 0.610) and the APPLE score (AUROC = 0.510). The random forest classification model, which integrated a probabilistic deep learning model and clinical features, showed a maximum AUROC of 0.759. Moreover, the ECG-based deep learning algorithm for predicting extensive LVAs achieved an AUROC of 0.775, with a sensitivity of 0.816 and a specificity of 0.896. The random forest classification model for predicting extensive LVAs achieved an AUROC of 0.897, with a sensitivity of 0.862, and a specificity of 0.935. The deep learning model based exclusively on ECG data and the machine learning model that combined a probabilistic deep learning model and clinical features both predicted the presence of LVAs with a higher degree of accuracy than the DR-FLASH and the APPLE risk scores.

Physically Explainable Deep Learning for Convective Initiation Nowcasting Using GOES-16 Satellite Observations

Abstract Convective initiation (CI) nowcasting remains a challenging problem for both numerical weather prediction models and existing nowcasting algorithms. In this study, an object-based probabilistic deep learning model is developed to predict CI based on multichannel infrared GOES-16 satellite observations. The data come from patches surrounding potential CI events identified in Multi-Radar Multi-Sensor Doppler weather radar products over the Great Plains region from June and July 2020 and June 2021. An objective radar-based approach is used to identify these events. The deep learning model significantly outperforms the classical logistic model at lead times up to 1 h, especially on the false alarm ratio. Through case studies, the deep learning model exhibits dependence on the characteristics of clouds and moisture at multiple altitudes. Model explanation further reveals that the contribution of features to model predictions is significantly dependent on the baseline, a reference point against which the prediction is compared. Under a moist baseline, moisture gradients in the lower and middle troposphere contribute most to correct CI forecasts. In contrast, under a clear-sky baseline, correct CI forecasts are dominated by cloud-top features, including cloud-top glaciation, height, and cloud coverage. Our study demonstrates the advantage of using different baselines in further understanding model behavior and gaining scientific insights.

Water quality level estimation using IoT sensors and probabilistic machine learning model

ABSTRACT Drinking water purity analysis is an essential framework that demands several real-world parameters to ensure the quality of water. So far, sensor-based analysis of water quality in specific environments is done concerning certain parameters including the PH level, hardness, TDS, etc. The outcome of such methods analyzes whether the environment provides potable water or not. Potable denotes the purified water that is free from all contaminations. This analysis gives an absolute solution whereas the demand for drinking water is a growing problem where the multiple-level estimations are essential to use the available water resources efficiently. In this article, we used a benchmark water quality assessment dataset for analysis. To perform a level assessment, we computed three major features namely correlation-entropy, dynamic scaling, and estimation levels, and annexed with the earlier feature vector. The assessment of the available data was performed using the statistical machine learning model that ensembles the random forest model and light gradient boost model (GBM). The probability of the ensemble model was done by the Kullback Libeler Divergence model. The proposed probabilistic model has achieved an accuracy of 96.8%, a sensitivity of 94.55%, and a specificity of 98.29%.

Probabilistic impact localization in composites using wavelet scattering transform and multi-output Gaussian process regression

Data-driven machine-learning models offer considerable promise for acoustic source localization. However, many existing models rely on training data that correlates time-of-flight (TOF) measurements with source locations, yet they struggle to handle the complexities arising from nonlinear wave propagation in materials with varying properties. Furthermore, these models overlook the noise and uncertainties inherent in real-world experiments when predicting outputs. This paper aims to bridge a gap in impact localization for such structures, particularly focusing on scenarios involving noisy field measurements. This study proposes a framework based on probabilistic machine learning to identify impact locations, utilizing wavelet scattering transform (WST) and Multi-Output Gaussian Process Regression (moGPR). WST extracts informative features from Lamb waves, capturing relevant signatures for training the probabilistic machine learning model, while moGPR estimates correlated impact location coordinates (x, y) while accounting for inherent uncertainties in the data. To assess the proposed method’s performance in handling measurement uncertainties, an experiment was conducted using a CFRP composite panel instrumented with a sparse array of piezoelectric transducers. The results demonstrate that the probabilistic framework effectively addresses measurement uncertainties, enabling reliable source location estimation with confidence intervals and providing valuable insights for decision-making.

A Survey on Software Vulnerability Exploitability Assessment

Knowing the exploitability and severity of software vulnerabilities helps practitioners prioritize vulnerability mitigation efforts. Researchers have proposed and evaluated many different exploitability assessment methods. The goal of this research is to assist practitioners and researchers in understanding existing methods for assessing vulnerability exploitability through a survey of exploitability assessment literature. We identify three exploitability assessment approaches: assessments based on original, manual Common Vulnerability Scoring System, automated Deterministic assessments, and automated Probabilistic assessments. Other than the original Common Vulnerability Scoring System, the two most common sub-categories are Deterministic, Program State based, and Probabilistic learning model assessments.

Accurate and robust predictions of pool boiling heat transfer with micro-structured surfaces using probabilistic machine learning models

The accurate and reliable prediction of enhanced heat transfer performance of micro-structured surfaces is crucial to optimally design and operate pool boiling systems. However, the existing empirical models predict the enhanced pool boiling heat transfer with very large errors up to ±81 % even using the experimental data from the same study, mainly due to the complex nature of the pool boiling processes. More importantly, the existing models predict only limited coolant types, surface geometries, and operating conditions. To overcome these challenges, this study examines three deterministic and two probabilistic machine learning (ML) models for accurate and reliable enhanced pool boiling prediction, while using carefully selected key six dimensionless numbers. The models were trained and tested using 1,241 data from 20 experimental studies with 80/20 % of train/test ratio, and the pre-trained models were tested for additional 519 data from 6 studies to evaluate the models reliability. The predicted mean absolute percentage error (MAPE) shows that Bayesian, deep, and 1-D convolutional neural network (1D-CNN) models outperform the random forest and natural gradient (NG) boost models due to their extended hidden layers. The machine learning models improve the MAPE by up to 30 % compared to the existing correlations. Furthermore, a parameter sensitivity analysis is performed using explainable artificial intelligence showing that the boiling Reynolds number is the most critical parameter followed by the kinetic Reynolds and Bond numbers. The probabilistic ML models can also provide the uncertainties to improve prediction reliability compared to the deterministic ones.

Improving Photometric Redshift Estimation for Cosmology with LSST Using Bayesian Neural Networks

We present results exploring the role that probabilistic deep learning models can play in cosmology from large-scale astronomical surveys through photometric redshift (photo-z) estimation. Photo-z uncertainty estimates are critical for the science goals of upcoming large-scale surveys such as the Legacy Survey of Space and Time (LSST); however, common machine learning methods typically provide only point estimates and lack uncertainties on predictions. We turn to Bayesian neural networks (BNNs) as a promising way to provide accurate predictions of redshift values with uncertainty estimates. We have compiled a galaxy data set from the Hyper Suprime-Cam Survey with grizy photometry, which is designed to be a smaller-scale version of large surveys like LSST. We use this data set to investigate the performance of a neural network and a probabilistic BNN for photo-z estimation and evaluate their performance with respect to LSST photo-z science requirements. We also examine the utility of photo-z uncertainties as a means to reduce catastrophic outlier estimates. The BNN outputs the estimate in the form of a Gaussian probability distribution. We use the mean and standard deviation as the redshift estimate and uncertainty. We find that the BNN can produce accurate uncertainties. Using a coverage test, we find excellent agreement with expectation—67.2% of galaxies between 0 < 2.5 have 1σ uncertainties that cover the spectroscopic value. We also include a comparison to alternative machine learning models using the same data. We find the BNN meets two out of three of the LSST photo-z science requirements in the range 0 < z < 2.5.

Simplifying Complex Observation Models in Continuous POMDP Planning with Probabilistic Guarantees and Practice

Solving partially observable Markov decision processes (POMDPs) with high dimensional and continuous observations, such as camera images, is required for many real life robotics and planning problems. Recent researches suggested machine learned probabilistic models as observation models, but their use is currently too computationally expensive for online deployment. We deal with the question of what would be the implication of using simplified observation models for planning, while retaining formal guarantees on the quality of the solution. Our main contribution is a novel probabilistic bound based on a statistical total variation distance of the simplified model. We show that it bounds the theoretical POMDP value w.r.t. original model, from the empirical planned value with the simplified model, by generalizing recent results of particle-belief MDP concentration bounds. Our calculations can be separated into offline and online parts, and we arrive at formal guarantees without having to access the costly model at all during planning, which is also a novel result. Finally, we demonstrate in simulation how to integrate the bound into the routine of an existing continuous online POMDP solver.

A Novel Distributed Process Monitoring Framework of VAE-Enhanced with Deep Neural Network

Intelligent manufacturing process needs to adopt distributed monitoring scenario due to its massive, high-dimensional and complex data. Distributed process monitoring has been introduced into global monitoring and local monitoring to analyze the characteristic relationship between process data. However, the existing framework methods ignore or suppress the fault information and thus cannot effectively identify the local faults and the time sequence characteristics between units in the batch production system. This paper proposes a novel distributed process monitoring framework based on Girvan-Newman algorithm modular subunit partitioning and probabilistic learning model with deep neural networks. First, Girvan-Newman algorithm is used to divide the complex manufacturing system modularized to reduce the latitude of data processing. Second, variational autoencoder (VAE) is adopted to ensure the stability of local analysis, and long short-term memory is adopted to improve the VAE model to detect global multi-time scale anomalies. Finally, distributed process fault detection is carried out for each subunit in a separate and integrated manner, and the performance of the framework in distributed process monitoring is analyzed through two fault detection indicators T2 and SPE statistics. A case study of the Tennessee Eastman Process is used to demonstrate the performance and applicability of the proposed framework. Results show that the proposed VAE enhancement framework based on the DNN could accurately identify faults in distributed process monitoring and locate the specific sub-units where the fault occurs. Compared with VAE-DNN method and traditional process monitoring methods, the framework proposed in this paper has higher fault detection rate and lower false alarm rate, and the detection rate of some faults can reach 100%.

Long-term cost planning of data-driven wind-storage hybrid systems

—To address the long-term cost planning problem accurately, typically a significant number of formulas are required. However, this increases computational complexity and limits the generalizability of the approach to practical problems. To overcome these challenges, this study adopts a data-driven approach that considers uncertainties to evaluate the long-term cost planning problem accurately for wind power generation with hybrid energy storage. A method for predicting wind power output is proposed using temporal convolutional networks to handle long-term uncertainties. Additionally, an instantaneous phase-synchronous wind power output feature extraction method based on Hilbert transform theory is introduced to capture time-series fluctuation characteristics and address short-term uncertainties. Furthermore, the long-term operational costs and planning of high-percentage new energy wind farms with hybrid energy storage are discussed by combining the wind power output characteristics with the operating characteristics and lifespan of hybrid energy storage devices. The experimental results demonstrate the significant advantages of the deep learning probabilistic prediction model combined with wind power feature extraction. The algorithm validates the effectiveness of considering multiple uncertainties in the planning of microgrids with hybrid energy storage, particularly in considering the uncertainty of the decreasing trend in the investment cost of battery energy storage.

Deep learning denoising diffusion probabilistic model applied to holographic data synthesis.

In this Letter, we demonstrate for the first time, to our knowledge, a holographic data synthesis based on a deep learning probabilistic diffusion model (DDPM). Several different datasets of color images corresponding to different types of objects are converted to complex-valued holographic data through backpropagation. Then, we train a DDPM using the resulting holographic datasets. The diffusion model is composed of a noise scheduler, which gradually adds Gaussian noise to each hologram in the dataset, and a U-Net convolutional neural network that is trained to reverse this process. Once the U-Net is trained, any number of holograms with similar features as those of the datasets can be generated just by inputting a Gaussian random noise to the model. We demonstrate the synthesis of holograms containing color images of 2D characters, vehicles, and 3D scenes with different characters at different propagation distances.

Evaluation and analysis of liquefaction potential of gravelly soils using explainable probabilistic machine learning model

Majority of the presently available machine learning (ML) models employed to assess the liquefaction potential of soils are for sands or sands containing silt fraction. In the current study, an explainable ML (EML) model has been developed using the updated liquefaction database of gravelly soils. The Chinese dynamic cone penetration test (DPT) and shear wave velocity test results of gravelly soils are used in the analysis. A new empirical correlation between these two in-situ tests’ results is developed using the final processed database. The light gradient boosting machine (LightGBM) is trained using this processed dataset and further tuned using Fast and Lightweight AutoML library (FLAML). The final tuned model shows relatively better deterministic and probabilistic predictive performance for the test sites as compared to the conventional method. An EML technique, SHapley Additive exPlanations (SHAP) is applied to provide further comprehension into the predictions. The developed LightGBM-SHAP model has achieved a right balance between explainability and accuracy. The obtained SHAP plots are consistent with almost all the existing domain knowledge (DK) in gravelly soil liquefaction. The developed model thus fills the gap between the ML-based procedures and the traditional DK of gravelly soil liquefaction.

Learning Generative Models Using Denoising Density Estimators.

Learning probabilistic models that can estimate the density of a given set of samples, and generate samples from that density, is one of the fundamental challenges in unsupervised machine learning. We introduce a new generative model based on denoising density estimators (DDEs), which are scalar functions parametrized by neural networks, that are efficiently trained to represent kernel density estimators of the data. Leveraging DDEs, our main contribution is a novel technique to obtain generative models by minimizing the Kullback-Leibler (KL)-divergence directly. We prove that our algorithm for obtaining generative models is guaranteed to converge consistently to the correct solution. Our approach does not require specific network architecture as in normalizing flows (NFs), nor use ordinary differential equation (ODE) solvers as in continuous NFs. Experimental results demonstrate substantial improvement in density estimation and competitive performance in generative model training.

Inter-Frame Compression for Dynamic Point Cloud Geometry Coding.

Efficient point cloud compression is essential for applications like virtual and mixed reality, autonomous driving, and cultural heritage. This paper proposes a deep learning-based inter-frame encoding scheme for dynamic point cloud geometry compression. We propose a lossy geometry compression scheme that predicts the latent representation of the current frame using the previous frame by employing a novel feature space inter-prediction network. The proposed network utilizes sparse convolutions with hierarchical multiscale 3D feature learning to encode the current frame using the previous frame. The proposed method introduces a novel predictor network for motion compensation in the feature domain to map the latent representation of the previous frame to the coordinates of the current frame to predict the current frame's feature embedding. The framework transmits the residual of the predicted features and the actual features by compressing them using a learned probabilistic factorized entropy model. At the receiver, the decoder hierarchically reconstructs the current frame by progressively rescaling the feature embedding. The proposed framework is compared to the state-of-the-art Video-based Point Cloud Compression (V-PCC) and Geometry-based Point Cloud Compression (G-PCC) schemes standardized by the Moving Picture Experts Group (MPEG). The proposed method achieves more than 88% BD-Rate (Bjøntegaard Delta Rate) reduction against G-PCCv20 Octree, more than 56% BD-Rate savings against G-PCCv20 Trisoup, more than 62% BD-Rate reduction against V-PCC intra-frame encoding mode, and more than 52% BD-Rate savings against V-PCC P-frame-based inter-frame encoding mode using HEVC. These significant performance gains are cross-checked and verified in the MPEG working group.

Are Neighbors Alike? A Semisupervised Probabilistic Collaborative Learning Model for Online Review Spammers Detection

Online platforms continually face the menace of review spammers who manipulate ratings and comments for personal gain, eroding trust in online reviews. Despite various methods proposed to tackle this issue, challenges, like collusion between spammers, scarce labels, and skewed data distribution, persist. This paper introduces a practical solution to these problems by integrating individual behavioral features with a reviewer network, and it proposes a novel semisupervised collaborative learning model. Imagine our approach as a detective tool for online platforms. It works by investigating who is reviewing what and how they are linked within the reviewer network, which is like uncovering hidden relationships among reviewers to catch spammers. Through this method, our model significantly boosts the accuracy and transparency in identifying spammers. We tested this approach on real-world review data from different types of online platforms, and it outperformed other methods. Importantly, the model is resilient, even when you have limited data or when some labels might not be perfect. By using our approach, online platforms can spot fake reviews and spammers in a more cost-effective way, ensuring that the reviews are more reliable and that it is easier for consumers to make informed decisions. It is a practical step forward in the fight against fake reviews online.