Probabilistic Learning Methods Research Articles

Long‐Term Foehn Reconstruction Combining Unsupervised and Supervised Learning

ABSTRACTFoehn winds, characterised by abrupt temperature increases and wind speed changes, significantly impact regions on the leeward side of mountain ranges, e.g., by spreading wildfires. Understanding how foehn occurrences change under climate change is crucial. As foehn is a meteorological phenomenon, its prevalence has to be inferred from meteorological measurements employing suitable classification schemes. Hence, this approach is typically limited to specific periods for which the necessary data are available. We present a novel approach for reconstructing historical foehn occurrences using a combination of unsupervised and supervised probabilistic statistical learning methods. We utilise in situ measurements (available for recent decades) to train an unsupervised learner (finite mixture model) for automatic foehn classification. These labelled data are then linked to reanalysis data (covering longer periods) using a supervised learner (lasso or boosting). This allows us to reconstruct past foehn probabilities based solely on reanalysis data. Applying this method to ERA5 reanalysis data for six stations across Switzerland and Austria achieves accurate hourly reconstructions of north and south foehn occurrence, respectively, dating back to 1940. This paves the way for investigating how seasonal foehn patterns have evolved over the past 83 years, providing valuable insights into climate change impacts on these critical wind events.

Uncertainty Quantification in Mineral Resource Estimation

AbstractMineral resources are estimated to establish potential orebody with acceptable quality (grade) and quantity (tonnage) to validate investment. Estimating mineral resources is associated with uncertainty from sampling, geological heterogeneity, shortage of knowledge and application of mathematical models at sampled and unsampled locations. The uncertainty causes overestimation or underestimation of mineral deposit quality and/or quantity, affecting the anticipated value of a mining project. Therefore, uncertainty is assessed to avoid any likely risks, establish areas more prone to uncertainty and allocate resources to scale down potential consequences. Kriging, probabilistic, geostatistical simulation and machine learning methods are used to estimate mineral resources and assess uncertainty, and their applicability depends on deposit characteristics, amount of data available and expertise of technical personnel. These methods are scattered in the literature making them challenging to access when needed for uncertainty quantification. Therefore, this review aims to compile information about uncertainties in mineral resource estimation scatted in the literature and develop a knowledge base of methodologies for uncertainty quantification. In addition, mineral resource estimation comprises different interdependent steps, in and through which uncertainty accumulates and propagates toward the final estimate. Hence, this review demonstrates stepwise uncertainty propagation and assessment through various phases of the estimation process. This can broaden knowledge about mineral resource estimation and uncertainty assessment in each step and increase the accuracy of mineral resource estimates and mining project viability.

Deep Image: A precious image based deep learning method for online malware detection in IoT environment

In this study, we address the challenge of online malware detection for IoT devices. We propose a method that monitors malware behavior, extracts dynamic features, and converts them into sparse binary images for analysis. The primary problem is to identify the most effective approach among clustering, probabilistic, and deep learning methods for analyzing this unique image dataset. We extract dynamic features from the monitored malware behavior, transforming them into binary images, which are then subjected to three different analysis methods. The clustering, probabilistic, and deep learning approaches are compared and evaluated in terms of various metrics. Our study contributes insights into the performance of various online malware detection approaches for IoT devices. We demonstrate that deep learning outperforms other methods, achieving the best results in seven out of eight metrics. The results of our analysis reveal that the deep learning approach exhibits the highest accuracy in seven of the eight evaluated metrics. We found that the lattice-based approach consistently returns the maximum maliciousness level, which can be instrumental in label flipping scenarios.

Multimodal Shot Prediction Based on Spatial-Temporal Interaction between Players in Soccer Videos

Sports data analysis has significantly advanced and become an indispensable technology for planning strategy and enhancing competitiveness. In soccer, shot prediction has been realized on the basis of historical match situations, and its results contribute to the evaluation of plays and team tactics. However, traditional event prediction methods required tracking data acquired with expensive instrumentation and event stream data annotated by experts, and the benefits were limited to only some professional athletes. To tackle this problem, we propose a novel shot prediction method using soccer videos. Our method constructs a graph considering player relationships with audio and visual features as graph nodes. Specifically, by introducing players’ importance into the graph edge based on their field positions and team information, our method enables the utilization of knowledge that reflects the detailed match situation. Next, we extract latent features considering spatial–temporal interactions from the graph and predict event occurrences with uncertainty based on the probabilistic deep learning method. In comparison with several baseline methods and ablation studies using professional soccer match data, our method was confirmed to be effective as it demonstrated the highest average precision of 0.948, surpassing other methods.

Shape completion in the dark: completing vertebrae morphology from 3D ultrasound

PurposeUltrasound (US) imaging, while advantageous for its radiation-free nature, is challenging to interpret due to only partially visible organs and a lack of complete 3D information. While performing US-based diagnosis or investigation, medical professionals therefore create a mental map of the 3D anatomy. In this work, we aim to replicate this process and enhance the visual representation of anatomical structures.MethodsWe introduce a point cloud-based probabilistic deep learning (DL) method to complete occluded anatomical structures through 3D shape completion and choose US-based spine examinations as our application. To enable training, we generate synthetic 3D representations of partially occluded spinal views by mimicking US physics and accounting for inherent artifacts.ResultsThe proposed model performs consistently on synthetic and patient data, with mean and median differences of 2.02 and 0.03 in Chamfer Distance (CD), respectively. Our ablation study demonstrates the importance of US physics-based data generation, reflected in the large mean and median difference of 11.8 CD and 9.55 CD, respectively. Additionally, we demonstrate that anatomical landmarks, such as the spinous process (with reconstruction CD of 4.73) and the facet joints (mean distance to ground truth (GT) of 4.96 mm), are preserved in the 3D completion.ConclusionOur work establishes the feasibility of 3D shape completion for lumbar vertebrae, ensuring the preservation of level-wise characteristics and successful generalization from synthetic to real data. The incorporation of US physics contributes to more accurate patient data completions. Notably, our method preserves essential anatomical landmarks and reconstructs crucial injections sites at their correct locations.

Modeling and material uncertainty quantification of RC structural components

It is well established that various sources of uncertainties play a critical role in the safety assessment of engineering structures. Some widely used frameworks, such as performance-based earthquake engineering (PBEE), explicitly consider the ground motion record-to-record randomness, while the material and modeling uncertainty remain to be primarily based on judgments or limited analysis. This paper presents the results of a comprehensive uncertainty quantification and sensitivity analysis of a reinforced concrete structural component. First, different modeling strategies are adopted to develop several parent models. Next, various sources of uncertainty are propagated through the parent models to generate thousands of children models. The children models are further combined with material uncertainty to produce grandchildren models, and nonlinear transient simulations are conducted using an innovative artificial acceleration at different seismic intensity levels. The results are post-processed using a range of probabilistic, statistical, and machine learning methods. The study finds that the modeling strategy and its associated variability can cause significant bias and dispersion in the drift response, while material uncertainty has a relatively minor effect. The study quantifies the importance of modeling uncertainty, which is often overlooked in engineering practice.

Identification of concrete surface damage based on probabilistic deep learning of images

Crack is common damage that can reduce the durability of concrete structures and accelerate structural degradation. With intent to improve the accuracy and efficiency of image-based crack detection, deep learning has been extensively used. However, the current deep learning paradigms for this purpose are deterministic machine learning approaches that cannot evaluate the uncertainty of diagnosis results. To bridge this gap, this paper proposes a hybrid probabilistic deep learning method for concrete surface damage classification. In the proposed method, Bayesian inference is embedded in a deep convolutional neural network, and the parameters in probability layers are modified from deterministic values to Gaussian distributions. Experimental results showed that it can identify cracks with an accuracy of 0.9909. The quantitative indicator of uncertainty in the identification results increases with the noise level and the degree of occlusion, which can reflect the ability of the model to capture uncertainty well. In addition, the hybrid probability method based on transfer learning can significantly improve the recognition accuracy, which can reach 0.9249 in complex crack images. Even if the image contains more noise, it can be recognized accurately but the uncertainty indicator is greater than 0.07.

Probabilistic and machine learning methods for uncertainty quantification in power outage prediction due to extreme events

Abstract. Strong hurricane winds damage power grids and cause cascading power failures. Statistical and machine learning models have been proposed to predict the extent of power disruptions due to hurricanes. Existing outage models use inputs including power system information, environmental parameters, and demographic parameters. This paper reviews the existing power outage models, highlighting their strengths and limitations. Existing models were developed and validated with data from a few utility companies and regions, limiting the extent of their applicability across geographies and hurricane events. Instead, we train and validate these existing outage models using power outages from multiple regions and hurricanes, including hurricanes Harvey (2017), Michael (2018), and Isaias (2020), in 1910 US cities. The dataset includes outages from 39 utility companies in Texas, 5 in Florida, 5 in New Jersey, and 11 in New York. We discuss the limited ability of state-of-the-art machine learning models to (1) make bounded outage predictions, (2) extrapolate predictions to high winds, and (3) account for physics-informed outage uncertainties at low and high winds. For example, we observe that existing models can predict outages higher than the number of customers (in 19.8 % of cities with an average overprediction ratio of 5.2) and cannot capture well the outage variance for high winds, especially above 70 m s−1. Our findings suggest that further developments are needed for power outage models for proper representation of hurricane-induced outages.

Physics-Driven Probabilistic Deep Learning for the Inversion of Physical Models With Application to Phenological Parameter Retrieval From Satellite Times Series

Recent Sentinel satellite constellations and deep learning methods offer great possibilities for estimating the states and dynamics of physical parameters on a global scale. Such parameters and their corresponding uncertainties can be retrieved by machine learning methods solving probabilistic inverse problems. Nevertheless, the scarcity of reference data to train supervised methodologies is a well-known constraint for remote sensing applications. To address such limitations, this work presents a new generic physics-guided probabilistic deep learning methodology to invert physical models. The presented methodology proposes a new strategy to combine probabilistic deep learning methods and physical models avoiding simulation-driven machine learning. The inverse problem is addressed through a Bayesian inference framework by proposing a new physically-constrained self-supervised representation learning methodology. To show the interest of the proposed strategy, the methodology is applied to the retrieval of phenological parameters from NDVI time series. As a result, the probability distributions of the intrinsic phenological model parameters are inferred. The feasibility of the method is evaluated on both simulated and real Sentinel-2 data and compared with different standard algorithms. Promising results show satisfactory accuracy predictions and low inference times for real applications.

Modeling Uncertainty Quantification of NDVI of Agricultural Fields through Bayesian Linear Regression in Time Series Prediction

The current research discusses the applications of Bayesian linear regression to predict the uncertainty of remote sensing data. To predict the uncertainty, the study considered the SENTINEL-2 satellite data of agricultural fields of Uttar Pradesh state of India. Using the stratified sampling method in Google Earth Engine, the random points generated are mapped to agricultural fields. Data was collected in the form of maximum Normalized Difference Vegetation Index (NDVI) values of each agricultural field. The dynamics of the time series predictions were explored with Bayesian linear regression, a probabilistic deep learning method. The model uncertainty defined as epistemic uncertainty is evaluated with the prior and posterior probability parameters of Bayesian statistics in linear regression. The number of regression lines predicted for the same data shows evidence of uncertainty. The Bayesian linear regression models show evidence of high uncertainty for the predicted NDVI values. The variation in model uncertainty is measured by dividing the dataset into samples and it is observed that with increase in data the uncertainty is reduced. Also, with the increase in data, the posterior density becomes sharper which corresponds to a decrease in variance. Further, the study extended the concept of regression analysis with Gaussian basis functions to determine the effect of model uncertainty with an increase in data. The analysis has shown the same result in knowing the effect of uncertainty with the increase in data. Further, a nonlinear polynomial regression model with a Gaussian distribution as a basis function was developed to evaluate the marginal probabilities of the evidence function in capturing the uncertainty with varying degrees of freedom. The polynomial regression with a Gaussian distribution using Bayesian statistics has captured the uncertainty and confirmed that the uncertainty is captured at lower degrees of freedom.

Link Characterization and Edge-Centric Predictive Modeling in an Ocean Network

One of the critical problems fishermen face in deep-sea fishing is the lack of low-cost communication mechanisms to the shore. The Offshore Communication Network (OCN) is a network of fishing vessels at sea whose goal is to provide wireless internet over the ocean. The impact of extreme weather conditions on wireless signals, the inability to deploy additional infrastructure, the movements induced by sea waves, the expanded mobility freedom at sea, and the misalignment of directional antenna links are all unique challenges that cause abrupt signal quality fluctuations in OCN. For this reason, it is necessary to integrate near real-time link quality assessment to improve the resilience of communication. This paper examines the characteristics of marine wireless links and the factors impacting communication using data collected through sea-trial experiments involving multiple fishing vessels. The paper proposes a Bayesian framework for forecasting signal strength by employing historical and real-time data. This hybrid learning integrates offline and online probabilistic learning methods to provide intelligence at edge devices. The evaluation of the learning scheme on real datasets and the comparison with baseline methods under different communication contexts show improved predictive accuracy in OCN links.

A Bayesian Mixture Neural Network for Remaining Useful Life Prediction of Lithium-Ion Batteries

Remaining useful life (RUL) is one of the essential ingredients in the battery management system. However, due to the characteristic of the dynamic and time-varying electrochemical system with nonlinear and complicated internal mechanisms, the uncertainty of RUL estimation has been expanded, and it is difficult to give an accurate time to reach the end of life. This article proposes the Bayesian mixture neural network (BMNN), a probabilistic deep learning method, to obtain more accurate RUL prediction and provide uncertainty estimation, while the quasi-Gramian angular field (Q-GAF) beneficial to identify prior distribution is utilized to transform time-series sequence into temporal images. BMNN consists of the Bayesian convolutional neural network (BCNN) extracting features in temporal images and Bayesian long short-term memory (B-LSTM) learning correlation between retention capacity and other degradation inducements. After concatenating two terms, the variational Bayesian neural network outputs the distribution of prediction results. In the experimental stage, the performance of the proposed method is validated on four different lithium-ion battery datasets and demonstrates higher stability, lower uncertainty, and more accuracy than other methods.

Multi-workflow scheduling and resource provisioning in Mobile Edge Computing using opposition-based Marine-Predator Algorithm

Workflow Scheduling in Mobile Edge Computing (MEC) tries to allocate the best possible set of resources for the workflows, considering objectives such as deadline, cost, energy, Quality of Service (QoS), and so on. However, MEC may be under different workloads from the IoT and this may not have the required amount of resources to efficiently handle the workflows. To mitigate this problem, in this paper, we use proactive resource provisioning and workload prediction methods. For this purpose, we present a workload prediction method using a multilayer feed-forward Artificial Neural Network (ANN) model and apply its results for resource provisioning. Afterward, we present an opposition-based version of the Marine-Predator Algorithm (MPA) algorithm, denoted as OMPA. In this algorithm, we present a probabilistic opposition-based learning (OBL) method, which benefits from the OBL, Quasi OBL, and dynamic OBL methods. Afterward, the OMPA algorithm is used for training the multi-layer feed-forward ANN model and multi-workflow scheduling by taking into account factors such as the makespan and number of Virtual Machines (VMs). Extensive experiments conducted in the iFogSim simulator and on the NASA and Saskatchewan datasets indicate that the proposed scheme can achieve better results compared to other metaheuristic algorithms and scheduling schemes.

Cryptanalysis of an optical cryptosystem with uncertainty quantification in a probabilistic model.

In this paper, a modified probabilistic deep learning method is proposed to attack the double random phase encryption by modeling the conditional distribution of plaintext. The well-trained probabilistic model gives both predictions of plaintext and uncertainty quantification, the latter of which is first introduced to optical cryptanalysis. Predictions of the model are close to real plaintexts, showing the success of the proposed model. Uncertainty quantification reveals the level of reliability of each pixel in the prediction of plaintext without ground truth. Subsequent simulation experiments demonstrate that uncertainty quantification can effectively identify poor-quality predictions to avoid the risk of unreliability from deep learning models.

Multi‐layer random walker image segmentation for overlapped cervical cells using probabilistic deep learning methods

A method for overlapping cell image segmentation is presented with a focus on multi-layer image processing in a three-phase scheme. In the first phase, a convolutional neural network is developed to provide a coarse cell segmentation with multiple output layers to identify cell cytoplasm, locations of cell nuclei, and the background, all as probabilistic image maps for the layer outputs. In the second phase, the probabilistic image maps from the convolutional neural network are used to identify locations of cell nuclei and cell cytoplasm. Then, multi-layer random walker image segmentation is used with cell nuclei as hard initial seeds and the cytoplasm estimates as soft seeds in a diffusion graph-based segmentation of the cells. With rough cell segmentation from both the trained convolutional neural network and the multi-layer random walker graph-based technique, a third phase combines and refines the cell segmentation using the Hungarian algorithm to optimise the assignment of individual pixel locations for the final cell segmentation. We evaluate the proposed method on cervical cell images generated from the International Symposium on Biomedical Imaging 2014 dataset with results that give a Dice similarity coefficient of 97.2% (compared to 93.2% for competitors) when trained on the generated dataset.

Radon potential mapping in Jangsu-gun, South Korea using probabilistic and deep learning algorithms

The adverse health effects associated with the inhalation and ingestion of naturally occurring radon gas produced during the uranium decay chain mean that there is a need to identify high-risk areas. This study detected radon-prone areas using a geographic information system (GIS)-based probabilistic and machine learning methods, including the frequency ratio (FR) model and a convolutional neural network (CNN). Ten influencing factors, namely elevation, slope, the topographic wetness index (TWI), valley depth, fault density, lithology, and the average soil copper (Cu), calcium oxide (Cao), ferric oxide (Fe2O3), and lead (Pb) concentrations, were analyzed. In total, 27 rock samples with high activity concentration index values were divided randomly into training and validation datasets (70:30 ratio) to train the models. Areas were categorized as very high, high, moderate, low, and very low radon areas. According to the models, approximately 40% of the study area was classified as very high or high risk. Finally, the radon potential maps were validated using the area under the receiver operating characteristic curve (AUC) analysis. This showed that the CNN algorithm was superior to the FR method; for the former, AUC values of 0.844 and 0.840 were obtained using the training and validation datasets, respectively. However, both algorithms had high predictive power. Slope, lithology, and TWI were the best predictors of radon-affected areas. These results provide new information regarding the spatial distribution of radon, and could inform the development of new residential areas. Radon screening is important to reduce public exposure to high levels of naturally occurring radiation.

Breaking Down the Computational Barriers to Real‐Time Urban Flood Forecasting

AbstractFlooding impacts are on the rise globally, and concentrated in urban areas. Currently, there are no operational systems to forecast flooding at spatial resolutions that can facilitate emergency preparedness and response actions mitigating flood impacts. We present a framework for real‐time flood modeling and uncertainty quantification that combines the physics of fluid motion with advances in probabilistic methods. The framework overcomes the prohibitive computational demands of high‐fidelity modeling in real‐time by using a probabilistic learning method relying on surrogate models that are trained prior to a flood event. This shifts the overwhelming burden of computation to the trivial problem of data storage, and enables forecasting of both flood hazard and its uncertainty at scales that are vital for time‐critical decision‐making before and during extreme events. The framework has the potential to improve flood prediction and analysis and can be extended to other hazard assessments requiring intense high‐fidelity computations in real‐time.

Non-parametric Bayesian dictionary learning based on Laplace noise

Sparse representation based on over-complete dictionaries is a hot issue in the field of computer vision and machine learning. In probability theory, over-complete dictionary can be learned by non-parametric Bayesian techniques with Beta Process. However, traditional probabilistic dictionary learning method assumes noise follows Gaussian distribution, which can only remove Gaussain noise. In order to remove outlier or complex noise, we propose a dictionary learning method based on non-parametric Bayesian technology by assuming the noise follows Laplacian distribution. Because the non-conjugacy of Laplacian distribution makes the calculation of posteriors of latent variables more complicate, thus we utilize a superposition of an infinite number of Gaussian distributions to substitute for L1 density function. The weights of mixture Gaussian distribution are controlled by an extra hidden variable. Then the Bayesian inference is applied to learn all the key parameters in the proposed probabilistic model, which avoids the processing of parameter setting and fine tuning. In the experiments, we mainly test the performance of different algorithms in removing salt-and-pepper noise and mixture noises. The experimental results show that the PSNRs of our algorithm are higher 2-4 dB at least than other classic algorithms.

Uncertainty Class Activation Map (U-CAM) Using Gradient Certainty Method.

Understanding and explaining deep learning models is an imperative task. Towards this, we propose a method that obtains gradient-based certainty estimates that also provide visual attention maps. Particularly, we solve for visual question answering task. We incorporate modern probabilistic deep learning methods that we further improve by using the gradients for these estimates. These have two-fold benefits: a) improvement in obtaining the certainty estimates that correlate better with misclassified samples and b) improved attention maps that provide state-of-the-art results in terms of correlation with human attention regions. The improved attention maps result in consistent improvement for various methods for visual question answering. Therefore, the proposed technique can be thought of as a tool for obtaining improved certainty estimates and explanations for deep learning models. We provide detailed empirical analysis for the visual question answering task on all standard benchmarks and comparison with state of the art methods.

Seven-Layer Model in Complex Networks Link Prediction: A Survey.

Link prediction is the most basic and essential problem in complex networks. This study analyzes the observed topological, time, attributive, label, weight, directional, and symbolic features and auxiliary information to find the lack of connection and predict the future possible connection. For discussion and analysis of the evolution of the network, the network model is of great significance. In the past two decades, link prediction has attracted extensive attention from experts in various fields, who have published numerous high-level papers, but few combine interdisciplinary characteristics. This survey analyzes and discusses the existing link prediction methods. The idea of stratification is introduced into the classification system of link prediction for the first time and proposes the design idea of a seven-layer model, namely the network, metadata, feature classification, selection input, processing, selection, and output layers. Among them, the processing layer divides link prediction methods into similarity-based, probabilistic, likelihood, supervised learning, semi-supervised learning, unsupervised learning, and reinforcement learning methods. The input features, evaluation metrics, complex analysis, experimental comparisons, relative merits, common dataset and open-source implementations for each link prediction method are then discussed in detail. Through analysis and comparison, we found that the link prediction method based on graph structure features has better prediction performance. Finally, the future development direction of link prediction in complex networks is discussed.