Probabilistic Machine Research Articles

On exploiting nonparametric kernel-based probabilistic machine learning over the large compositional space of high entropy alloys for optimal nanoscale ballistics

The large compositional space of high entropy alloys (HEA) often presents significant challenges in comprehensively deducing the critical influence of atomic composition on their mechanical responses. We propose an efficient nonparametric kernel-based probabilistic computational mapping to obtain the optimal composition of HEAs under ballistic conditions by exploiting the emerging capabilities of machine learning (ML) coupled with molecular-level simulations. Compared to conventional ML models, the present Gaussian approach is a Bayesian paradigm that can have several advantages, including small training datasets concerning computationally intensive simulations and the ability to provide uncertainty measurements of molecular dynamics simulations therein. The data-driven analysis reveals that a lower concentration of Ni with a higher concentration of Al leads to higher dissipation of kinetic energy and lower residual velocity, but with higher penetration depth of the projectile. To deal with such conflicting computationally intensive functional objectives, the ML-based simulation framework is further extended in conjunction with multi-objective genetic algorithm for identifying the critical elemental compositions to enhance kinetic energy dissipation with minimal penetration depth and residual velocity of the projectile simultaneously. The computational framework proposed here is generic in nature, and it can be extended to other HEAs with a range of non-aligned multi-physical property demands.

Simulation-based inference on virtual brain models of disorders

Connectome-based models, also known as virtual brain models (VBMs), have been well established in network neuroscience to investigate pathophysiological causes underlying a large range of brain diseases. The integration of an individual’s brain imaging data in VBMs has improved patient-specific predictivity, although Bayesian estimation of spatially distributed parameters remains challenging even with state-of-the-art Monte Carlo sampling. VBMs imply latent nonlinear state space models driven by noise and network input, necessitating advanced probabilistic machine learning techniques for widely applicable Bayesian estimation. Here we present simulation-based inference on VBMs (SBI-VBMs), and demonstrate that training deep neural networks on both spatio-temporal and functional features allows for accurate estimation of generative parameters in brain disorders. The systematic use of brain stimulation provides an effective remedy for the non-identifiability issue in estimating the degradation limited to smaller subset of connections. By prioritizing model structure over data, we show that the hierarchical structure in SBI-VBMs renders the inference more effective, precise and biologically plausible. This approach could broadly advance precision medicine by enabling fast and reliable prediction of patient-specific brain disorders.

Error mitigation in variational quantum eigensolvers using tailored probabilistic machine learning

Quantum computing technology has the potential to revolutionize the simulation of materials and molecules in the near future. A primary challenge in achieving near-term quantum advantage is effectively mitigating the noise effects inherent in current quantum processing units (QPUs). This challenge is also decisive in the context of quantum-classical hybrid schemes employing variational quantum eigensolvers (VQEs) that have attracted significant interest in recent years. In this paper, we present a method that employs parametric Gaussian process regression (GPR) within an active learning framework to mitigate noise in quantum computations, focusing on VQEs. Our approach, grounded in probabilistic machine learning, exploits a custom prior based on the VQE ansatz to capture the underlying correlations between VQE outputs for different variational parameters, thereby enhancing both accuracy and efficiency. We demonstrate the effectiveness of our method on a two-site Anderson impurity model and a eight-site Heisenberg model, using the IBM open-source quantum computing framework, Qiskit, showcasing substantial improvements in the accuracy of VQE outputs while reducing the number of direct QPU energy evaluations. This paper contributes to the ongoing efforts in quantum-error mitigation and optimization, bringing us a step closer to realizing the potential of quantum computing in quantum matter simulations. Published by the American Physical Society 2024

Uncertainty quantification for probabilistic machine learning in earth observation using conformal prediction

Machine learning is increasingly applied to Earth Observation (EO) data to obtain datasets that contribute towards international accords. However, these datasets contain inherent uncertainty that needs to be quantified reliably to avoid negative consequences. In response to the increased need to report uncertainty, we bring attention to the promise of conformal prediction within the domain of EO. Unlike previous uncertainty quantification methods, conformal prediction offers statistically valid prediction regions while concurrently supporting any machine learning model and data distribution. To support the need for conformal prediction, we reviewed EO datasets and found that only 22.5% of the datasets incorporated a degree of uncertainty information, with unreliable methods prevalent. Current open implementations require moving large amounts of EO data to the algorithms. We introduced Google Earth Engine native modules that bring conformal prediction to the data and compute, facilitating the integration of uncertainty quantification into existing traditional and deep learning modelling workflows. To demonstrate the versatility and scalability of these tools we apply them to valued EO applications spanning local to global extents, regression, and classification tasks. Subsequently, we discuss the opportunities arising from the use of conformal prediction in EO. We anticipate that accessible and easy-to-use tools, such as those provided here, will drive wider adoption of rigorous uncertainty quantification in EO, thereby enhancing the reliability of downstream uses such as operational monitoring and decision-making.

Water quality level estimation using IoT sensors and probabilistic machine learning model

ABSTRACT Drinking water purity analysis is an essential framework that demands several real-world parameters to ensure the quality of water. So far, sensor-based analysis of water quality in specific environments is done concerning certain parameters including the PH level, hardness, TDS, etc. The outcome of such methods analyzes whether the environment provides potable water or not. Potable denotes the purified water that is free from all contaminations. This analysis gives an absolute solution whereas the demand for drinking water is a growing problem where the multiple-level estimations are essential to use the available water resources efficiently. In this article, we used a benchmark water quality assessment dataset for analysis. To perform a level assessment, we computed three major features namely correlation-entropy, dynamic scaling, and estimation levels, and annexed with the earlier feature vector. The assessment of the available data was performed using the statistical machine learning model that ensembles the random forest model and light gradient boost model (GBM). The probability of the ensemble model was done by the Kullback Libeler Divergence model. The proposed probabilistic model has achieved an accuracy of 96.8%, a sensitivity of 94.55%, and a specificity of 98.29%.

Probabilistic impact localization in composites using wavelet scattering transform and multi-output Gaussian process regression

Data-driven machine-learning models offer considerable promise for acoustic source localization. However, many existing models rely on training data that correlates time-of-flight (TOF) measurements with source locations, yet they struggle to handle the complexities arising from nonlinear wave propagation in materials with varying properties. Furthermore, these models overlook the noise and uncertainties inherent in real-world experiments when predicting outputs. This paper aims to bridge a gap in impact localization for such structures, particularly focusing on scenarios involving noisy field measurements. This study proposes a framework based on probabilistic machine learning to identify impact locations, utilizing wavelet scattering transform (WST) and Multi-Output Gaussian Process Regression (moGPR). WST extracts informative features from Lamb waves, capturing relevant signatures for training the probabilistic machine learning model, while moGPR estimates correlated impact location coordinates (x, y) while accounting for inherent uncertainties in the data. To assess the proposed method’s performance in handling measurement uncertainties, an experiment was conducted using a CFRP composite panel instrumented with a sparse array of piezoelectric transducers. The results demonstrate that the probabilistic framework effectively addresses measurement uncertainties, enabling reliable source location estimation with confidence intervals and providing valuable insights for decision-making.

Verifiable Quantum Advantage without Structure

We show the following hold, unconditionally unless otherwise stated, relative to a random oracle: — There are NP search problems solvable by quantum polynomial-time (QPT) machines but not classical probabilistic polynomial-time (PPT) machines. — There exist functions that are one-way, and even collision resistant, against classical adversaries but are easily inverted quantumly. Similar counterexamples exist for digital signatures and CPA-secure public key encryption (the latter requiring the assumption of a classically CPA-secure encryption scheme). Interestingly, the counterexample does not necessarily extend to the case of other cryptographic objects such as PRGs. — There are unconditional publicly verifiable proofs of quantumness with the minimal rounds of interaction: for uniform adversaries, the proofs are non-interactive, whereas for non-uniform adversaries the proofs are two message public coin. — Our results do not appear to contradict the Aaronson-Ambanis conjecture. Assuming this conjecture, there exist publicly verifiable certifiable randomness, again with the minimal rounds of interaction. By replacing the random oracle with a concrete cryptographic hash function such as SHA2, we obtain plausible Minicrypt instantiations of the above results. Previous analogous results all required substantial structure, either in terms of highly structured oracles and/or algebraic assumptions in Cryptomania and beyond.

IGATE analysis improves the interpretability of single-cell immune landscape of influenza infection.

Influenza poses a persistent health burden worldwide. To design equitable vaccines effective across all demographics, it is essential to better understand how host factors such as genetic background and aging affect the single-cell immune landscape of influenza infection. Cytometry by time-of-flight (CyTOF) represents a promising technique in this pursuit, but interpreting its large, high-dimensional data remains difficult. We have developed a new analytical approach, in silico gating annotating training elucidating (iGATE), based on probabilistic support vector machine classification. By rapidly and accurately "gating" tens of millions of cells in silico into user-defined types, iGATE enabled us to track 25 canonical immune cell types in mouse lung over the course of influenza infection. Applying iGATE to study effects of host genetic background, we show that the lower survival of C57BL/6 mice compared with BALB/c was associated with a more rapid accumulation of inflammatory cell types and decreased IL-10 expression. Furthermore, we demonstrate that the most prominent effect of aging is a defective T cell response, reducing survival of aged mice. Finally, iGATE reveals that the 25 canonical immune cell types exhibited differential influenza infection susceptibility and replication permissiveness in vivo, but neither property varied with host genotype or aging. The software is available at https://github.com/UmichWenLab/iGATE.

Probabilistic classification of the severity classes of unhealthy air pollution events.

Air pollution events can be categorized as extreme or non-extreme on the basis of their magnitude of severity. High-risk extreme air pollution events will exert a disastrous effect on the environment. Therefore, public health and policy-making authorities must be able to determine the characteristics of these events. This study proposes a probabilistic machine learning technique for predicting the classification of extreme and non-extreme events on the basis of data features to address the above issue. The use of the naïve Bayes model in the prediction of air pollution classes is proposed to leverage its simplicity as well as high accuracy and efficiency. A case study was conducted on the air pollution index data of Klang, Malaysia, for the period of January 01, 1997, to August 31, 2020. The trained naïve Bayes model achieves high accuracy, sensitivity, and specificity on the training and test datasets. Therefore, the naïve Bayes model can be easily applied in air pollution analysis while providing a promising solution for the accurate and efficient prediction of extreme or non-extreme air pollution events. The findings of this study provide reliable information to public authorities for monitoring and managing sustainable air quality over time.

Reverse engineering morphogenesis through Bayesian optimization of physics-based models

Morphogenetic programs coordinate cell signaling and mechanical interactions to shape organs. In systems and synthetic biology, a key challenge is determining optimal cellular interactions for predicting organ shape, size, and function. Physics-based models defining the subcellular force distribution facilitate this, but it is challenging to calibrate parameters in these models from data. To solve this inverse problem, we created a Bayesian optimization framework to determine the optimal cellular force distribution such that the predicted organ shapes match the experimentally observed organ shapes. This integrative framework employs Gaussian Process Regression, a non-parametric kernel-based probabilistic machine learning modeling paradigm, to learn the mapping functions relating to the morphogenetic programs that maintain the final organ shape. We calibrated and tested the method on Drosophila wing imaginal discs to study mechanisms that regulate epithelial processes ranging from development to cancer. The parameter estimation framework successfully infers the underlying changes in core parameters needed to match simulation data with imaging data of wing discs perturbed with collagenase. The computational pipeline identifies distinct parameter sets mimicking wild-type shapes. It enables a global sensitivity analysis to support the regulation of actomyosin contractility and basal ECM stiffness to generate and maintain the curved shape of the wing imaginal disc. The optimization framework, combined with experimental imaging, identified that Piezo, a mechanosensitive ion channel, impacts fold formation by regulating the apical-basal balance of actomyosin contractility and elasticity of ECM. This workflow is extensible toward reverse-engineering morphogenesis across organ systems and for real-time control of complex multicellular systems.

Feedback stabilization of probabilistic finite state machines based on deep Q-network.

As an important mathematical model, the finite state machine (FSM) has been used in many fields, such as manufacturing system, health care, and so on. This paper analyzes the current development status of FSMs. It is pointed out that the traditional methods are often inconvenient for analysis and design, or encounter high computational complexity problems when studying FSMs. The deep Q-network (DQN) technique, which is a model-free optimization method, is introduced to solve the stabilization problem of probabilistic finite state machines (PFSMs). In order to better understand the technique, some preliminaries, including Markov decision process, ϵ-greedy strategy, DQN, and so on, are recalled. First, a necessary and sufficient stabilizability condition for PFSMs is derived. Next, the feedback stabilization problem of PFSMs is transformed into an optimization problem. Finally, by using the stabilizability condition and deep Q-network, an algorithm for solving the optimization problem (equivalently, computing a state feedback stabilizer) is provided. Compared with the traditional Q learning, DQN avoids the limited capacity problem. So our method can deal with high-dimensional complex systems efficiently. The effectiveness of our method is further demonstrated through an illustrative example.

Efficient analysis of composites manufacturing using multi-fidelity simulation and probabilistic machine learning

This paper introduces an innovative approach for the efficient analysis of composites manufacturing processes and phenomena. The method combines low- and high-fidelity simulation schemes with limited amounts of experimental data to train surrogate machine learning (ML) models. Guided by a novel approach, Spatially Weighted Gaussian Process Regression (SWGPR), a predictive model is efficiently constructed and calibrated by assigning datapoint-dependent noise levels to simulation points, establishing a multi-scale data-driven uncertainty structure. This study demonstrates the effectiveness of the method in accurately predicting process-induced deformations (PIDs) for L-shaped cross-ply laminates using minimal experimental efforts. The presented method aims to provide a cost-effective and broadly applicable framework for understanding and improving the design, development, and manufacturing of composites.

Data-driven prediction of natural period for existing RC high-rise buildings using probabilistic machine learning methods

The natural period of existing reinforced concrete (RC) high-rise buildings is a fundamental structural parameter for dynamic analysis and condition assessment, while the estimation of such structural characteristic is an important task for life-cycle maintenance. A commonly adopted approach for predicting natural periods involves the use of design-oriented empirical formulas. However, as shown by real-world observations in both this study and others, these formulas with limited building features may not fully capture the complexity and uncertainty of the natural period for existing buildings. Therefore, it is desired to develop an alternative method that is both feature-based and data-driven, to predict natural periods accurately. This study proposes a feature-based probabilistic machine learning method to determine the natural periods and quantify the uncertainty intervals of the predictions for existing RC high-rise buildings. Specifically, the proposed method comprises a random-forest method using quantile regression. Through in situ measurements and an extensive literature review, a database comprising over 500 actual records was constructed, including the building features and their dynamic properties. The results show that the proposed method achieves a coefficient of determination of 0.96 and an uncertainty interval confined within 20 % for the test set. These findings indicate that the proposed method may be both more accurate and reliable. Furthermore, an in-depth investigation into the model interpretability was conducted to reveal the relative importance of different building features. The analyses indicate that the building height emerges as the most crucial feature, while the building width and its lateral resistant system may also demonstrate noticeable influence on predicting natural periods.

Complexity-calibrated benchmarks for machine learning reveal when prediction algorithms succeed and mislead

Recurrent neural networks are used to forecast time series in finance, climate, language, and from many other domains. Reservoir computers are a particularly easily trainable form of recurrent neural network. Recently, a “next-generation” reservoir computer was introduced in which the memory trace involves only a finite number of previous symbols. We explore the inherent limitations of finite-past memory traces in this intriguing proposal. A lower bound from Fano’s inequality shows that, on highly non-Markovian processes generated by large probabilistic state machines, next-generation reservoir computers with reasonably long memory traces have an error probability that is at least ∼60%\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\sim 60\\%$$\\end{document} higher than the minimal attainable error probability in predicting the next observation. More generally, it appears that popular recurrent neural networks fall far short of optimally predicting such complex processes. These results highlight the need for a new generation of optimized recurrent neural network architectures. Alongside this finding, we present concentration-of-measure results for randomly-generated but complex processes. One conclusion is that large probabilistic state machines—specifically, large ϵ\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\epsilon$$\\end{document}-machines—are key to generating challenging and structurally-unbiased stimuli for ground-truthing recurrent neural network architectures.

Evolution of Natural Language Processing: A Review

Over the years, Natural Language Processing (NLP) has evolved dramatically, moving from early rule based systems to the current era dominated by advanced deep learning models. An overview of the significant turning points and patterns that have influenced the development of NLP is given in this study. In the early days of Natural Language Processing (NLP), rule based methods were the main focus. Linguists would manually create rules to analyze and comprehend human language. Although these systems were somewhat successful, they were unable to handle the complexity and unpredictability of spoken language. A major change was brought about by the introduction of probabilistic models and machine learning techniques with the emergence of statistical approaches. The development of methods like n gram models and hidden Markov models during this time allowed computers to handle linguistic patterns. Large scale linguistic resources like word embeddings and annotated corpora started to appear, which further accelerated the development of NLP. Machine learning algorithms have led to notable advancements in tasks such as machine translation, named entity recognition, and part of speech tagging. NLP has seen a revolution in recent years thanks to deep learning, which uses neural networks to learn intricate language representations. Sequential dependencies in language can now be better understood because of models like Long Short Term Memory Networks (LSTMs) and Recurrent Neural Networks (RNNs). The addition of attention mechanisms, as demonstrated by Transformer and other models, improves the model's ability to manage long range dependencies and perform better on a variety of NLP tasks. In the future, NLP will develop in ways that go beyond performance measurements, exploring interpretability, ethical issues, and the incorporation of multimodal data. As the area develops, it becomes increasingly important to eliminate biases, ensure ethical AI deployment, and improve user centric experiences. This introduction lays the groundwork for an in depth examination of the development of NLP, highlighting significant turning points, difficulties, and potential future directions in this vibrant and quickly developing subject.

Application of Computer-Aided Technology in Teaching Spoken English

Computer-aided technology has revolutionized the teaching of spoken English by offering innovative tools and resources that enhance the learning experience. Through interactive software and multimedia platforms, learners can engage in immersive language exercises, receive real-time feedback, and access personalized instruction tailored to their proficiency levels. This paper introduces a groundbreaking approach for integrating computer-aided technology into the teaching of spoken English, utilizing Automated Probabilistic Markov Chain Machine Learning (APMC-ML). Recognizing the significance of effective communication skills in the modern world, particularly in the realm of English language proficiency, this research endeavors to enhance the pedagogical process through innovative technological means. The methodology involves the development and implementation of an interactive computer-aided system that leverages APMC-ML algorithms to facilitate language learning. This system encompasses various modules designed to cater to different aspects of spoken English acquisition, including pronunciation, vocabulary, fluency, and conversational skills. Through a user-friendly interface, learners are provided with personalized learning experiences tailored to their individual proficiency levels and learning preferences. The application of APMC-ML in teaching spoken English represents a paradigm shift in language education, offering a data-driven approach that complements traditional teaching methods. By harnessing the power of machine learning and probabilistic modeling, educators can optimize the learning experience, expedite skill acquisition, and promote greater language proficiency among learners.

Assessment of the effect of processing parameters on peel failure of laser-assisted automated fiber placed thermoplastic composites

In this study, thermoplastic composites are fabricated using laser-assisted automated fiber placement, focusing on the influence of compaction force, roller head speed, and temperature. Employing wedge peel testing, digital and electron microscopy, and probabilistic machine learning, the research examines how these parameters affect mechanical performance and failure. Key observations include increased bonding area and peel force with higher processing temperatures, while greater compaction force introduces more variability in bonding and peel strength. Roller head speed shows minimal impact. The study effectively links processing parameters with wedge peel failure, illustrating the complex interactions in composite material behavior.

Security and Privacy Column

Cryptographic protocols are really hard to design: they mix difficulties stemming from distributed systems, rely on cryptographic primitives and need to provide complex guarantees in the presence of an arbitrary attacker running in parallel. Rigorous security proofs are basically the only hope to avoid design flaws as witnessed by numerous examples. This has been recognized since the early 80's and two main approaches have been actively developed since. The computational model is rooted in complexity theory and relies on a reduction of the security of the protocol to a security assumption of a cryptographic primitive which itself may rely on the hardness of a computational problem. Attackers are any probabilistic polynomial time Turing Machines (PPTMs). A security proof in this model provides strong guarantees. However, getting proofs in such a detailed model correct is challenging. The symbolic model takes a different approach: messages are modeled as first-order logic terms and cryptographic primitives such as encryption schemes are idealized. Typically, only the intended functionality of an encryption is modeled by an equation such as dec(enc(m,k),k) = m to state that decryption and encryption cancel out when the same key k is used. Crucially, only the properties that are explicitly stated hold and the adversary actions are restricted to those. At the cost of being more abstract this approach is amenable to automation and tools nowadays succeed in proving industrial protocols, such as TLS or Signal.

Accurate and robust predictions of pool boiling heat transfer with micro-structured surfaces using probabilistic machine learning models

The accurate and reliable prediction of enhanced heat transfer performance of micro-structured surfaces is crucial to optimally design and operate pool boiling systems. However, the existing empirical models predict the enhanced pool boiling heat transfer with very large errors up to ±81 % even using the experimental data from the same study, mainly due to the complex nature of the pool boiling processes. More importantly, the existing models predict only limited coolant types, surface geometries, and operating conditions. To overcome these challenges, this study examines three deterministic and two probabilistic machine learning (ML) models for accurate and reliable enhanced pool boiling prediction, while using carefully selected key six dimensionless numbers. The models were trained and tested using 1,241 data from 20 experimental studies with 80/20 % of train/test ratio, and the pre-trained models were tested for additional 519 data from 6 studies to evaluate the models reliability. The predicted mean absolute percentage error (MAPE) shows that Bayesian, deep, and 1-D convolutional neural network (1D-CNN) models outperform the random forest and natural gradient (NG) boost models due to their extended hidden layers. The machine learning models improve the MAPE by up to 30 % compared to the existing correlations. Furthermore, a parameter sensitivity analysis is performed using explainable artificial intelligence showing that the boiling Reynolds number is the most critical parameter followed by the kinetic Reynolds and Bond numbers. The probabilistic ML models can also provide the uncertainties to improve prediction reliability compared to the deterministic ones.

Predicting undergraduate student evaluations of teaching using probabilistic machine learning: The importance of motivational climate

The purpose of this study was to understand the complex interactions within a course among motivational climate variables and student evaluations of teaching (SETs) by developing online simulators using probabilistic machine learning. We used data from 2938 undergraduate students in 30 classes to create online simulators based on Bayesian Belief Networks. We created bubble charts, line graphs, and radar charts to show the relationships between the study variables. Findings showed that (a) the motivational climate variables—as measured by the MUSIC Model of Motivation variables—are the largest predictors of SETs, (b) student interest (in the coursework and instructional methods) is the overall largest predictor of SETs, (c) the relationships between the motivational climate variables and SETS are nonlinear, (d) the ease of the course and demographic variables are only weakly associated with SETs, and (e) the largest predictors of instructor and course rating are similar, but somewhat different.