Automatic Relevance Determination Research Articles

System fragility analysis of highway bridge using multi-output Gaussian process regression surrogate model

This study presents a multi-output Gaussian process regression (GPR) surrogate model for seismic-fragility analysis of bridge structural systems. Multi-output GPR can model the correlations among multiple outputs, accuracy and stability are achieved with fewer training data, which reduces the computational cost of fragility analysis. Furthermore, the explainability of the constructed surrogate model is implemented by adopting an automatic relevance-determination (ARD) kernel in the GPR. The estimated hyperparameters can provide the contribution of the uncertainty of each input parameter to the outputs. The fragility analysis using the multi-output GPR surrogate model was verified by applying it to a seismic isolation highway bridge with multiple spans and a curved geometry. The effectiveness of the multi-output GPR was demonstrated by the construction of an accurate and stable surrogate model with 46 inputs and 28 outputs. The relative contributions of the uncertainties to the structural properties and input earthquake loads could also be understood. The fragility curves, at both the component and system levels, were appropriately obtained using a sufficient number of samples in a Monte Carlo calculation. Furthermore, the failure modes were evaluated, identifying which structural components contributed to the system failure. This enabled discussions on structural system failures from the viewpoint of the structural dynamic characteristics of the bridge and earthquake-load properties.

Evaluation of a hyperspectral image pipeline toward building a generalisation capable crop dry matter content prediction model

Hyperspectral imaging has proven to be a reliable technique for estimating dry matter, a common variable when considering the quality of the fresh produce. However, developing models capable of generalising across different crops is challenging. In this study, several pipelines were explored towards achieving a robust and accurate generic regression model were evaluated and the development of Automatic Relevance Determination (ARD) and Partial Least Squares (PLS) algorithms for fruit and vegetable dry matter estimation. The models were built using a VIS-NIR dataset that includes both fruit and vegetables, namely, apples, broccoli and leek (n = 779). The PLS regression model obtained Root Mean Square on Prediction (RMSEP) = 0.0137, outperforming ARD regression (RMSEP = 0.0140) on a 10x5-fold cross-validation protocol. The evaluated preprocessing techniques affect the two regression algorithms differently, with the best results achieved when the pipeline was used without feature extraction. Overall, the pipeline using either ARD or PLS regression shows strong performance and generalisation for Visible-Near Infrared (VIS-NIR)-based dry matter estimation across diverse fruits and vegetables.

A Gaussian process embedded feature selection method based on automatic relevance determination

In Gaussian Process, feature importance is inversely proportional to the corresponding length scale when applying the Automatic Relevance Determination (ARD) structured kernel function. Features can be selected by ranking them according to their importance. Among the ARD-based feature selection methods, no uniform score exists for quantifying the output variation explained by feature subsets. This study proposes two feature selection approaches using two cumulative feature importance scores, one titled derivative decomposition ratio and the other normalized sensitivity, to determine the optimal feature subset. The performance of the approaches is assessed to test if irrelevant features are accurately identified and if the feature rankings are correct. The approaches are applied to identify relevant dimensionless inputs for a hybrid model estimating liquid entrainment fraction in two-phase flow. The results reveal that the proposed methods can identify the optimal feature subset for the hybrid model without significantly worsening its Root Mean Squared Error.

Image classification based on deep learning with automatic relevance determination and structured Bayesian pruning

Image classification based on deep learning with automatic relevance determination and structured Bayesian pruning

Analysis of vibrational modes and weak interactions in terahertz spectra of γ-aminobutyric acid and its isomer L-2-aminobutyric acid

Aminobutyric acid, a non-proteinogenic amino acid, is present in the human body and influences physiological functions, exhibiting promising pharmaceutical prospects. γ-Aminobutyric Acid (GABA) and its isomer, L-2-Aminobutyric Acid (L-AABA), exhibit unique characteristic absorption peaks in the 0.6 to 2.5 THz range. To elucidate the mechanisms behind these absorption peaks, this study employed Density Functional Theory (DFT) for theoretical simulations of the crystal cell models of L-AABA and GABA, achieving terahertz spectra consistent with experimental results. Additionally, the study utilized Vibrational Mode Automatic Relevance Determination (VMARD) and Interaction Region Indicator (IRI) methodologies to interpret the vibrational modes corresponding to different absorption peaks and to detail the locations and types of intermolecular interactions within the crystals. Research indicates that the combination of terahertz time-domain spectroscopy (THz-TDS) and theoretical calculations can serve as an effective method for identifying isomers of Aminobutyric Acid and detecting intermolecular weak interactions such as hydrogen bonds within these crystals.

Pointwise profile monitoring considering covariates based on Gaussian process

In the intelligent manufacturing process, sensors are increasingly becoming an important means of measurement and data acquisition. Profile data from sensors are generally huge, irregular, and correlated. Meanwhile, profiles are usually affected by many external factors, which are referred to as covariates in this paper. Existing statistical process control methods are mainly based on statistical models, which have many limitations in dealing with such complex profile data. Motivated by the real life problem of industrial busbar temperature monitoring, this paper proposes a nonparametric pointwise profile monitoring scheme for unbalanced data considering time varying covariates. The developed Gaussian process-based model provides flexibility for predicting autocorrelated profiles. To incorporate time varying covariates, the Gaussian process model kernel is modified by an automatic relevance determination structure. The control chart is then constructed based on the difference between the observed and predicted profiles. Instantaneous anomalies within a profile can be detected by pointwise monitoring without waiting for the entire profile data collection. The effectiveness and applicability of the proposed monitoring scheme are validated by both simulations and a real case of busbar temperature monitoring in the workshop of an automotive manufacturer.

Interpretable Recurrent Variational State-Space Model for Fault Detection of Complex Systems Based on Multisensory Signals

It is necessary to develop a health monitoring system (HMS) for complex systems to improve safety and reliability and prevent potential failures. Time-series signals are collected from multiple sensors installed on the equipment that can reflect the health condition of them. In this study, a novel interpretable recurrent variational state-space model (IRVSSM) is proposed for time-series modeling and anomaly detection. To be specific, the deterministic hidden state of a recursive neural network is used to capture the latent structure of sensor data, while the stochastic latent variables of a nonlinear deep state-space model capture the diversity of sensor data. Temporal dependencies are modeled through a nonlinear transition matrix; an automatic relevance determination network is introduced to selectively emphasize important sensor data. Experimental results demonstrate that the proposed algorithm effectively captures vital information within the sensor data and provides accurate and reliable fault diagnosis during the steady-state phase of liquid rocket engine operation.

State-of-Health prediction of lithium-ion batteries based on a low dimensional Gaussian Process Regression

An accurate determination of the condition of a battery is a key challenge in operation. As the performance of lithium-ion batteries is degrading over time, an accurate prediction of the State-of-Health would improve the overall efficiency and safety. This paper presents a prediction method for the State-of-Health based on a Gaussian Process Regression with an automatic relevance determination kernel in a single model for three different types of battery cells. After reducing the dimension of the problem and a sensitivity analysis of the features, the model is trained, validated, and further tested on unseen data. A minimum test error is obtained with a mean absolute error of 1.33%. Combined with the low uncertainty of the prediction results, this shows the applicability and the great potential of forecasting the condition of a battery using data-driven methods.

Pyaging: a Python-based compendium of GPU-optimized aging clocks.

Aging is intricately linked to diseases and mortality. It is reflected in molecular changes across various tissues which can be leveraged for the development of biomarkers of aging using machine learning models, known as aging clocks. Despite advancements in the field, a significant challenge remains: the lack of robust, Python-based software tools for integrating and comparing these diverse models. This gap highlights the need for comprehensive solutions that can handle the complexity and variety of data in aging research. To address this gap, I introduce pyaging, a comprehensive open-source Python package designed to facilitate aging research. pyaging harmonizes dozens of aging clocks, covering a range of molecular data types such as DNA methylation, transcriptomics, histone mark ChIP-Seq, and ATAC-Seq. The package is not limited to traditional model types; it features a diverse array, from linear and principal component models to neural networks and automatic relevance determination models. Thanks to a PyTorch-based backend that enables GPU acceleration, pyaging is capable of rapid inference, even when dealing with large datasets and complex models. In addition, the package's support for multi-species analysis extends its utility across various organisms, including humans, various mammals, and Caenorhabditis elegans. pyaging is accessible on GitHub, at https://github.com/rsinghlab/pyaging, and the distribution is available on PyPi, at https://pypi.org/project/pyaging/. The software is also archived on Zenodo, at https://zenodo.org/doi/10.5281/zenodo.10335011.

Operational Status Evaluation of Smart Electricity Meters Using Gaussian Process Regression With Optimized-ARD Kernel

Operational Status Evaluation (OSE) is essential for the health management of Smart Electricity Meters (SEM). This paper develops a machine learning-enabled OSE method for SEM. Specifically, the Gaussian Process Regression (GPR) is developed for modeling analysis, where an Optimized Automatic Relevance Determination (OARD) kernel is first used. Though the conventional ARD kernel structure can capture the potential mapping relationship between running time, temperature, humidity, and Measurement Error (ME), it cannot extract the influence degree of different stresses on the ME. To address this problem, an OARD kernel structure is proposed to identify the contribution of each part in ARD structure to the target data. Furthermore, the quartiles line instead of 95% confidence interval is exploited to enhance the performance of GPR for long-term prediction. Combining the two above improvements, a novel OSE model is established for the future-oriented long-term operation of the SEM. It is the first-known data-driven application that utilizes the GPR with OARD kernel to perform OSE for SEM. Actual SEM datasets collected from both dry and hot region are used for model validation and prediction. The results demonstrate that the proposed GPR model with OARD Matern52 kernel outperforms other conventional kernel approaches in the aspect of interpretability. More importantly, the operational status of the SEM in future can be assessed via the proposed OSE framework.

Probabilistic seismic performance estimation through surrogate model and unbiased multi‐fidelity Monte Carlo predictor

AbstractThis study introduces an approach for probabilistic seismic performance estimation, which focuses on the probability of intensity measures exceeding a specified value based on engineering demand parameters. Conventional methods face challenges owing to the increase in computational costs associated with the uncertainties in earthquake scenarios. To address this, we use high‐fidelity (HF) and low‐fidelity (LF) model data to develop a multilevel hierarchy of surrogate models, which improves the simulation‐based probabilistic estimation. However, designing a reliable LF model and ensuring the accuracy of the surrogate model hierarchy remains challenging. Herein, we present a multi‐fidelity Monte Carlo (MFMC) predictor combined with a conventional surrogate model to improve probabilistic seismic performance estimation, thereby leveraging LF model efficiency and HF data accuracy for unbiased results. We addressed the challenge of constructing a suitable LF model by using a surrogate model trained from limited HF data. The MFMC predictor improves the accuracy of probabilistic analysis than the surrogate models trained from limited HF data. Further, the automatic relevance determination method is introduced to select the most appropriate inputs for the surrogate model. A case study featuring a special moment‐resistant frame, subject to uncertainties from both ground motion and structural properties, illustrates the efficiency of the method. A LF model is developed using the Kriging method, followed by the construction of the MFMC predictor for probabilistic analysis, thereby leveraging both limited HF and numerous LF data. Comparing the exceedance probability curves obtained from the MFMC predictor with those from direct Monte Carlo simulations and conventional Kriging showed that our proposed method offers a promising tool for probabilistic seismic performance estimation under uncertainty.

Hydrogen bonding and Hirshfeld surface analysis of gallic acid and its monohydrate based on terahertz spectroscopy

The terahertz (THz) spectroscopy of gallic acid (GA) and gallic acid monohydrate (GAMD) in the frequency range of 0.2–3.0 THz were investigated using terahertz time-domain spectroscopy (THz-TDS) system. The THz absorption spectrum of GAMD exhibited a red-shift compared to that of GA. To resolve the absorption spectra of the two substances, density functional theory (DFT) was employed to calculate and simulate the theoretical spectra of the two substances in this frequency range, while the vibration mode automatic relevance determination (VMARD) method combined with GaussView and VMD software was utilized to determine the origin of each absorption peak. Additionally, Hirshfeld surface analysis and the Atoms-in-Molecules (AIM) topological analysis were conducted to explore the intermolecular interactions between GA and GAMD. The results revealed that the characteristic absorption peaks of both GA and GAMD are attributed to overall atomic and molecular motions, whereas those of GAMD are accompanied by water molecule translation. Furthermore, H…O/O…H hydrogen bonding was identified as the main contributor to the interaction between GA and GAMD. The addition of water molecules led to differentiation in the position and strength of intermolecular interactions; specifically, hydrogen bonding in GA primarily involved the carboxyl group with the hydroxyl group, while hydrogen bonding in GAMD occurred between the water molecule and the hydroxyl group of the central molecule.

Characteristic Analysis and Short-Impending Prediction of Aircraft Bumpiness over Airport Approach Areas and Flight Routes

Based on the Quick Access Recorder (QAR) data covering over 9000 routes in China, the monthly and intra-day distribution characteristics of aircraft bumpiness at different levels were analyzed, and the relationships between the eddy dissipation rate (EDR) and other aircraft flight status elements during bumpiness occurrence were also analyzed. Afterward, aircraft bumpiness routes were constructed using 19 machine learning models. The analyses show that (1) aircraft bumpiness was mainly concentrated between 0:00 a.m. and 17:00 p.m. Severe aircraft bumpiness occurred more frequently in the early morning in January, especially between 5:00 a.m. and 6:00 a.m., and moderate bumpiness always occurred from 3:00 a.m. to 11:00 a.m. (2) The relationship between the left and right attack angles and aircraft bumpiness on the routes was more symmetrical, with a center at 0 degrees, unlike in the approach area where the hotspots were mainly concentrated in the range of −5 to 0 degrees. In the approach area, the larger the Mach number, the more severe the bumpiness. (3) The performances of the Automatic Relevance Determination Regression (ARD), Partial Least Squares Regression (PLS), Elastic-Net Regression (ENR), Classification and Regression Tree (CART), Passive Aggressive Regression (PAR), Random Forest (RF), Stochastic Gradient Descent Regression (SGD), and Tweedie Regression (TWD) based models were relatively good, while the performances of the Huber Regression (HUB), Least Angle Regression (LAR), Polynomial Regression (PLN), and Ridge Regressor (RR) based models were very poor. The aircraft bumpiness prediction models performed best over the approach area of ZBDT (airport in Datong), ZULS (airport in Lhasa), ZPPP (airport in Kunming), and ZLQY (airport in Qingyang). The model performed best in predicting the ZLLL-ZBDT air route (flight routes for Lanzhou to Datong) with different prediction times.

Bubble transport during SGTR accident in lead-cooled fast reactor: A machine learning

Steam generator tube rupture (SGTR) is one of the safety issues for pool-type lead-cooled fast reactors (LFR). Accurately quantifying and predicting the bubble transport result is essential for evaluating the accident. This paper addresses tracking the bubble motion using an Eulerian-Lagrangian method in CFD based on the Europe Lead cooling System (ELSY) primary system model at 1/8 centrosymmetric structure. The steady and transient bubble distributions in the system under different leakage heights are obtained. Furthermore, the simulation results are predicted by machine learning, where Gaussian Process Regression (GPR) is employed for both steady conditions and transient conditions. The data-driven GPR model is established for the bubble transport prediction, which can capture complex non-linear relationships between input variables (i.e. bubble diameter, SG leakage height, final position, system contaminate) and output (reached bubble percentage) responses directly from data. For steady conditions, the prediction results by the kernel function of Automatic Relevance Determination (ARD) Rational Quadratic show the best accuracy in predicting the percentages of bubbles reaching the core, top of the steam generator, and staying in the system, with a total Root Mean Square Error (RMSE) of 3.22%. Four typical transient cases of bubble accumulation in the core are selected for prediction. All the cases are well predicted by the kernel function of ARD Matern 3/2 with low mean averaged error.

Generalized Variable Selection Algorithms for Gaussian Process Models by LASSO-Like Penalty

With the rapid development of modern technology, massive amounts of data with complex pattern are generated. Gaussian process models that can easily fit the nonlinearity in data become more and more popular nowadays. It is often the case that in some data only a few features are important or active. However, unlike classical linear models, it is challenging to identify active variables in Gaussian process models. One of the most commonly used methods for variable selection in Gaussian process models is automatic relevance determination, which is known to be open-ended. There is no rule of thumb to determine the threshold for dropping features, which makes the variable selection in Gaussian process models ambiguous. In this work, we propose two variable selection algorithms for Gaussian process models, which use the artificial nuisance columns as baseline for identifying the active features. Moreover, the proposed methods work for both regression and classification problems. The algorithms are demonstrated using comprehensive simulation experiments and an application to multi-subject electroencephalography data that studies alcoholic levels of experimental subjects. Supplementary materials for this article are available online.

Prediction of algal bloom using a combination of sparse modeling and a machine learning algorithm: Automatic relevance determination and support vector machine

Algae can produce odor substances and toxins that make the smell and taste of water unpleasant and impair the quality of human and aquatic life. Appropriate countermeasures can be implemented in advance in water purification processes to prevent algal disorders if the occurrence of algal blooms is accurately predicted. Several models have been developed to predict algal blooms. However, a comprehensive model that can be universally applied under various conditions is lacking. In this study, automatic relevance determination, a sparse modeling algorithm, and support vector machine were combined to construct prediction models for algal blooms in four Japanese dam reservoirs to predict their occurrence over 7 days. Automatic relevance determination was applied to a dataset consisting of monthly water quality data and daily hydraulic and meteorological data to identify variables relevant to the concentrations of Microcystis spp. and Dolichospermum spp., which are bloom-forming cyanobacteria and are dominant in freshwater ecosystems. A dataset of selected variables was used to train and validate the support vector machine models. The results of variable selection by automatic relevance determination revealed that the average concentration of total nitrogen in the past year and the average maximum temperature in the past 7 days may have an association with the algal concentration. Support vector machine models resulted in 92.3% accuracy and 86.4% precision for Microcystis spp. and 71.4% accuracy and 77.5% precision for Dolichospermum spp. on average in binary classification. The competitive relationship between Microcystis spp. and Dolichospermum spp., which differs according to the nutrient level and temperature, probably affects the prediction performance of the models. Our study suggests that the combination of sparse modeling and machine learning is applicable to the construction of a prediction model for site-specific algal bloom events in dam reservoirs.

Sparse Bayesian machine learning for the interpretable identification of nonlinear structural dynamics: Towards the experimental data-driven discovery of a quasi zero stiffness device

Data-driven discovery of governing laws for complex nonlinear structural dynamic systems remains a challenging issue of paramount importance. This work addresses the above issue by leveraging the available noisy data and integrating sparse Bayesian machine learning (ML) techniques to discover the governing equations. The problem of discovery is re-cast as the automatic relevance determination of models (model selection) from the library of potential candidate basis terms and their coefficients are determined (parameter identification) using sparse Bayesian linear regression. Two sparsity promoting ML algorithms based on relevance vector machines have been employed. Both these approaches use Bayesian statistics and quantify the uncertainty associated with the model predictions. Results from four representative numerical examples of nonlinear structural dynamics illustrate excellent performance of both proposed approaches. The results have been validated with the true governing equations and time response data. Comparison has also been made with a recent and popular sparse discovery approach. Finally, the proposed framework is applied to real datasets that were generated from an in-house designed experimental setup of a quasi zero stiffness device and good performance has been observed.

Development of machine learning-based solubility models for estimation of Hydrogen solubility in oil: Models assessment and validation

This research was done to build computational models for estimating solubility of hydrogen (S) in a given system based on the inputs of temperature (T) and pressure (P). In fact, multiples models were built considering double inputs and single output. To achieve this, three different regression algorithms were trained and tested on a comprehensive dataset using the optimization technique of Grey Wolf Optimization (GWO). The three models used were Elastic Net, Support Vector Regression (SVR), and Automatic Relevance Determination (ARD) Regression. R2 score, root mean square error (RMSE), and mean absolute error were just few of the measures used to assess the models' overall efficacy (MAE). The Elastic Net model achieved the highest R2 score of 0.965 with an RMSE of 8.84×10−2 and an MAE of 7.79×10−2. The SVR technique also performed well, indicated R2 of 0.949, an RMSE of 9.81×10−2, and an MAE of 7.79×10−2. The ARD Regression model revealed an R2 of 0.905 with an RMSE of 1.53×10−1 and an MAE of 1.32×10−1. These findings highlight the potential of machine learning models and metaheuristic algorithms such as GWO in accurately estimating gas solubility in oil, which can have significant implications for various industrial applications.

Sparse Bayesian Learning With Weakly Informative Hyperprior and Extended Predictive Information Criterion.

This article considers the regression problem with sparse Bayesian learning (SBL) when the number of weights P is larger than the data size N, i.e., P》 N. The situation induces overfitting and makes regression tasks, such as prediction and basis selection, challenging. We show a strategy to address this problem. Our strategy consists of two steps. The first is to apply an inverse gamma hyperprior with a shape parameter close to zero over the noise precision of automatic relevance determination (ARD) prior. This hyperprior is associated with the concept of a weakly informative prior in terms of enhancing sparsity. The model sparsity can be controlled by adjusting a scale parameter of inverse gamma hyperprior, leading to the prevention of overfitting. The second is to select an optimal scale parameter. We develop an extended predictive information criterion (EPIC) for optimal selection. We investigate the strategy through relevance vector machine (RVM) with a multiple-kernel scheme dealing with highly nonlinear data, including smooth and less smooth regions. This setting is one form of the regression task with SBL in the P》 N situation. As an empirical evaluation, regression analyses on four artificial datasets and eight real datasets are performed. We see that the overfitting is prevented, while predictive performance may be not drastically superior to comparative methods. Our methods allow us to select a small number of nonzero weights while keeping the model sparse. Thus, the methods are expected to be useful for basis and variable selection.

Development of machine learning-based models for describing processes in a continuous solar-driven biomass gasifier

The synergy of two renewable and efficient sources in producing clean fuels, i.e., solar energy and biomass, can result in high efficiency. In this regard, developing syngas production systems based on solar biomass gasification has attracted much attention. However, experimental setups on solar-driven gasifier processes are costly and time-intensive. In such a situation, an accurate and low-cost alternative is to develop data-driven machine learning (ML) models to predict the processes involved in solar-driven biomass gasifiers. In the present study, several ML models, including random forest (RF), RANdom SAmple Consensus (RANSAC), stochastic gradient descent (SGD), automatic relevance determination (ARD) regression, and elastic net linear (ENL) regression, were developed to accurately predict the various processing of a continuous solar-driven biomass gasifier, including CO production rate and H2 production rate (H2), carbon feeding rate, solar power input, thermochemical reactor efficiency, solar-to-fuel energy conversion efficiency, solar energy input, and carbon consumption rate. Using efficient ML methods, the eight formulas and the eight models for H2, CO production rate, carbon feeding rate, carbon consumption rate, solar energy input, solar power input, thermochemical reactor efficiency, and solar-to-fuel energy conversion efficiency are made in this study. Using the linear form can be reached the best R-Squared (R2) values for all formulas and the model, and the best R2 values are between 0.998 and 0.999 for the formulas by the elastic net and the ARD regression, and also the best R2 values are between 0.998 and 0.999 for the models by the RF and the RANSAC regressor. The R2 values for H2, CO production rate, carbon consumption rate, solar energy input, solar power input, thermochemical reactor efficiency, and solar-to-fuel energy conversion efficiency for formulas, respectively, are 0.998, 0.998, 0.999, 0.999, 0.999, 0.996, and 0.998 by the elastic net. For temperature of the carbon feeding rate, this value is 0.999 by the ARD regressor.