Prior Distribution Of Parameters Research Articles

A variational Bayesian based robust filter for unknown measurement bias and inaccurate noise statistics

In many practical fields, the unknown time-varying measurement biases (additive and multiplicative bias) and heavy-tailed measurement noise caused by some unpredictable anomalous behaviors may degrade the performance of conventional Kalman filter seriously. To solve the state estimation problem of systems with time-varying measurement biases and heavy-tailed measurement noise, this paper proposes a new variational Bayesian (VB) based robust filter. Firstly, the non-Gaussian measurement likelihood probability density function (ML-PDF) with multiplicative and additive measurement bias is built. Then, the conjugate prior distributions for unknown bias and noise scale parameters are selected, and the VB method is utilized to jointly infer the system state, unknown measurement biases and inaccurate measurement noise covariance matrix. Finally, a VB based robust filter is derived and its effectiveness is verified by the numerical simulations.

Bayesian Method for Quality Control with Weibull Distribution

Weibull distribution is one of the continous probability density function. This distribution is known as a flexible distribution. One of it’s flexibility is can be transformed into other distributions such as exponential distribution depends on parameter selected. Similar to other distribution, Weibull distribution also characterized by cumulative distribution function, mean, variance and moment generating function. One of the well-known application of this distribution is in the field of quality control utilizing reliability data and the well-known tool in quality control is control chart. Therefore, reliability data is not follow normal distribution then Shewhart control chart cannot be applied. To solve this problem, control chart can be formed with the control limit is obtained by Bayesian method. In applying Bayesian method, prior distribution of shape parameter is assumed to be uniform distribution and variables for reliability is assumed to be invers of Weibull distribution. The prior distribution is combine with likelihood function then posterior distribution is obtained which in then used as the control limits by find it’s mean.

A multiobjective optimization framework for site investigation program based on Bayesian approach and NSGA‐II

AbstractSite investigation provides essential geotechnical parameter information for analysis and design. However, three conflicting objectives, namely exploration effort, robustness and parameter uncertainty, pose a challenge to the development of an optimal site investigation program. In this study, a three objective optimization framework for the site investigation program is proposed based on the Bayesian approach and the non‐dominated sorting genetic algorithm (NSGA‐II). The only inputs required by the proposed framework are prior distribution of geotechnical parameters and error information. The prior distribution of geotechnical parameters is derived from integrating engineering experience and measurements from basic exploration boreholes. The error information is obtained based on literature and expert judgment related to the specific project. Firstly, a design pool of candidate investigation programs is generated using Bayesian approach to determine the locations and number of exploration boreholes. The NSGA‐II is then applied to identify the optimal program that balances lower cost, higher robustness, and lower uncertainty. The proposed multiobjective optimization framework is illustrated and validated through a real site investigation case in Chongqing, China, aimed at determining the ultimate bearing capacity of the rock foundation. The spatial correlation of parameters within the study area is also considered. The optimal program is represented by the location and number of exploration boreholes. By comparing measurements with predictions from different site investigation programs, the efficiency of the proposed multiobjective framework is demonstrated. Additionally, the influence of engineering experience and random field modeling on the investigation program is discussed.

Inverse heat transfer problem for the characterization of nanofluids produced with different types of palladium nanoparticles

The objective of this work is the measurement of physical properties of distilled water nanofluids containing palladium nanocubes, palladium cerium oxide nanoparticles, and their respective hydrides. Due to their biocompatibility and favorable photothermal effects, palladium nanoparticles can be used to promote localized absorption of external energy sources in the thermal treatment of cancer, aiming at a thermal damage constrained to the tumor region without significant effects to the healthy tissues. An inverse problem is solved here within the Bayesian framework of statistics with the Markov Chain Monte Carlo method, by using nonintrusive transient measurements taken with an infrared camera during the heating of different water-based nanofluids. The mathematical model used in this work takes into account natural convection effects, due to the nonuniform heat source caused by the diode-laser that heats the nanofluids. Prior distributions for the model parameters were selected based on additional independent measurements, theoretical models and by the careful implementation of the experiments. The proposed model and the estimated parameters were validated, with an excellent agreement between the measured temperatures and those obtained from stochastic simulations during the solution of the inverse problem.

Deep network fault diagnosis for imbalanced small-sized samples via a coupled adversarial autoencoder based on the Bayesian method.

Deep network fault diagnosis methods heavily rely on abundant labeled data for effective model training. However, small-sized samples and imbalanced samples often lead to insufficient features, resulting in accuracy degradation and even instability in the diagnosis model. To address this challenge, this paper introduces a coupled adversarial autoencoder (CoAAE) based on the Bayesian method. This model aims to solve the issue of insufficient samples by generating fake samples and integrating them with the original ones. Within the CoAAE framework, the probability density distribution of the original data is captured using an encoder and fake samples are generated by random sampling from this distribution and decoding them. This process is the adversarial interaction between the encoder and a classifier to obtain the prior distribution of the encoder's parameters. The encoder's parameters are updated through the decoder's reconstruction process, leading to the posterior distribution. Concurrently, the decoder is trained to enhance its ability to reconstruct samples accurately. To address the imbalance in the original samples, a parallel coupled network is employed. This network shares the weights of the extraction layer in the encoder, enabling it to learn the joint distribution between fault-related and normal samples. To evaluate the effectiveness of the proposed data augmentation method, experiments were conducted on a bearing database from Case Western Reserve University using ResNet18 as the deep learning diagnosis model representative. The results demonstrate that CoAAE can effectively augment imbalanced datasets and outperform other advanced methods.

Normalizing flow based uncertainty estimation for deep regression analysis

Uncertainty estimation is a critical component of building safe and reliable machine learning models. Accurate estimation of uncertainties is essential for identifying and mitigating potential risks and ensuring that machine learning systems operate reliably in real-world scenarios. Various approaches, such as ensemble and Bayesian neural networks have been developed by sampling probability predictions from submodels, which is computationally expensive. At present, these techniques are incapable of precisely delineating the boundary separating in-distribution (ID) and out-of-distribution (OOD) data. To fill up this research gap, this paper presents a normalizing flow based framework to directly predict parameters of prior distributions over the probability with a neural network, the proposed model is able to effectively differentiate between ID and OOD data in regression problems. The posterior distributions learned by the model precisely represent uncertainties for OOD data based solely on ID data, without the need for OOD data during training. This approach has shown promising results in a number of applications, including image depth estimation and image adversarial attacks.

Locally, Bayesian and non parametric Bayesian optimal designs for unit exponential regression model

. This study introduces optimal designs for the unit exponential (UE) non linear model using local, Bayesian, and non parametric Bayesian approaches. In the local approach, optimal designs were derived by substituting initial estimates for the unknown parameters. However, recognizing the inefficiency of these designs when initial estimates are distant from their true values, we adopted a Bayesian approach, employing the D-optimal criterion with uniform and truncated normal prior distributions for unknown parameters. In situations lacking informative or historical knowledge of parameters, a non parametric Bayesian approach was employed, incorporating the Dirichlet Process (DP) prior to the space of distribution functions. Finally, the efficiency of various optimal designs was analyzed for comparison.

Sensitivity analysis of prior distributions in Bayesian graphical modeling: Guiding informed prior choices for conditional independence testing

Bayesian analysis methods provide a significant advancement in network psychometrics, allowing researchers to use the edge inclusion Bayes factor for testing conditional independence between pairs of variables in the network. Using this methodology requires setting prior distributions on the network parameters and on the network’s structure. However, the impact of both prior distributions on the inclusion Bayes factor is underexplored. In this paper, we focus on a specific class of Markov Random Field models for ordinal and binary data. We first discuss the different choices of prior distributions for the network parameters and the network structure, and then perform an extensive simulation study to assess the sensitivity of the inclusion Bayes factor to these distributions. We pay particular attention to the effect of the scale of the prior on the inclusion Bayes factor. To improve the accessibility of the results, we also provide an interactive Shiny app. Finally, we present the R package simBgms, which provides researchers with a user-friendly tool to perform their own simulation studies for Bayesian Markov Random Field models. All of this should help researchers make more informed, evidence-based decisions when preparing to analyze empirical data using network psychometric models.

Bayesian random-effects meta-analysis with empirical heterogeneity priors for application in health technology assessment with very few studies.

In Bayesian random-effects meta-analysis, the use of weakly informative prior distributions is of particular benefit in cases where only a few studies are included, a situation often encountered in health technology assessment (HTA). Suggestions for empirical prior distributions are available in the literature but it is unknown whether these are adequate in the context of HTA. Therefore, a database of all relevant meta-analyses conducted by the Institute for Quality and Efficiency in Health Care (IQWiG, Germany) was constructed to derive empirical prior distributions for the heterogeneity parameter suitable for HTA. Previously, an extension to the normal-normal hierarchical model had been suggested for this purpose. For different effect measures, this extended model was applied on the database to conservatively derive a prior distribution for the heterogeneity parameter. Comparison of a Bayesian approach using the derived priors with IQWiG's current standard approach for evidence synthesis shows favorable properties. Therefore, these prior distributions are recommended for future meta-analyses in HTA settings and could be embedded into the IQWiG evidence synthesis approach in the case of very few studies.

Resolution-independent generative models based on operator learning for physics-constrained Bayesian inverse problems

The Bayesian inference approach is widely used to solve inverse problems due to its versatile and natural ability to handle ill-posedness. However, there are often challenges when dealing with situations involving continuous fields or parameters with large-resolution discrete representations. Furthermore, the prior distribution of the unknown parameters is also commonly difficult to determine. Therefore, in this study, an operator learning-based generative adversarial network (OL-GAN) is proposed and integrated into the Bayesian inference framework to address these issues. Compared with classical Bayesian approaches, the distinctive characteristic of the proposed method is that it learns the joint distributions of parameters and responses. By using the trained generative model to handle the prior in Bayes’ rule, the posteriors of the unknown parameters can theoretically be approximated by any sampling algorithms (e.g., Markov chain Monte Carlo, MCMC) under the proposed framework. More importantly, efficient sampling can be implemented in a low-dimensional latent space shared by the components of the joint distribution. The latent space is typically a simple and easy-to-sample distribution (e.g., Gaussian, uniform), which significantly reduces the computational cost associated with the Bayesian inference while avoiding prior selection. Furthermore, the generator is resolution-independent due to the incorporation of operator learning. Predictions can thus be obtained at desired coordinates, and inversions can be performed even if the observation data are misaligned with the training data. Finally, the effectiveness of the proposed method is validated through several numerical experiments.

Non-probabilistic credible reliability analysis of the composite laminate

For the reliability analysis of composite laminate structures, there is a problem of difficult data acquisition. Traditional reliability analysis methods require a large amount of data to maintain calculation accuracy. Therefore, the traditional reliability analysis method is not applicable to composite laminates. For the above-mentioned problems, this paper presents a method to calculate the credible reliability of composite laminate structures using the Bayesian theorem. The method is distinguished from the traditional uncertainty analysis methods. The likelihood function and the newly introduced samples are first used to correct the prior distribution of the uncertain parameters. Furthermore, the posterior distribution, containing information from both the sample and prior distributions, can be obtained. Ultimately, based on the acquired posterior distribution, the probability density function can be updated with newly introduced samples. Using the Hashin failure criterion, the failure probabilities of the fibers and matrix of the composite laminate structure are quantified, respectively. Then, the credible reliability of the composite laminate is obtained. Numerical example results show the proposed method can maintain high accuracy with fewer samples and demonstrate the validity of the model. For the numerical calculations, the deterministic parameters of the distribution function and the functional distribution parameters were verified, composite laminates and aircraft wing structures were analyzed. The experimental results show that the error between the probability density function of the parameters and the probability density function predicted by the sample is less than 0.045. Compared with the traditional reliability analysis methods for composite structures, this method can effectively measure the credible reliability of composite laminate structures with fewer samples and similar accuracy. The method enables real-time updating of the parameter probability density function by newly introduced samples and improves the confidence level.

A dynamic power prior approach to non-inferiority trials for normal means.

Non-inferiority trials compare new experimental therapies to standard ones (active control). In these experiments, historical information on the control treatment is often available. This makes Bayesian methodology appealing since it allows a natural way to exploit information from past studies. In the present paper, we suggest the use of previous data for constructing the prior distribution of the control effect parameter. Specifically, we consider a dynamic power prior that possibly allows to discount the level of borrowing in the presence of heterogeneity between past and current control data. The discount parameter of the prior is based on the Hellinger distance between the posterior distributions of the control parameter based, respectively, on historical and current data. We develop the methodology for comparing normal means and we handle the unknown variance assumption using MCMC. We also provide a simulation study to analyze the proposed test in terms of frequentist size and power, as it is usually requested by regulatory agencies. Finally, we investigate comparisons with some existing methods and we illustrate an application to a real case study.

Optimizing prior distribution parameters for probabilistic prediction of remaining useful life using deep learning

In this study, a deep learning-based probabilistic remaining useful life (RUL) prediction model is proposed to improve the strong prior limitations of traditional probabilistic RUL prediction methods through a flexible prior distribution and strategy for sequential optimization of hyperparameters with regularization factor. It enables output richer probabilistic lifetime density distributions and confidence intervals with various parameters and overcome the problem of poor accuracy of short RUL predictions to some extent. Eventually, the model is effectively validated on a benchmark dataset, and the experimental results show that the probabilistic lifetime prediction model with optimized prior distribution parameters significantly improves prediction performance and demonstrates good learning performance and robustness of test results compared with traditional point estimation methods and parameter-free models. This study informs maintenance decisions and reliability assessments in engineering systems and guides the research and application of probabilistic-based prediction methods in deep learning framework.

A sample size-dependent prior strategy for bridging the Bayesian-frequentist gap in point null hypothesis testing

Bayes factor, as a measure of the evidence provided by the data in favor of one hypothesis against its alternative, can be highly sensitive to the prior distributions of parameters involved in the hypotheses as well as to the sample size. This may cause a noticeable difference between the Bayesian and classical (frequentist) hypothesis testing results. In the worst-case scenario, the two results are in conflict, which is termed the Jeffreys-Lindley paradox. In this article, we propose a sample size-dependent prior strategy to bridge the Bayesian-frequentist gap from a decision-theoretical perspective. The central idea behind the proposed strategy is to adaptively adjust prior distributions for the parameters in line with the sample size to manage the risk of type I error in Bayesian hypothesis testing at the same level as that prespecified in frequentist hypothesis testing. The proposed strategy is inspired by the work of Maurice Stevenson Bartlett (M.S. Bartlett, A comment on D. V. Lindley’s statistical paradox, Biometrika, 44, 533–534, 1957), who suggested a sample size-dependent prior to make the Bayes factor independent of the sample size. In contrast to his work, we propose a strategy that leverages the use of sample size-dependent priors in Bayesian hypothesis testing and risk management when deciding the two hypotheses. To demonstrate the effectiveness of the proposed strategy, normal mean tests in the cases that (i) the variance is known (z-test) and (ii) the variance is unknown (t-test) are examined. It turns out that the Bayesian testing results coming out from the proposed strategy become consistent with their frequentist counterparts and the Jeffreys-Lindley paradox disappears.

Comparison between pystan and numpyro in Bayesian item response theory: evaluation of agreement of estimated latent parameters and sampling performance.

The purpose of this study is to compare two libraries dedicated to the Markov chain Monte Carlo method: pystan and numpyro. In the comparison, we mainly focused on the agreement of estimated latent parameters and the performance of sampling using the Markov chain Monte Carlo method in Bayesian item response theory (IRT). Bayesian 1PL-IRT and 2PL-IRT were implemented with pystan and numpyro. Then, the Bayesian 1PL-IRT and 2PL-IRT were applied to two types of medical data obtained from a published article. The same prior distributions of latent parameters were used in both pystan and numpyro. Estimation results of latent parameters of 1PL-IRT and 2PL-IRT were compared between pystan and numpyro. Additionally, the computational cost of the Markov chain Monte Carlo method was compared between the two libraries. To evaluate the computational cost of IRT models, simulation data were generated from the medical data and numpyro. For all the combinations of IRT types (1PL-IRT or 2PL-IRT) and medical data types, the mean and standard deviation of the estimated latent parameters were in good agreement between pystan and numpyro. In most cases, the sampling time using the Markov chain Monte Carlo method was shorter in numpyro than that in pystan. When the large-sized simulation data were used, numpyro with a graphics processing unit was useful for reducing the sampling time. Numpyro and pystan were useful for applying the Bayesian 1PL-IRT and 2PL-IRT. Our results show that the two libraries yielded similar estimation result and that regarding to sampling time, the fastest libraries differed based on the dataset size.

An informative prior distribution on functions with application to functional regression

We provide a prior distribution for a functional parameter so that its trajectories are smooth and vanish on a given subset. This distribution can be interpreted as the distribution of an initial Gaussian process conditioned to be zero on a given subset. Precisely, we show that the initial Gaussian process is the sum of the conditioned process and an independent process with probability one and that all the processes have the same almost sure regularity. This prior distribution is use to provide an interpretable estimate of the coefficient function in the linear scalar‐on‐function regression; by interpretable, we mean a smooth function that may possibly be zero on some intervals. We apply our model in a simulation and real case studies with two different priors for the null region of the coefficient function. In one case, the null region is known to be an unknown single interval. In the other case, it can be any unknown unions of intervals.

Bayesian ridge estimators based on copula-based joint prior distributions for logistic regression parameters

Ridge regression was originally proposed as an alternative to ordinary least-squares regression to address multicollinearity in linear regression and was later extended to logistic and Cox regressions. The ridge estimator is interpreted as the Bayesian posterior mean or median in the Bayesian framework when the regression coefficients have multivariate normal priors. We previously proposed using vine copula-based joint priors on regression coefficients in linear and Cox regressions, including an interaction that promotes the use of ridge regression because the interaction term can result in multicollinearity. We showed that vine copula-based priors improve the estimation accuracy over the multivariate normal prior, and they would be a promising approach in other regression types, such as logistic regression. In this study, we focus on a case involving two covariates and their interaction terms, and propose a vine copula-based prior for Bayesian ridge estimators under a logistic model. Simulation and data analysis results show that Archimedean (Clayton and Gumbel) copula priors are superior to other priors (the Gaussian copula and trivariate normal priors) in the presence of multicollinearity.

The influence of inter-regional delays in generating large-scale brain networks of phase synchronization

Large-scale networks of phase synchronization are considered to regulate the communication between brain regions fundamental to cognitive function, but the mapping to their structural substrates, i.e., the structure-function relationship, remains poorly understood. Biophysical Network Models (BNMs) have demonstrated the influences of local oscillatory activity and inter-regional anatomical connections in generating alpha-band (8–12 Hz) networks of phase synchronization observed with Electroencephalography (EEG) and Magnetoencephalography (MEG). Yet, the influence of inter-regional conduction delays remains unknown. In this study, we compared a BNM with standard “distance-dependent delays”, which assumes constant conduction velocity, to BNMs with delays specified by two alternative methods accounting for spatially varying conduction velocities, “isochronous delays” and “mixed delays”. We followed the Approximate Bayesian Computation (ABC) workflow, i) specifying neurophysiologically informed prior distributions of BNM parameters, ii) verifying the suitability of the prior distributions with Prior Predictive Checks, iii) fitting each of the three BNMs to alpha-band MEG resting-state data (N = 75) with Bayesian optimization for Likelihood-Free Inference (BOLFI), and iv) choosing between the fitted BNMs with ABC model comparison on a separate MEG dataset (N = 30). Prior Predictive Checks revealed the range of dynamics generated by each of the BNMs to encompass those seen in the MEG data, suggesting the suitability of the prior distributions. Fitting the models to MEG data yielded reliable posterior distributions of the parameters of each of the BNMs. Finally, model comparison revealed the BNM with “distance-dependent delays”, as the most probable to describe the generation of alpha-band networks of phase synchronization seen in MEG. These findings suggest that distance-dependent delays might contribute to the neocortical architecture of human alpha-band networks of phase synchronization. Hence, our study illuminates the role of inter-regional delays in generating the large-scale networks of phase synchronization that might subserve the communication between regions vital to cognition.

Interface State Density Prediction between an Insulator and a Semiconductor by Gaussian Process Regression Models for a Modified Process.

During data-driven process condition optimization on a laboratory scale, only a small-size data set is accessible and should be effectively utilized. On the other hand, during process development, new operations are frequently inserted or current operations are modified. These accessible data sets are somewhat related but not exactly the same type. In this study, we focus on the prediction of the quality of the interface between an insulator and GaN as a semiconductor for the potential application of GaN power semiconductor devices. The quality of the interface was represented as the interface state density, Dit, and the inserted operation to the process was the ultraviolet (UV)/O3-gas treatment. Our retrospective evaluation of model-building approaches for Dit prediction from a process condition revealed that for the UV/O3-treated interfaces, data of interfaces without the treatment contributed to performance improvement. Such performance improvement was not observed when using a data set of Si as the semiconductor. As a modeling method, the automatic relevance vector-based Gaussian process regression with the prior distribution of the length-scale parameters exhibited a relatively high predictive performance and represented a reasonable uncertainty of prediction as reflected by the distance to the training data set. This feature is a prerequisite for a potential application of Bayesian optimization. Furthermore, hyperparameters in the prior distribution of the length-scales could be optimized by leave-one-out cross-validation.

Discussion of specifying prior distributions in reliability applications

AbstractThe paper Specifying Prior Distributions in Reliability Applications mainly provides an overview of methods for selecting non‐informative prior distributions for parameters of basic lifetime distributions, as often used in reliability analyses. This discussion raises some related issues and comments on opportunities beyond basic Bayesian statistical methods which may be useful in reliability scenarios. The main emphasis in this discussion is on practical reliability analyses with few data available, where there is often need for informative priors rather than for non‐informative priors, in order to take expert judgement into account. Furthermore, while rather abstract considerations of non‐informativeness of prior distributions is of theoretic interest, in most practical scenarios one aims at decision support, and the influence of assumed priors on the final decisions should be considered, ideally with robustness of the final decision with regard to all priors which are deemed to be reasonable.