Relevant Prior Information Research Articles

Conjugate gradient‐based active noise cancellation

AbstractActive noise cancellation schemes should synthesize proper secondary perturbations without relevant prior information. Their performance depends strongly on their adaptive mechanisms, which, sometimes, do not fulfill the severe requirements of practical setups. This research demonstrates for the first time that the adaptive conjugate gradient optimization method can be used for active noise cancellation tasks. Additionally, a stochastic model is devised for estimating its steady‐state solution, even when the secondary path is not correctly estimated. Such a theoretical analysis demonstrates the asymptotic unbiasedness of the proposed scheme when the secondary plant is perfectly estimated. Simulations reveal that the devised method can attain better asymptotic performance and a higher convergence rate than the traditional FX‐LMS algorithm, at the expense of an increased computational burden. Furthermore, simulations show that the method outperforms the standard ones for nonstationary inputs (such as speech signals).

Bayesian estimation in veterinary pharmacology: A conceptual and practical introduction.

Sophisticated mathematical and computational tools have become widespread and important in veterinary pharmacology. Although the theoretical basis and practical applications of these have been widely explored in the literature, statistical inference in the context of these models has received less attention. Optimization methods, often with frequentist statistical inference, have been predominant. In contrast, Bayesian statistics have not been widely applied, but offer both practical utility and arguably greater interpretability. Veterinary pharmacology applications are generally well supported by relevant prior information, from either existing substantive knowledge, or an understanding of study and model design. This facilitates practical implementation of Bayesian analyses that can take advantage of this knowledge. This essay will explore the specification of Bayesian models relevant to veterinary pharmacology, including demonstration of prior selection, and illustrate the capability of these models to generate practically useful statistics, including uncertainty statements, that are difficult or impossible to obtain otherwise. Case studies using simulated data will describe applications in clinical trials, pharmacodynamics, and pharmacokinetics, all including multilevel modeling. This content may serve as a suitable starting point for researchers in veterinary pharmacology and related disciplines considering Bayesian estimation for their applied work.

Multicollinearity in Multiple Linear Regression: Detection, Consequences, and Remedies

Abstract: Multiple linear regression is a widely used statistical tool for modeling relationships between a dependent variable and multiple explanatory variables. However, it assumes that these explanatory variables are independent, which is not always the case in practical scenarios, leading to a phenomenon known as multicollinearity. Multicollinearity occurs when explanatory variables in a regression model are strongly correlated with each other, causing several issues in regression analysis. This paper discusses the detection and remedies for multicollinearity in detail. Detection methods include examining the determinant of the correlation matrix, inspecting correlation coefficients, using partial regression coefficients, calculating Variance Inflation Factors (VIFs), and assessing the condition number and condition index. These techniques help researchers identify the presence and severity of multicollinearity in their dataset. To address multicollinearity, several remedies are proposed, including obtaining more data, dropping collinear variables, using relevant prior information, employing generalized inverses, and employing principal component regression. Ridge regression, which introduces bias to reduce variance, is also discussed as an effective technique to combat multicollinearity. Understanding multicollinearity and employing appropriate detection and remediation strategies is crucial for obtaining reliable and meaningful results from multiple linear regression models

A Taxonomic Survey of Physics-Informed Machine Learning

Physics-informed machine learning (PIML) refers to the emerging area of extracting physically relevant solutions to complex multiscale modeling problems lacking sufficient quantity and veracity of data with learning models informed by physically relevant prior information. This work discusses the recent critical advancements in the PIML domain. Novel methods and applications of domain decomposition in physics-informed neural networks (PINNs) in particular are highlighted. Additionally, we explore recent works toward utilizing neural operator learning to intuit relationships in physics systems traditionally modeled by sets of complex governing equations and solved with expensive differentiation techniques. Finally, expansive applications of traditional physics-informed machine learning and potential limitations are discussed. In addition to summarizing recent work, we propose a novel taxonomic structure to catalog physics-informed machine learning based on how the physics-information is derived and injected into the machine learning process. The taxonomy assumes the explicit objectives of facilitating interdisciplinary collaboration in methodology, thereby promoting a wider characterization of what types of physics problems are served by the physics-informed learning machines and assisting in identifying suitable targets for future work. To summarize, the major twofold goal of this work is to summarize recent advancements and introduce a taxonomic catalog for applications of physics-informed machine learning.

Towards learning optimized kernels for complex Langevin

We present a novel strategy aimed at restoring correct convergence in complex Langevin simulations. The central idea is to incorporate system-specific prior knowledge into the simulations, in order to circumvent the NP-hard sign problem. In order to do so, we modify complex Langevin using kernels and propose the use of modern auto-differentiation methods to learn optimal kernel values. The optimization process is guided by functionals encoding relevant prior information, such as symmetries or Euclidean correlator data. Our approach recovers correct convergence in the non-interacting theory on the Schwinger-Keldysh contour for any real-time extent. For the strongly coupled quantum anharmonic oscillator we achieve correct convergence up to three-times the real-time extent of the previous benchmark study. An appendix sheds light on the fact that for correct convergence not only the absence of boundary terms, but in addition the correct Fokker-Plank spectrum is crucial.

Quantifying Observed Prior Impact

When summarizing a Bayesian analysis, it is important to quantify the contribution of the prior distribution to the final posterior inference because this informs other researchers whether the prior information needs to be carefully scrutinized, and whether alternative priors are likely to substantially alter the conclusions drawn. One appealing and interpretable way to do this is to report an effective prior sample size (EPSS), which captures how many observations the information in the prior distribution corresponds to. However, typically the most important aspect of the prior distribution is its location relative to the data, and therefore traditional information measures are somewhat deficit for the purpose of quantifying EPSS, because they concentrate on the variance or spread of the prior distribution (in isolation from the data). To partially address this difficulty, Reimherr et al. (2014) introduced a class of EPSS measures based on prior-likelihood discordance. In this paper, we take this idea further by proposing a new measure of EPSS that not only incorporates the general mathematical form of the likelihood (as proposed by Reimherr et al., 2014) but also the specific data at hand. Thus, our measure considers the location of the prior relative to the current observed data, rather than relative to the average of multiple datasets from the working model, the latter being the approach taken by Reimherr et al. (2014). Consequently, our measure can be highly variable, but we demonstrate that this is because the impact of a prior on a Bayesian analysis can intrinsically be highly variable. Our measure is called the (posterior) mean Observed Prior Effective Sample Size (mOPESS), and is a Bayes estimate of a meaningful quantity. The mOPESS well communicates the extent to which inference is determined by the prior, or framed differently, the amount of sampling effort saved due to having relevant prior information. We illustrate our ideas through a number of examples including Gaussian conjugate and non-conjugate models (continuous observations), a Beta-Binomial model (discrete observations), and a linear regression model (two unknown parameters).

A Multitask Learning Approach to Personalized Blood Glucose Prediction.

Blood glucose prediction algorithms are key tools in the development of decision support systems and closed-loop insulin delivery systems for blood glucose control in diabetes. Deep learning models have provided leading results among machine learning algorithms to date in glucose prediction. However these models typically require large amounts of data to obtain best personalised glucose prediction results. Multitask learning facilitates an approach for leveraging data from multiple subjects while still learning accurate personalised models. In this work we present results comparing the effectiveness of multitask learning over sequential transfer learning, and learning only on subject-specific data with neural network and support vector regression. The multitask learning approach shows consistent leading performance in predictive metrics at both short-term and long-term prediction horizons. We obtain a predictive accuracy (RMSE) of 18.8 ±2.3, 25.3 ±2.9, 31.8 ±3.9, 41.2 ±4.5, 47.2 ±4.6mg/dL at 30, 45, 60, 90, and 120min prediction horizons respectively, with at least 93% clinically acceptable predictions using the Clarke Error Grid (EGA) at each prediction horizon. We also identify relevant prior information such as glycaemic variability that can be incorporated to improve predictive performance at long-term prediction horizons. Furthermore, we show consistent performance - ≤ 5% change in both RMSE and EGA (Zone A) - in rare cases of adverse glycaemic events with 1-6 weeks of training data. In conclusion, a multitask approach can allow for deploying personalised models even with significantly less subject-specific data without compromising performance.

Fast Hyperparameter Calibration of Sparsity Enforcing Penalties in Total Generalised Variation Penalised Reconstruction Methods for XCT Using a Planted Virtual Reference Image

The reconstruction problem in X-ray computed tomography (XCT) is notoriously difficult in the case where only a small number of measurements are made. Based on the recently discovered Compressed Sensing paradigm, many methods have been proposed in order to address the reconstruction problem by leveraging inherent sparsity of the object’s decompositions in various appropriate bases or dictionaries. In practice, reconstruction is usually achieved by incorporating weighted sparsity enforcing penalisation functionals into the least-squares objective of the associated optimisation problem. One such penalisation functional is the Total Variation (TV) norm, which has been successfully employed since the early days of Compressed Sensing. Total Generalised Variation (TGV) is a recent improvement of this approach. One of the main advantages of such penalisation based approaches is that the resulting optimisation problem is convex and as such, cannot be affected by the possible existence of spurious solutions. Using the TGV penalisation nevertheless comes with the drawback of having to tune the two hyperparameters governing the TGV semi-norms. In this short note, we provide a simple and efficient recipe for fast hyperparameters tuning, based on the simple idea of virtually planting a mock image into the model. The proposed trick potentially applies to all linear inverse problems under the assumption that relevant prior information is available about the sought for solution, whilst being very different from the Bayesian method.

Visual short-term memory load modulates repetition related fMRI signal adaptation

While several computational models have suggested how predictive coding could be implemented on an algorithmic level, reference to cognitive processes remains rather sparse. A crucial process might be elevating relevant prior information from long-term memory to render it highly accessible for subsequent comparison with sensory input. In many models, visual short-term memory (VSTM) is considered as information from long-term memory in a state of elevated activity. We measured the BOLD signal in face-specific cortical areas using repetition suppression (RS) paradigm. RS has been associated with predictive processing in previous studies. We show that RS within the fusiform face area is significantly attenuated when VSTM is loaded with other, non-facial visual information. Although an unequivocal inference is not possible, the data indicate a role of VSTM for predictive processes as indexed by expectation-related RS.

Bayesian Meta-analysis of Safety Outcomes Using Blinded Clinical Trial Data.

During the development of an investigational new drug, identifying potential safety risks is imperative. Early detection, as well as federal regulations, requires safety monitoring procedures during clinical trials while the data are still blinded. We introduce a Bayesian meta-analytic approach for detecting a potential elevated safety risk conferred by an investigational treatment, as compared to control, using blinded data from multiple trials. Simulation studies of our approach demonstrate good operating characteristics when some relevant prior information is available. The utility of our procedure is demonstrated on data from a real clinical trial program with informative results based on blinded data compared with unblinded data. At any time during the drug development program, our approach provides easily interpretable posterior probability statements about the risk ratio for the safety event(s) of interest.

How Do I Know What My Theory Predicts?

To get evidence for or against a theory relative to the null hypothesis, one needs to know what the theory predicts. The amount of evidence can then be quantified by a Bayes factor. Specifying the sizes of the effect one’s theory predicts may not come naturally, but I show some ways of thinking about the problem, some simple heuristics that are often useful when one has little relevant prior information. These heuristics include the room-to-move heuristic (for comparing mean differences), the ratio-of-scales heuristic (for regression slopes), the ratio-of-means heuristic (for regression slopes), the basic-effect heuristic (for analysis of variance effects), and the total-effect heuristic (for mediation analysis).

Bayesian method for solving the problem of multicollinearity in regression

The popular method of estimation in regression, Ordinary Least Squares (OLS) often displays inefficiency especially with large variances and wide confidence intervals thereby making precise estimate difficult when there is strong multicollinearity. Bayesian method of estimation is expected to improve the efficiency of estimated regression model when there is relevant prior information and belief of situation being modelled is available. This study however provided an alternative approach to OLS when there is almost perfect multicollinearity while its performance were compared with the aid of simulation approach to OLS estimator. Results of the simulation study indicate that with respect to Mean Squared Error (MSE) criterion and other criteria, the proposed method perform better than OLS. Keywords: Multicollinearity, Regression, Standard Error, Simulation AMS 2010 Mathematics Subject Classification: 62F15, 62GO5, 62H10

Bayesian estimation of logistic regression with misclassified covariates and response

ABSTRACTMeasurement error is a commonly addressed problem in psychometrics and the behavioral sciences, particularly where gold standard data either does not exist or are too expensive. The Bayesian approach can be utilized to adjust for the bias that results from measurement error in tests. Bayesian methods offer other practical advantages for the analysis of epidemiological data including the possibility of incorporating relevant prior scientific information and the ability to make inferences that do not rely on large sample assumptions. In this paper we consider a logistic regression model where both the response and a binary covariate are subject to misclassification. We assume both a continuous measure and a binary diagnostic test are available for the response variable but no gold standard test is assumed available. We consider a fully Bayesian analysis that affords such adjustments, accounting for the sources of error and correcting estimates of the regression parameters. Based on the results from our example and simulations, the models that account for misclassification produce more statistically significant results, than the models that ignore misclassification. A real data example on math disorders is considered.

Dual-Energy Head CT Enables Accurate Distinction of Intraparenchymal Hemorrhage from Calcification in Emergency Department Patients

Purpose To evaluate the ability of dual-energy (DE) computed tomography (CT) to differentiate calcification from acute hemorrhage in the emergency department setting. Materials and Methods In this institutional review board-approved study, all unenhanced DE head CT examinations that were performed in the emergency department in November and December 2014 were retrospectively reviewed. Simulated 120-kVp single-energy CT images were derived from the DE CT acquisition via postprocessing. Patients with at least one focus of intraparenchymal hyperattenuation on single-energy CT images were included, and DE material decomposition postprocessing was performed. Each focal hyperattenuation was analyzed on the basis of the virtual noncalcium and calcium overlay images and classified as calcification or hemorrhage. Sensitivity, specificity, and accuracy were calculated for single-energy and DE CT by using a common reference standard established by relevant prior and follow-up imaging and clinical information. Results Sixty-two cases with 68 distinct intraparenchymal hyperattenuating lesions in which the reference standards were available were included in the study, of which 41 (60%) were confirmed as calcification and 27 (40%) were confirmed as hemorrhage. Sensitivity, specificity, and accuracy of DE CT for the detection of hemorrhage were 96% (95% confidence interval [CI]: 81%, 100%), 100% (95% CI: 91%, 100%), and 99% (95% CI: 92%, 100%) and those of single-energy CT were 74% (95% CI: 54%, 89%), 95% (95% CI: 83%, 99%), and 87% (95% CI: 76%, 94%), respectively. Six of 68 (9%) lesions were classified as indeterminate and three (4%) were misinterpreted with single-energy CT alone and were correctly classified with DE CT. Conclusion DE CT by using material decomposition enables accurate differentiation between calcification and hemorrhage in patients presenting for emergency head imaging and can be especially useful in problem-solving complex cases that are difficult to determine based on conventional CT appearance alone. (©) RSNA, 2016 Online supplemental material is available for this article.

Lindley-Shannon Information for Comparison of Priors Under Paired Comparisons Model

One of the main differences between classical statistics and Bayesian statistics is that the latter can utilise prior information in a formal way. This information can be quantified in terms of a probability distribution which is known as the prior distribution. If there is no relevant prior information available then there are ways to derive a 'non-informative' prior distribution. In this study, the informative and non-informative prior distributions for the parameters of the Bradley-Terry model for paired comparison data are compared.

Predictive distributions for between-study heterogeneity and simple methods for their application in Bayesian meta-analysis.

Numerous meta-analyses in healthcare research combine results from only a small number of studies, for which the variance representing between-study heterogeneity is estimated imprecisely. A Bayesian approach to estimation allows external evidence on the expected magnitude of heterogeneity to be incorporated.The aim of this paper is to provide tools that improve the accessibility of Bayesian meta-analysis. We present two methods for implementing Bayesian meta-analysis, using numerical integration and importance sampling techniques. Based on 14 886 binary outcome meta-analyses in the Cochrane Database of Systematic Reviews, we derive a novel set of predictive distributions for the degree of heterogeneity expected in 80 settings depending on the outcomes assessed and comparisons made. These can be used as prior distributions for heterogeneity in future meta-analyses.The two methods are implemented in R, for which code is provided. Both methods produce equivalent results to standard but more complex Markov chain Monte Carlo approaches. The priors are derived as log-normal distributions for the between-study variance, applicable to meta-analyses of binary outcomes on the log odds-ratio scale. The methods are applied to two example meta-analyses, incorporating the relevant predictive distributions as prior distributions for between-study heterogeneity.We have provided resources to facilitate Bayesian meta-analysis, in a form accessible to applied researchers, which allow relevant prior information on the degree of heterogeneity to be incorporated. © 2014 The Authors. Statistics in Medicine published by John Wiley & Sons Ltd.

Adaptive framework for uncertainty analysis in electromagnetic field measurements.

Misinterpretation of uncertainty in the measurement of the electromagnetic field (EMF) strength may lead to an underestimation of exposure risk or an overestimation of required measurements. The Guide to the Expression of Uncertainty in Measurement (GUM) has internationally been adopted as a de facto standard for uncertainty assessment. However, analyses under such an approach commonly assume unrealistic static models or neglect relevant prior information, resulting in non-robust uncertainties. This study proposes a principled and systematic framework for uncertainty analysis that fuses information from current measurements and prior knowledge. Such a framework dynamically adapts to data by exploiting a likelihood function based on kernel mixtures and incorporates flexible choices of prior information by applying importance sampling. The validity of the proposed techniques is assessed from measurements performed with a broadband radiation meter and an isotropic field probe. The developed framework significantly outperforms GUM approach, achieving a reduction of 28% in measurement uncertainty.

Contextual Abductive Reasoning with Side-Effects

AbstractThe belief bias effect is a phenomenon which occurs when we think that we judge an argument based on our reasoning, but are actually influenced by our beliefs and prior knowledge. Evans, Barston and Pollard carried out a psychological syllogistic reasoning task to prove this effect. Participants were asked whether they would accept or reject a given syllogism. We discuss one specific case which is commonly assumed to be believable but which is actually not logically valid. By introducing abnormalities, abduction and background knowledge, we adequately model this case under the weak completion semantics. Our formalization reveals new questions about possible extensions in abductive reasoning. For instance, observations and their explanations might include some relevant prior abductive contextual information concerning some side-effect or leading to a contestable or refutable side-effect. A weaker notion indicates the support of some relevant consequences by a prior abductive context. Yet another definition describes jointly supported relevant consequences, which captures the idea of two observations containing mutually supportive side-effects. Though motivated with and exemplified by the running psychology application, the various new general abductive context definitions are introduced here and given a declarative semantics for the first time, and have a much wider scope of application. Inspection points, a concept introduced by Pereira and Pinto, allows us to express these definitions syntactically and intertwine them into an operational semantics.

Estimating the standard deviation of soil properties with limited samples through the Bayesian approach

Characterizing the standard deviation of soil properties is important to a geotechnical probabilistic analysis, and the task is usually achieved with sufficient samples. However, sometimes soil samples or soil tests in a project could be limited (e.g., only one sample), making classical statistics approaches less applicable to the estimating. In this technical note, we introduce a new Bayesian algorithm to estimate the standard deviation of soil properties, using limited project-specific samples along with relevant prior information from the literature. In addition to the methodology, a few demonstrations are also given in the paper, to re-evaluate the standard deviation of soil properties with the new algorithm. Like many Bayesian algorithms, the new application could be useful for site characterizations when samples are limited.

Bayesian Semiparametric Predictive Modeling with Applications in Dose-Response Prediction

A framework is proposed for making quality predictions in situations for which only systematically inaccurate data are available. The predictions are based on the systematically inaccurate data, complete data from similar situations, and expert knowledge. The proposed predictive model is well suited to functional data and is computationally simple, fast, and stable. We focus primarily on a particular problem presenting itself in the pharmaceutical industry. Predicting both side effect and endpoint dose responses before the initiation of a clinical trial has enormous ethical and financial importance in the pharmaceutical industry. The proposed Bayesian semiparametric predictive model is used to predict unobserved clinical dose-response curves conditional on preclinical data, data from similar compounds, and prior knowledge. The model allows for nonlinear dose-response curves and the incorporation of relevant prior information. Posterior sampling is achieved through a simple and computationally efficient Gibbs sampler. The predictions from the model are drawn from the posterior distribution of the average dose-response curve for the candidate compound, allowing straightforward incorporation into a risk assessment model unlike the deterministic predictions often used currently. The model is used on actual data from the pharmaceutical industry, showing that the model is capable of predicting lack or presence of trend with appropriate uncertainty.