Prediction Error Estimation Research Articles

A variational Bayesian inference technique for model updating of structural systems with unknown noise statistics

Dynamic models of structural and mechanical systems can be updated to match the measured data through a Bayesian inference process. However, the performance of classical (non-adaptive) Bayesian model updating approaches decreases significantly when the pre-assumed statistical characteristics of the model prediction error are violated. To overcome this issue, this paper presents an adaptive recursive variational Bayesian approach to estimate the statistical characteristics of the prediction error jointly with the unknown model parameters. This approach improves the accuracy and robustness of model updating by including the estimation of model prediction error. The performance of this approach is demonstrated using numerically simulated data obtained from a structural frame with material non-linearity under earthquake excitation. Results show that in the presence of non-stationary noise/error, the non-adaptive approach fails to estimate unknown model parameters, whereas the proposed approach can accurately estimate them.

Body composition among Malawian young adolescents: Cross-validating predictive equations for bioelectric impedance analysis using deuterium dilution method.

Body composition can be measured by several methods, each with specific benefits and disadvantages. Bioelectric impedance offers a favorable balance between accuracy, cost and ease of measurement in a range of settings. In this method, bioelectric measurements are converted to body composition measurements by prediction equations specific to age, population and bioimpedance device. Few prediction equations exist for populations in low-resource settings. We formed a prediction equation for total body water in Malawian adolescents using deuterium dilution as reference. We studied 86 boys and 92 girls participating in the 11-14-year follow-up of the Lungwena Antenatal Intervention Study, a randomized trial of presumptive infection treatment among pregnant women. We measured body composition by Seca m515 bioimpedance analyser. Participants ingested a weight-standardized dose of deuterium oxide, after which we collected saliva at baseline, at 3 and 4 h post-ingestion, measured deuterium concentration using Fourier-transform infrared spectroscopy and calculated total body water. We formed predictive equations for total body water using anthropometrics plus resistance and reactance at a range of frequencies, applying multiple regression and repeated cross-validation in model building and in prediction error estimation. The best predictive model for percentage total body water (TBW %) was 100*(1.11373 + 0.0037049*height (cm)2/resistance(Ω) at 50 kHz- 0.25778*height(m)- 0.01812*BMI(kg/m2)- 0.02614*female sex). Calculation of absolute TBW (kg) by multiplying TBW (%) with body weight had better predictive power than a model directly constructed to predict absolute total body water (kg). This model explained 96.4% of variance in TBW (kg) and had a mean prediction error of 0.691 kg. Mean bias was 0.01 kg (95% limits of agreement -1.34, 1.36) for boys and -0.01 kg (1.41, 1.38) for girls. Our equation provides an accurate, cost-effective and participant-friendly body composition prediction method among adolescents in clinic-based field studies in rural Africa, where electricity is available.

Serum Creatinine and Tacrolimus Assessment With VAMS Finger-Prick Microsampling: A Diagnostic Test Study.

Kidney transplant recipients require frequent venipunctures. Microsampling methods that use a finger-prick draw of capillary blood, like volumetric absorptive microsamplers (VAMS), have the potential to reduce the pain, inconvenience, and volume of blood loss associated with venipuncture. This study aimed to provide diagnostic accuracy using VAMS for measurement of tacrolimus and creatinine compared to gold standard venous blood in adult kidney transplant recipients. Diagnostic test study. Prospective blood samples for measurement of tacrolimus and creatinine were collected using Mitra VAMS and venipuncture immediately before and 2 hours after tacrolimus dosing. A convenience sample of 40 adult kidney transplant participants in the outpatient setting. Method comparison was assessed by Passing-Bablok regression and Bland-Altman analysis. The predictive performance of VAMS measurement compared to venipuncture was also assessed through estimation of the median prediction error and median absolute percentage prediction error. A total of 74 tacrolimus samples and 70 creatinine samples were analyzed from 40 participants. Passing-Bablok regression showed a systematic difference between VAMS and venipuncture when measuring tacrolimus and creatinine with a slope of 1.08 (95% CI, 1.03-1.13) and a slope of 0.65 (95% CI, 0.6-0.7), respectively. These values were then corrected for the systematic difference. When used forBland-Altman analysis, corrected values oftacrolimus and creatinine showed a bias of-0.1μg/L and 0.04mg/dL, respectively. Tacrolimus (corrected) and creatinine (corrected) microsampling values when compared to corresponding venipuncture values met median prediction error and median absolute percentage prediction error predefined acceptability limits of<15%. This study was conducted in a controlled environment using a trained nurse to collect VAMS samples. In this study, VAMS was used to reliably measured tacrolimus and creatinine. This represents a clear opportunity for more frequent and less invasive sampling for patients.

Advances and critical assessment of machine learning techniques for prediction of docking scores

AbstractHere we present three distinct machine learning (ML) approaches (TensorFlow, XGBoost, and SchNetPack) for docking score prediction. AutoDock Vina is used to evaluate the inhibitory potential of ZINC15 in‐vivo and in‐vitro‐only sets towards the SARS‐CoV‐2 main protease. The in‐vivo set (59 884 compounds) is used for ML training (max. 80%), validation (5%), and testing (15%). The in‐vitro‐only set (174 014 compounds) is used for the evaluation of prediction capability of the trained ML models. Contributions to the prediction error are analyzed with respect to compounds' charge, number of atoms, and expected inhibitory potential (docking score). Methods for the prediction error estimation of new compounds are considered, yet critically rejected. The ML input weighted with respect to the desired property (i.e., low docking score) in the machine learning models shows to be a promising option to improve the ML performance. Proposed models provide significant reduction in number of intriguing compounds that need to be investigated.

Wave Equation Modeling via Physics-Informed Neural Networks: Models of Soft and Hard Constraints for Initial and Boundary Conditions.

Therapeutic ultrasound waves are the main instruments used in many noninvasive clinical procedures. They are continuously transforming medical treatments through mechanical and thermal effects. To allow for effective and safe delivery of ultrasound waves, numerical modeling methods such as the Finite Difference Method (FDM) and the Finite Element Method (FEM) are used. However, modeling the acoustic wave equation can result in several computational complications. In this work, we study the accuracy of using Physics-Informed Neural Networks (PINNs) to solve the wave equation when applying different combinations of initial and boundary conditions (ICs and BCs) constraints. By exploiting the mesh-free nature of PINNs and their prediction speed, we specifically model the wave equation with a continuous time-dependent point source function. Four main models are designed and studied to monitor the effects of soft or hard constraints on the prediction accuracy and performance. The predicted solutions in all the models were compared to an FDM solution for prediction error estimation. The trials of this work reveal that the wave equation modeled by a PINN with soft IC and BC (soft-soft) constraints reflects the lowest prediction error among the four combinations of constraints.

Small area estimation under a semi‐parametric covariate measured with error

SummaryIn recent years, small area estimation has played an important role in statistics as it deals with the problem of obtaining reliable estimates for parameters of interest in areas with small or even zero sample sizes corresponding to population sizes. Nested error linear regression models are often used in small area estimation assuming that the covariates are measured without error and also the relationship between covariates and response variable is linear. Small area models have also been extended to the case in which a linear relationship may not hold, using penalised spline (P‐spline) regression, but assuming that the covariates are measured without error. Recently, a nested error regression model using a P‐spline regression model, for the fixed part of the model, has been studied assuming the presence of measurement error in covariate, in the Bayesian framework. In this paper, we propose a frequentist approach to study a semi‐parametric nested error regression model using P‐splines with a covariate measured with error. In particular, the pseudo‐empirical best predictors of small area means and their corresponding mean squared prediction error estimates are studied. Performance of the proposed approach is evaluated through a simulation and also by a real data application. We propose a frequentist approach to study a semi‐parametric nested error regression model using P‐splines with a covariate measured with error.

On the informativity of direct identification experiments in dynamical networks

Data informativity is a crucial property to ensure the consistency of the prediction error estimate. This property has thus been extensively studied in the open-loop and in the closed-loop cases, but has only been briefly touched upon in the dynamic network case. In this paper, we consider the prediction error identification of the modules in a row of a dynamic network using the full input approach. Our main contribution is to propose a number of easily verifiable data informativity conditions for this identification problem. Among these conditions, we distinguish a sufficient data informativity condition that can be verified based on the topology of the network and a necessary and sufficient data informativity condition that can be verified via a rank condition on a matrix of coefficients that are related to a full-order model structure of the network. These data informativity conditions allow to determine different situations (i.e., different excitation patterns) leading to data informativity. In order to be able to distinguish between these different situations, we also propose an optimal experiment design problem that allows to determine the excitation pattern yielding a certain pre-specified accuracy with the least excitation power.

Experimental evaluation and application of genetic programming to develop predictive correlations for hydrochar higher heating value and yield to optimize the energy content

The hydrothermal carbonization (HTC) process has been found to consistently improve biomass fuel characteristics by raising the higher heating value (HHV) of the hydrochar as process severity is increased. However, this is usually associated with a decrease in the solid yield (SY) of hydrochar, making it difficult to determine the optimal operating conditions to obtain the highest energy yield (EY), which combines the two parameters. In this study, a graph-based genetic programming (GP) method was used for developing correlations to predict HHV, SY, and EY for hydrochars based on published values from 42 biomasses and a broad range of HTC experimental systems and operating conditions, i.e., 5 ≤ holding time (min) ≤ 2208, 120 ≤ temperature (°C) ≤ 300, and 0. 0096 ≤ biomass to water ratio ≤ 0.5. In addition, experiments were carried out with 5 pomaces at 4 temperatures and two reactor scales, 1 L and 18.75 L. The correlations were evaluated using this experimental data set in order to estimate prediction errors in similar experimental systems. The use of the correlations to predict HTC conditions to achieve the maximum EY is demonstrated for three common feedstocks, wheat straw, sewage sludge, and a fruit pomace. The prediction was confirmed experimentally with pomace at the optimized HTC conditions; we observed 6.9 % error between the measured and predicted EY %. The results show that the correlations can be used to predict the optimal operating conditions to produce hydrochar with the desired fuel characteristics with a minimum of actual HTC runs.

Mitigation of the Inefficiency in Imbalance Settlement Designs Using Day-Ahead Prices

The design of electricity imbalance pricing mechanisms is internationally controversial. Policies on whether to permit virtual trading for market participants and whether, or how, to impose penalty incentives on the imbalance volumes vary widely. Furthermore, market designs vary depending whether the imbalance prices are obtained directly from real-time trading or based upon the offer and demand functions from the day-ahead energy markets. This paper develops an analytical framework for evaluating designs for imbalance settlement mechanisms and we have selected the Japanese electricity market, which has undergone several revisions in its imbalance mechanism, as a good example to assess such variations. We develop a predictive approach for the imbalance volumes and price densities using two-step quantile regressions and derive a new trading optimization for a virtual trader's arbitrage position. We construct supporting models to estimate prediction errors for renewable power and demand as drivers of imbalance volume. The empirical analysis reveals that even in a mechanism with imbalance penalties based upon day-ahead reference prices, virtual trading may still be beneficial to market participants as well as to the system operator. We also find that greater market transparency is crucial for increased benefits. The insights generalize beyond the Japan case study.

Peak ground acceleration prediction using artificial neural networks for Kachchh, Gujarat, India

The uncertainty in the empirical ground motion prediction models (GMMs) for any region depends on several parameters. In the present work, we apply an artificial neural network (ANN) to design a GMM of peak ground acceleration (PGA) for Kachchh, Gujarat, India, utilizing independent input parameters viz., moment magnitudes, hypocentral distances, focal depths and site proxy (in terms of average seismic shear-wave velocity from the surface to a depth of 30 m (Vs30)). The study has been performed using a PGA dataset consisting of eight engineering seismoscope records of the 2001 Mw7.7 Bhuj earthquake and 237 strong-motion records of 32 significant Bhuj aftershocks of Mw3.3–5.6 (during 2002–2008) with epicentral distances ranging from 1.0 to 288 km. We apply a feed-forward back propagation ANN method with 8 hidden nodes, which is found to be optimal for the selected PGA database and input–output mapping. The standard deviation of the error has been utilized to examine the performance of our model. We also test the ground motion predictability of our ANN model using real recordings of the 2001 Bhuj mainshock, two Mw5.6 Kachchh aftershocks and the 1999 Mw6.4 Chamoli mainshock. The standard deviation of PGA prediction error estimates in log10 units is found to be ± 0.2554. Also, the model predictability of our ANN model suggests a good prediction of the PGA for earthquakes of Mw5.6–7.7, which are occurring in Kachchh, Gujarat, India.

Dual counterstream architecture may support separation between vision and predictions

According to a predictive coding framework, visual processing involves the computation of prediction errors between sensory data and a generative model that is supplied via feedback projections. This implies that vision is cognitively penetrable by all sorts of top-down influences. In this paper, we review anatomical and functional data which suggest that feedforward and feedback projections are organized into two parallel processing streams: the supragranular and the infragranular counterstreams. The supragranular counterstream computes surface and motion representation in depth. It represents the best interpretation of what is given in the input image based on physical regularities that are built into this network. By contrast, the infragranular counterstream integrates vision with cognition, because it represents what is likely to be found in the environment based on the predictions derived from learned statistical regularities. The two counterstreams work in parallel, but independently of each other. They compete for dominance, and only one is allowed to deliver its output to higher-order areas at any instance of time. Such an arrangement allows the supragranular counterstream to remain cognitively impenetrable to top-down influences.

A VTA GABAergic computational model of dissociated reward prediction error computation in classical conditioning

Classical Conditioning is a fundamental learning mechanism where the Ventral Striatum is generally thought to be the source of inhibition to Ventral Tegmental Area (VTA) Dopamine neurons when a reward is expected. However, recent evidences point to a new candidate in VTA GABA encoding expectation for computing the reward prediction error in the VTA. In this system-level computational model, the VTA GABA signal is hypothesised to be a combination of magnitude and timing computed in the Peduncolopontine and Ventral Striatum respectively. This dissociation enables the model to explain recent results wherein Ventral Striatum lesions affected the temporal expectation of the reward but the magnitude of the reward was intact. This model also exhibits other features in classical conditioning namely, progressively decreasing firing for early rewards closer to the actual reward, twin peaks of VTA dopamine during training and cancellation of US dopamine after training.

A Model Guided Approach to Evoke Homogeneous Behavior During Temporal Reward and Loss Discounting.

BackgroundThe tendency to devaluate future options as a function of time, known as delay discounting, is associated with various factors such as psychiatric illness and personality. Under identical experimental conditions, individuals may therefore strongly differ in the degree to which they discount future options. In delay discounting tasks, this inter-individual variability inevitably results in an unequal number of discounted trials per subject, generating difficulties in linking delay discounting to psychophysiological and neural correlates. Many studies have therefore focused on assessing delay discounting adaptively. Here, we extend these approaches by developing an adaptive paradigm which aims at inducing more comparable and homogeneous discounting frequencies across participants on a dimensional scale.MethodThe proposed approach probabilistically links a (common) discounting function to behavior to obtain a probabilistic model, and then exploits the model to obtain a formal condition which defines how to construe experimental trials so as to induce any desired discounting probability. We first infer subject-level models on behavior on a non-adaptive delay discounting task and then use these models to generate adaptive trials designed to evoke graded relative discounting frequencies of 0.3, 0.5, and 0.7 in each participant. We further compare and evaluate common models in the field through out-of-sample prediction error estimates, to iteratively improve the trial-generating model and paradigm.ResultsThe developed paradigm successfully increases discounting behavior during both reward and loss discounting. Moreover, it evokes graded relative choice frequencies in line with model-based expectations (i.e., 0.3, 0.5, and 0.7) suggesting that we can successfully homogenize behavior. Our model comparison analyses indicate that hyperboloid models are superior in predicting unseen discounting behavior to more conventional hyperbolic and exponential models. We report out-of-sample error estimates as well as commonalities and differences between reward and loss discounting, demonstrating for instance lower discounting rates, as well as differences in delay perception in loss discounting.ConclusionThe present work proposes a model-based framework to evoke graded responses linked to cognitive function at a single subject level. Such a framework may be used in the future to measure cognitive functions on a dimensional rather than dichotomous scale.

On the multi-level embedding of crypto-image reversible data hiding

Crypto-space reversible image steganography has attracted increasing attention, given its ability to embed authentication information without revealing the image content. This paper presents an efficient reversible data hiding scheme for crypto-images: a block predictor is applied to compute prediction errors, then an adaptive block mapping algorithm is utilized to compress them whose amplitudes are within a small threshold, finally, this strategy can be applied in a multi-level manner to achieve a higher embedding capacity. Due to the correlations among adjacent pixels in the block, images can be sufficiently compressed to reserve abundant space for additional data embedding. Different from the prior arts, the compression code of the image is fully encrypted. Experimental results verify that the embedded data and original image can be perfectly recovered, the security is higher compared with the state-of-the-arts, and a significant improvement in the average embedding rate is achieved on two large-scale image datasets.

Evaluating the extent to which homeostatic plasticity learns to compute prediction errors in unstructured neuronal networks.

The brain is believed to operate in part by making predictions about sensory stimuli and encoding deviations from these predictions in the activity of "prediction error neurons." This principle defines the widely influential theory of predictive coding. The precise circuitry and plasticity mechanisms through which animals learn to compute and update their predictions are unknown. Homeostatic inhibitory synaptic plasticity is a promising mechanism for training neuronal networks to perform predictive coding. Homeostatic plasticity causes neurons to maintain a steady, baseline firing rate in response to inputs that closely match the inputs on which a network was trained, but firing rates can deviate away from this baseline in response to stimuli that are mismatched from training. We combine computer simulations and mathematical analysis systematically to test the extent to which randomly connected, unstructured networks compute prediction errors after training with homeostatic inhibitory synaptic plasticity. We find that homeostatic plasticity alone is sufficient for computing prediction errors for trivial time-constant stimuli, but not for more realistic time-varying stimuli. We use a mean-field theory of plastic networks to explain our findings and characterize the assumptions under which they apply.

Improved Small Domain Estimation via Compromise Regression Weights

Shrinkage estimates of small domain parameters typically use a combination of a noisy “direct” estimate that only uses data from a specific small domain and a more stable regression estimate. When the regression model is misspecified, estimation performance for the noisier domains can suffer due to substantial shrinkage toward a poorly estimated regression surface. In this article, we introduce a new class of robust, empirically-driven regression weights that target estimation of the small domain means under potential misspecification of the global regression model. Our regression weights are a convex combination of the model-based weights associated with the best linear unbiased predictor (BLUP) and those associated with the observed best predictor (OBP). The mixing parameter in this convex combination is found by minimizing a novel, unbiased estimate of the mean-squared prediction error for the small domain means, and we label the associated small domain estimates the “compromise best predictor,” or CBP. Using a data-adaptive mixture for the regression weights enables the CBP to preserve the robustness of the OBP while retaining the main advantages of the EBLUP whenever the regression model is correct. We demonstrate the use of the CBP in an application estimating gait speed in older adults. Supplementary materials for this article are available online.

Spatiotemporal distribution of labeled data can bias the validation and selection of supervised learning algorithms: A marine remote sensing example

Supervised learning allows the prediction of variables measured in-situ from variables that can be measured from satellites. A labeled data set for this purpose is typically created by matching in-situ and satellite data and split into subsets for model training and initial validation. However, the available data are often not randomly distributed in space and time. In theory, this can bias estimates of prediction errors. Here, remote sensing of chlorophyll a in the Baltic Sea serves as an example to demonstrate the importance of this problem in marine remote sensing, and to test how well different statistical designs for validation mitigate it. Semi-synthetic data sets were created by combining daily chlorophyll a fields from a biogeochemical model hindcast with real-world locations and times of in-situ measurements, generated by sampling 2,000 combinations of cruises from an oceanographic database. These data sets were matched with co-located satellite data and used to train and validate four algorithms using remote sensing reflectances as input. The algorithms were validated using different methods including random hold-out sets and various block cross-validation designs based on geographical location, time, or location in predictor space. The resulting error estimates were compared to true errors calculated from differences to the biogeochemical model outputs serving as response variable. All validation methods underestimated prediction errors, in many cases by >30%. While a simple band-ratio algorithm had the smallest true errors (e.g., absolute percentage difference: APD = 50%), estimated errors were smallest for more complicated and in fact less accurate machine learning algorithms. For example, ten-fold cross-validation led to selection of the truly best algorithm among four candidates for <10% of data sets. The biases were smallest, but not absent, for spatial block cross-validation, which selected the truly best algorithm for 21–46% of data sets, depending on the error measure. When the analyses were repeated with data that were randomly distributed in space and time, the biases of error estimates based on random splits became much smaller (e.g., 10-fold cross-validation estimated errors within 2% of their true values and selected the truly best algorithm for >99% of data sets), spatial block cross-validation overestimated prediction errors (often by >40%), all algorithms achieved lower true errors, and a random forest made the most accurate predictions overall (APD = 27%). These results show that more attention should be paid to statistical methods for estimating the errors of supervised learning algorithms, e.g., by using multiple validation methods in combination and critically discussing error estimates considering questions of dependence, representativeness, and stationarity. Furthermore, non-random spatiotemporal distribution of labeled data can be a barrier to harnessing the full potential of machine learning algorithms in marine remote sensing.

Ensemble of surrogates combining Kriging and Artificial Neural Networks for reliability analysis with local goodness measurement

In engineering problems, reliability assessment is often computationally expensive. Surrogate models with active learning approaches are commonly used to solve this problem. Kriging is one of the most popular surrogate models used in this domain. Recently, artificial neural networks (ANN) have also attracted a lot of interest for structural reliability assessment due to their powerful capability. Many active learning approaches have been developed based on Kriging models and ANN. But selecting an appropriate model or technique for a reliability assessment problem with limited knowledge of the limit state function remains a challenging task. Ensemble of surrogates seems to be a good approach to tackle this challenge. In this work, two active learning approaches are proposed to combine Kriging and ANN models for reliability analysis. One is the local best surrogate (LBS) approach and the other is the local weighted average surrogate (LWAS) approach. Cross-validation and Jackknife techniques are used to estimate prediction errors of the surrogate models. In addition, two methods are proposed to locally measure the goodness of the surrogate models and calculate the prediction errors of the ensemble of surrogates. The surrogate models are updated by selecting the new sample points that have large prediction errors and are close to the limit state. The efficiency and accuracy of the proposed approaches are demonstrated by 4 representative examples and two finite element problems. The results show that the proposed methods can be effective in evaluating the reliability of high dimension and rare event problems with less computational costs than the single typical surrogate model with active learning approaches (e.g. AK-MCS). Moreover, compared with the ensemble of surrogate models based on global goodness measurement, the proposed approaches also outperform in most cases. Finally, it should be mentioned that the proposed approaches are not only suitable for the combination of Kriging and ANN but also can be extended to other multiple surrogate models including support vector machine, polynomial chaos expansion, and so on.

Deception attacks on Kalman filtering with interval estimation performance loss

This paper studies the problem of optimal deception attacks against remote state estimation, where the measurement data is transmitted through an unreliable wireless channel. A malicious agent is capable of intercepting and modifying raw data, with the goal to cause the maximum estimation quality degradation and deceive χ2 anomaly detectors. Contrary to existing studies that focused on greedy attack performance, we consider a more general scenario that the attacker aims to maximize the summation of estimation errors in a fixed interval. It is shown that the information-based optimal attack is a linear combination of the minimum mean-square error estimates of all historical prediction errors. The combination coefficients can be obtained by solving a convex optimization problem. Moreover, the proposed attack can be generalized to deceive interval χ2 detectors with different lengths by slightly modifying the stealthiness constraint. The effectiveness of the proposed method is verified with numerical examples and comparative studies.

The Object According State Prediction to Diagnostic Data

The predicting methodology the state of the object based on diagnostic data is considered. With the selected parameter that determines the state of the object, it is measured in real time at a fixed sampling step. According to the measurement data, the value of this parameter is predicted in the future. This operation is implemented by an extrapolator of the l order - a l degree polynomial, built using the least squares method based on the previous measurements results. The changing process model of the diagnosed parameter is a random time function described by the stationary centered random component sum and a mathematical expectation deterministic change. The estimating prediction error method and the extrapolator parameters influence on its value are presented.