Gaussian Error Model Research Articles

Fault detection for district heating substations: Beyond three-sigma approaches

The topic of this paper is fault detection for district heating substations, which is an important enabler for the transition towards fourth-generation district heating systems. Classical fault detection approaches are often based on anomaly detection, commonly making the implicit assumption that the errors between the measurements and the predictions made by the baseline model are i.i.d. and following an underlying Gaussian distribution. Our analysis shows that this does not hold up in the field, showing clear seasonality in the error over time. We propose to replace the Gaussian error model by a quantile regression model in order to provide a more nuanced fault threshold, conditioned on time and other input variables. Additionally, we observed that properly training the baseline model comes with its own challenges due to this time dependency, which we propose to resolve by employing an ensemble of models, trained on different periods of time. We demonstrate our method on unlabelled operational data obtained from a Swedish district heating operator to illustrate its use in the field. In addition, we validate it on labelled data from our residential lab setup, testing a variety of common faults.

State updating in Xin’anjiang model by Asynchronous Ensemble Kalman filtering with enhanced error models

For flood simulation in humid catchments, utilizing discharge observations to update the states of hydrological models may enhance performance. Asynchronous Ensemble Kalman Filter (AEnKF), an asynchronous variant of the Ensemble Kalman Filter (EnKF), holds substantial application potential in hydrological assimilation due to its ability to utilize more observations with almost no additional computational time. This study employs AEnKF to update the state variables of the Xin’anjiang model, necessitating the use of error models to perturb both model states and observations to generate ensemble spread. The Bias-corrected Gaussian Error Model (BGEM) is used to mitigate the systematic bias brought by perturbating soil moisture, and the Maximum a Posteriori Estimation Method (MAP) is employed for the estimation of hyperparameters of error models. Through synthetic and real-world data testing, it has been validated that the rectification of soil moisture perturbations using the BGEM significantly reduces the systematic bias induced by Gaussian perturbations. Moreover, the assimilation scheme introduced in this study, based on AEnKF with enhanced error models, outperforms the EnKF with those models. It substantially reduces the accumulation of past errors in the initial conditions at the start of the forecast, thereby aiding in elevating the accuracy of flood forecasting.

Modelling count data with partial differential equation models in biology

Partial differential equation (PDE) models are often used to study biological phenomena involving movement-birth–death processes, including ecological population dynamics and the invasion of populations of biological cells. Count data, by definition, is non-negative, and count data relating to biological populations is often bounded above by some carrying capacity that arises through biological competition for space or nutrients. Parameter estimation, parameter identifiability, and making model predictions usually involves working with a measurement error model that explicitly relating experimental measurements with the solution of a mathematical model. In many biological applications, a typical approach is to assume the data are normally distributed about the solution of the mathematical model. Despite the widespread use of the standard additive Gaussian measurement error model, the assumptions inherent in this approach are rarely explicitly considered or compared with other options. Here, we interpret scratch assay data, involving migration, proliferation and delays in a population of cancer cells using a reaction–diffusion PDE model. We consider relating experimental measurements to the PDE solution using a standard additive Gaussian measurement error model alongside a comparison to a more biologically realistic binomial measurement error model. While estimates of model parameters are relatively insensitive to the choice of measurement error model, model predictions for data realisations are very sensitive. The standard additive Gaussian measurement error model leads to biologically inconsistent predictions, such as negative counts and counts that exceed the carrying capacity across a relatively large spatial region within the experiment. Furthermore, the standard additive Gaussian measurement error model requires estimating an additional parameter compared to the binomial measurement error model. In contrast, the binomial measurement error model leads to biologically plausible predictions and is simpler to implement. We provide open source Julia software on GitHub to replicate all calculations in this work, and we explain how to generalise our approach to deal with coupled PDE models with several dependent variables through a multinomial measurement error model, as well as pointing out other potential generalisations by linking our work with established practices in the field of generalised linear models.

Incorporating Measurement Error in Astronomical Object Classification

Most general-purpose classification methods, such as support-vector machine (SVM) and random forest (RF), fail to account for an unusual characteristic of astronomical data: known measurement error uncertainties. In astronomical data, this information is often given in the data but discarded because popular machine learning classifiers cannot incorporate it. We propose a simulation-based approach that incorporates heteroscedastic measurement error into an existing classification method to better quantify uncertainty in classification. The proposed method first simulates perturbed realizations of the data from a Bayesian posterior predictive distribution of a Gaussian measurement error model. Then, a chosen classifier is fit to each simulation. The variation across the simulations naturally reflects the uncertainty propagated from the measurement errors in both labeled and unlabeled data sets. We demonstrate the use of this approach via two numerical studies. The first is a thorough simulation study applying the proposed procedure to SVM and RF, which are well-known hard and soft classifiers, respectively. The second study is a realistic classification problem of identifying high-z (2.9 ≤ z ≤ 5.1) quasar candidates from photometric data. The data are from merged catalogs of the Sloan Digital Sky Survey, the Spitzer IRAC Equatorial Survey, and the Spitzer-HETDEX Exploratory Large-Area Survey. The proposed approach reveals that out of 11,847 high-z quasar candidates identified by a random forest without incorporating measurement error, 3146 are potential misclassifications with measurement error. Additionally, out of 1.85 million objects not identified as high-z quasars without measurement error, 936 can be considered new candidates with measurement error.

Gaussian process error modeling for chiral effective-field-theory calculations of np↔dγ at low energies

We calculate the energy-dependent cross section of the np↔dγ process in chiral effective field theory and apply state-of-the-art tools for quantification of theory uncertainty. We focus on the low-energy regime, where the magnetic dipole and the electric dipole transitions cross over, including the range relevant for big-bang nucleosynthesis. Working with the leading one- and two-body electromagnetic currents, we study the order-by-order convergence of this observable in the chiral expansion of the nuclear potential. We find that the Gaussian process error model describes the observed convergence very well, allowing us to present Bayesian credible intervals for the truncation error with correlations between the cross sections at different energies taken into account. We obtain a 1σ estimate of about 0.2% for the uncertainty from the truncation of the nuclear potential. This is an important step towards calculations with statistically interpretable uncertainties for astrophysical reactions involving light nuclei.

Quantifying bycatch risk factors for the Chinese distant water fishery

AbstractMitigating bycatch of non-target fish species is a common objective in fisheries management that may be supported by the gathering of data from fishery observer programs and quantitative analysis of bycatch risk factors. We build three GLM models based on Chinese tuna longline fishery observer data in terms of analysis of total bycatch rate (TB rate), total bycatch ratio (TB ratio), and species-specific bycatch rate, respectively. The positive log-linear models assumed a Gaussian observation error model and a linear combination of categorical independent variables, including area, year, month, depth, and bycatch species. Results show that distributions of TB rate and TB ratio followed different trends and a latitudinal decrease was observed from both the northern and southern hemisphere of the equator. Comprehensively, the Pacific is a better place to fish compared to the Indian and Atlantic Oceans in terms of relatively lower TB rate and TB ratio. Fishing in open oceans can somehow avoid a high TB ratio than fishing in coastal waters. As a result, we recommend area 2SW, 2SE, 2 NW, 14SW, 14SE, and 14 NW as appropriate fishing ground for Albacore (Thunnus alalunga) while area 11N, 11S, 18SW, and 18SE to be appropriate fishing ground for fishing bigeye tuna (Thunnus obesus). Setting fishing gears deeper than 500 m would also help to get a low TB rate and TB ratio.

Trajectory prediction of ballistic missiles using Gaussian process error model

Ballistic Missile Trajectory Prediction (BMTP) is critical to air defense systems. Most Trajectory Prediction (TP) methods focus on the coast and reentry phases, in which the Ballistic Missile (BM) trajectories are modeled as ellipses or the state components are propagated by the dynamic integral equations on time scales. In contrast, the boost-phase TP is more challenging because there are many unknown forces acting on the BM in this phase. To tackle this difficult problem, a novel BMTP method by using Gaussian Processes (GPs) is proposed in this paper. In particular, the GP is employed to train the prediction error model of the boost-phase trajectory database, in which the error refers to the difference between the true BM state at the prediction moment and the integral extrapolation of the BM state. And the final BMTP is a combination of the dynamic equation based numerical integration and the GP-based prediction error. Since the trained GP aims to capture the relationship between the numerical integration and the unknown error, the modified BM state prediction is closer to the true one compared with the original TP. Furthermore, the GP is able to output the uncertainty information of the TP, which is of great significance for determining the warning range centered on the predicted BM state. Simulation results show that the proposed method effectively improves the BMTP accuracy during the boost phase and provides reliable uncertainty estimation boundaries.

One-Step Deadbeat Control of a 5-Link Biped Using Data-Driven Nonlinear Approximation of the Step-to-Step Dynamics

For bipedal robots to walk over complex and constrained environments (e.g., narrow walkways, stepping stones), they have to meet precise control objectives of speed and foot placement at every single step. This control that achieves the objectives precisely at every step is known as one-step deadbeat control. The high dimensionality of bipedal systems and the under-actuation (number of joint exceeds the actuators) presents a formidable computational challenge to achieve real-time control. In this paper, we present a computationally efficient method for one-step deadbeat control and demonstrate it on a 5-link planar bipedal model with 1 degree of under-actuation. Our method uses computed torque control using the 4 actuated degrees of freedom to decouple and reduce the dimensionality of the stance phase dynamics to a single degree of freedom. This simplification ensures that the step-to-step dynamics are a single equation. Then using Monte Carlo sampling, we generate data for approximating the step-to-step dynamics followed by curve fitting using a control affine model and a Gaussian process error model. We use the control affine model to compute control inputs using feedback linearization and fine tune these using iterative learning control using the Gaussian process error enabling one-step deadbeat control. We demonstrate the approach in simulation in scenarios involving stabilization against perturbations, following a changing velocity reference, and precise foot placement. We conclude that computed torque control-based model reduction and sampling-based approximation of the step-to-step dynamics provides a computationally efficient approach for real-time one-step deadbeat control of complex bipedal systems.

Accounting for erroneous model structures in biokinetic process models

In engineering practice, model-based design requires not only a good process-based model, but also a good description of stochastic disturbances and measurement errors to learn credible parameter values from observations. However, typical methods use Gaussian error models, which often cannot describe the complex temporal patterns of residuals. Consequently, this results in overconfidence in the identified parameters and, in turn, optimistic reactor designs. In this work, we assess the strengths and weaknesses of a method to statistically describe these patterns with autocorrelated error models. This method produces increased widths of the credible prediction intervals following the inclusion of the bias term, in turn leading to more conservative design choices. However, we also show that the augmented error model is not a universal tool, as its application cannot guarantee the desired reliability of the resulting wastewater reactor design.

Glacial-cycle simulations of the Antarctic Ice Sheet with the Parallel Ice Sheet Model (PISM) – Part 2: Parameter ensemble analysis

Abstract. The Parallel Ice Sheet Model (PISM) is applied to the Antarctic Ice Sheet over the last two glacial cycles (≈210 000 years) with a resolution of 16 km. An ensemble of 256 model runs is analyzed in which four relevant model parameters have been systematically varied using full-factorial parameter sampling. Parameters and plausible parameter ranges have been identified in a companion paper (Albrecht et al., 2020) and are associated with ice dynamics, climatic forcing, basal sliding and bed deformation and represent distinct classes of model uncertainties. The model is scored against both modern and geologic data, including reconstructed grounding-line locations, elevation–age data, ice thickness, surface velocities and uplift rates. An aggregated score is computed for each ensemble member that measures the overall model–data misfit, including measurement uncertainty in terms of a Gaussian error model (Briggs and Tarasov, 2013). The statistical method used to analyze the ensemble simulation results follows closely the simple averaging method described in Pollard et al. (2016). This analysis reveals clusters of best-fit parameter combinations, and hence a likely range of relevant model and boundary parameters, rather than individual best-fit parameters. The ensemble of reconstructed histories of Antarctic Ice Sheet volumes provides a score-weighted likely range of sea-level contributions since the Last Glacial Maximum (LGM) of 9.4±4.1 m (or 6.5±2.0×106km3), which is at the upper range of most previous studies. The last deglaciation occurs in all ensemble simulations after around 12 000 years before present and hence after the meltwater pulse 1A (MWP1a). Our ensemble analysis also provides an estimate of parametric uncertainty bounds for the present-day state that can be used for PISM projections of future sea-level contributions from the Antarctic Ice Sheet.

A Bayesian inversion framework to evaluate parameter and predictive inference of a simple soil respiration model in a cool-temperate forest in western Japan

Careful modelling of soil carbon sequestration is essential to evaluate future terrestrial feedback to the earth climate system through atmosphere–surface carbon exchange. Few studies have evaluated, in bio- and geo-applications, parameter and predictive uncertainty of soil respiration models by considering the difference between observations and model predictions; i.e. residual error, which is assumed neither to be independent nor to be described by a normal (i.e. Gaussian) probability distribution with a mean of zero and constant variance. In this paper, we use 2-year observations of soil carbon flux from 2017 to 2018 (hereafter referred to as ‘long-term simulation’) obtained with two open-top chambers to estimate parameter and predictive uncertainty of a simple soil respiration model based on Bayesian statistics in a cool-temperate forest in western Japan. We also use a Gaussian innovative residual error model in which a generalised likelihood uncertainty estimation that accounts for correlated, heteroscedastic, non-normally distributed (i.e. non-Gaussian) residual error flexibly handles statistics varying in skewness and kurtosis. Results show that the effects of correlation and heteroscedasticity were eliminated adequately. Additionally, the posterior distribution of the residuals had a pattern intermediate to those of Gaussian and Laplacian (or double-exponential) distributions. Consequently, the predicted soil respiration rate, and range of uncertainty therein, well-matched the observational data. Furthermore, we compare results of parameter and predictive inference of the soil respiration model from the long-term simulation with those constrained of short-term simulations (i.e. 4-month subsets of the 2-year dataset) to determine the extent to which the approach used affects the estimation of parameter and predictive uncertainty. No significant difference in parameter estimates was found between the long-term simulation versus any of the short-term simulations, whereas short-term simulation analysis of the uncertainty at 50 %—i.e. between the lower (25 %) and upper (75 %) quartiles of the probability range—indicated distinctive variations in model parameters in summer when more vigorous activity of trees and organisms promotes carbon cycling between the atmosphere and ecosystem. Overall we demonstrate that the Bayesian inversion approach is useful as a means by which to evaluate effectively parameter and predictive uncertainty of a soil respiration model with precise representation of residual errors.

Bayesian approach to smoothing parameter selection in spline estimate for regression curve

Spline functions have proved to be very useful in statistics, in particular, to estimate the nonparametric regression. Many different smoothing parameter selectors for the smoothing spline are proposed in the literature such as cross-validation (CV), generalised cross-validation (GCV). In this article, we propose the Bayesian approach to estimate the smoothing parameter and the variance of the Gaussian error model in the context of the nonparametric regression. We use the Markov chain Monte Carlo (MCMC) method to compute the estimators given by the proposed Bayesian approach. The performance of the Bayesian approach is compared with the classical generalised cross-validation method through simulation and real data.

Using Generalized Gaussian Distributions to Improve Regression Error Modeling for Deep Learning-Based Speech Enhancement

From a statistical perspective, the conventional minimum mean squared error (MMSE) criterion can be considered as the maximum likelihood (ML) solution under an assumed homoscedastic Gaussian error model. However, in this paper, a statistical analysis reveals the super-Gaussian and heteroscedastic properties of the prediction errors in nonlinear regression deep neural network (DNN)-based speech enhancement when estimating clean log-power spectral (LPS) components at DNN outputs with noisy LPS features in DNN input vectors. Accordingly, we propose treating all dimensions of the prediction error vector as statistically independent random variables and model them with generalized Gaussian distributions (GGDs). Then, the objective function with the GGD error model is derived according to the ML criterion. Experiments on the TIMIT corpus corrupted by simulated additive noises show consistent improvements of our proposed DNN framework over the conventional DNN framework in terms of various objective quality measures under 14 unseen noise types evaluated and at various signal-to-noise ratio levels. Furthermore, the ML optimization objective with GGD outperforms the conventional MMSE criterion, achieving improved generalization and robustness.

Nonlinear Filtering Method Using a Switching Error Model for Outlier-Contaminated Observations

This article considers a nonlinear filtering method for handling outliers. The presence of outliers that are gross observation errors can greatly reduce the accuracy of filtering methods that assume Gaussian distributed errors. There are some existing methods that assume a Gaussian observation error, and estimate the error covariance matrix at each time step to avoid overfitting to the outliers. However, the estimates of the covariance matrix under such methods can become unstable. This results in underfitting or overfitting to observations when filtering. This paper presents a new method for handling outliers in nonlinear filtering problems by extending the unscented Kalman filter. In this method, two Gaussian observation error models with distinct covariance matrices are used: one for observations with regular errors, and another with a larger variance specified by a scale parameter for outliers. In addition to the system state, this method estimates an indicator variable that switches between the two models and the scale parameter for outliers. With the inclusion of the indicator variable and scale parameter, the estimated error covariance matrix can handle both regular observations and outliers appropriately at each time step. Furthermore, by estimating the scale parameter, the proposed method can be applied to a dataset without additional tuning to account for outlier characteristics. Through numerical experiments, we find that our method can estimate the system state better than existing methods for both datasets with and without outliers.

An Experiment on Autoregressive and Threshold Autoregressive Models with Non-Gaussian Error with Application to Realized Volatility

This article explores the fitting of Autoregressive (AR) and Threshold AR (TAR) models with a non-Gaussian error structure. This is motivated by the problem of finding a possible probabilistic model for the realized volatility. A Gamma random error is proposed to cater for the non-negativity of the realized volatility. With many good properties, such as consistency even for non-Gaussian errors, the maximum likelihood estimate is applied. Furthermore, a non-gradient numerical Nelder–Mead method for optimization and a penalty method, introduced for the non-negative constraint imposed by the Gamma distribution, are used. In the simulation experiments, the proposed fitting method found the true model with a rather insignificant bias and mean square error (MSE), given the true AR or TAR model. The AR and TAR models with Gamma random error are then tested on empirical realized volatility data of 30 stocks, where one third of the cases are fitted quite well, suggesting that the model may have potential as a supplement for current Gaussian random error models with proper adaptation.

Robust Beamforming Design in a NOMA Cognitive Radio Network Relying on SWIPT

This paper studies a multiple-input single-output non-orthogonal multiple access cognitive radio network relying on simultaneous wireless information and power transfer. A realistic non-linear energy harvesting model is applied and a power splitting architecture is adopted at each secondary user (SU). Since it is difficult to obtain perfect channel state information (CSI) in practice, instead either a bounded or Gaussian CSI error model is considered. Our robust beamforming and power splitting ratio are jointly designed for two problems with different objectives, namely, that of minimizing the transmission power of the cognitive base station and that of maximizing the total harvested energy of the SUs, respectively. The optimization problems are challenging to solve, mainly because of the non-linear structure of the energy harvesting and CSI errors models. We converted them into convex forms by using semi-definite relaxation. For the minimum transmission power problem, we obtain the rank-2 solution under the bounded CSI error model, while for the maximum energy harvesting problem, a two-loop procedure using a 1-D search is proposed. Our simulation results show that the proposed scheme significantly outperforms its traditional orthogonal multiple access counterpart. Furthermore, the performance using the Gaussian CSI error model is generally better than that using the bounded CSI error model.

Basis Pursuit Denoise With Nonsmooth Constraints

Level-set optimization formulations with data-driven constraints minimize a regularization functional subject to matching observations to a given error level. These formulations are widely used, particularly for matrix completion and sparsity promotion in data interpolation and denoising. The misfit level is typically measured in the ℓ <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sub> norm, or other smooth metrics. In this paper, we present a new flexible algorithmic framework that targets nonsmooth level-set constraints, including ℓ <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sub> , ℓ <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">∞</sub> , and even ℓ <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">0</sub> norms. These constraints give greater flexibility for modeling deviations in observation and denoising, and have significant impact on the solution. Measuring error in the ℓ <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sub> and ℓ <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">0</sub> norms makes the result more robust to large outliers, while matching many observations exactly. We demonstrate the approach for basis pursuit denoise (BPDN) problems as well as for extensions of BPDN to matrix factorization, with applications to interpolation and denoising of 4D seismic data. The new methods are particularly promising for seismic applications, where the amplitude in the data varies significantly, and measurement noise in low-amplitude regions can wreak havoc for standard Gaussian error models.

Bayesian calibration of terrestrial ecosystem models: a study of advanced Markov chain Monte Carlo methods

Abstract. Calibration of terrestrial ecosystem models is important but challenging. Bayesian inference implemented by Markov chain Monte Carlo (MCMC) sampling provides a comprehensive framework to estimate model parameters and associated uncertainties using their posterior distributions. The effectiveness and efficiency of the method strongly depend on the MCMC algorithm used. In this work, a differential evolution adaptive Metropolis (DREAM) algorithm is used to estimate posterior distributions of 21 parameters for the data assimilation linked ecosystem carbon (DALEC) model using 14 years of daily net ecosystem exchange data collected at the Harvard Forest Environmental Measurement Site eddy-flux tower. The calibration of DREAM results in a better model fit and predictive performance compared to the popular adaptive Metropolis (AM) scheme. Moreover, DREAM indicates that two parameters controlling autumn phenology have multiple modes in their posterior distributions while AM only identifies one mode. The application suggests that DREAM is very suitable to calibrate complex terrestrial ecosystem models, where the uncertain parameter size is usually large and existence of local optima is always a concern. In addition, this effort justifies the assumptions of the error model used in Bayesian calibration according to the residual analysis. The result indicates that a heteroscedastic, correlated, Gaussian error model is appropriate for the problem, and the consequent constructed likelihood function can alleviate the underestimation of parameter uncertainty that is usually caused by using uncorrelated error models.

Trajectory evaluation for manipulators with motion and sensor uncertainties

PurposeUncertainty can arise for a manipulator because its motion can deviate unpredictably from the assumed dynamical model and because sensors might provide information regarding the system state that is imperfect because of noise and imprecise measurement. This paper aims to propose a method to estimate the probable error ranges of the entire trajectory for a manipulator with motion and sensor uncertainties. The aims are to evaluate whether a manipulator can safely avoid all obstacles under uncertain conditions and to determine the probability that the end effector arrives at its goal area.Design/methodology/approachAn effective, analytical method is presented to evaluate the trajectory error correctly, and a motion plan was executed using Gaussian models by considering sensor and motion uncertainties. The method used an integrated algorithm that combined a Gaussian error model with an extended Kalman filter and a linear–quadratic regulator. Iterative linearization of the nonlinear dynamics was used around every section of the trajectory to derive all of the prior probability distributions before execution.FindingsSimulation and experimental results indicate that the proposed trajectory planning method based on the motion and sensor uncertainties is indeed highly convenient and efficient.Originality/valueThe proposed approach is applicable to manipulators with motion and sensor uncertainties. It helps determine the error distribution of the predefined trajectory. Based on the evaluation results, the most appropriate trajectory can be selected among many predefined trajectories according to the error ranges and the probability of arriving at the goal area. The method has been successfully applied to a manipulator operating on the Chinese Space Station.

Uncertainty quantification under group sparsity

SummaryQuantifying the uncertainty in penalized regression under group sparsity is an important open question. We establish, under a high-dimensional scaling, the asymptotic validity of a modified parametric bootstrap method for the group lasso, assuming a Gaussian error model and mild conditions on the design matrix and the true coefficients. Simulation of bootstrap samples provides simultaneous inferences on large groups of coefficients. Through extensive numerical comparisons, we demonstrate that our bootstrap method performs much better than popular competitors, highlighting its practical utility. The theoretical results generalize to other block norm penalization and sub-Gaussian errors, which further broadens the potential applications.