Gaussian Likelihood Research Articles

Celebrating the 25th International Conference of The International Environmetrics Society

Celebrating the 25th International Conference of The International Environmetrics Society

Comparing practice-ready forecast models for weekly and monthly fluctuations of average daily traffic and enhancing accuracy by weighting methods

Abstract Knowing daily traffic for the current year is recognized as being essential in many fields of transport analysis and practice, and short-term forecasting models offer a set of tools to meet these needs. This paper examines and compares the accuracy of three representative parametric and non-parametric prediction models, selected by the analysis of the numerous methods proposed in the literature for their good combination of forecast accuracy and ease of calibration, using real-life data on Italian motorway stretches. Non-parametric K-NN regression model, Gaussian maximum likelihood model and double seasonality Holt–Winters exponential smoothing model confirm their goodness to predict the weekly and monthly fluctuations of average daily traffic with varying degrees of performance, while maintaining an easy use in professional practice, i.e. requiring ordinary professional skills and conventional analysis tools. Since combining several prediction models can give, on average, more accuracy than that of the individual models, the paper compares two weighting methods of easy implementation and susceptible to a direct use, namely the widely used information entropy method and the less widespread Shapley value method. Despite being less common than the information entropy method, the Shapley value method proves to be more capable in better combining single forecasts and produces improvements in the predictions for test data. With these remarks, the paper might be of interest to traffic technicians or analysts, in various and not uncommon tasks they might find in their work.

Low-fidelity scale factor improves Bayesian multi-fidelity prediction by reducing bumpiness of discrepancy function

This study explores why the use of the low-fidelity scale factor can substantially improve the accuracy of the Bayesian multi-fidelity surrogate (MFS). It is shown analytically that the Bayesian MFS framework utilizes the scale factor to reduce the waviness and variation of the discrepancy function by maximizing the Gaussian process-based likelihood function. Less wavy functions are more accurately fitted, and variation reduction mitigates the effect of fitting error. Bumpiness is another way used to combine waviness and variation. Two examples, Borehole3 and Hartmann6, illustrated that indeed the Bayesian MFS reduced bumpiness using the scale factor. The finding may be useful for MFS using surrogates lacking uncertainty structure, so that likelihood is not an option, but bumpiness may be.

Rate-Energy Region in Wireless Information and Power Transfer: New Receiver Architecture and Practical Modulation

When simultaneous wireless information and power transfer is carried out, a fundamental tradeoff between achievable rate and harvested energy exists because the received power is used for two different purposes. The tradeoff is well characterized by the rate-energy region, and several techniques have been proposed to improve the achievable rate-energy region. However, the existing techniques still have a considerable loss in either energy or rate and thus the known achievable rate-energy regions are far from the ideal one. Deriving tight upper and lower bounds on the rate-energy region of our proposed scheme, we prove that the rate-energy region can be expanded almost to the ideal upper bound. Contrary to the existing techniques, in the proposed scheme, the information decoding circuit not only extracts amplitude and phase information but also combines the extracted information with the amplitude information obtained from the rectified signal. Consequently, the required energy for decoding can be minimized, and thus the proposed scheme achieves a near-optimal rate-energy region, which implies that the fundamental tradeoff in the achievable rate-energy region is nearly eliminated. To practically account for the theoretically achievable rate-energy region, we also present practical examples with an M-ary multi-level circular QAM with Gaussian maximum likelihood detection.

Bayesian Block Structure Sparse Based T–S Fuzzy Modeling for Dynamic Prediction of Hot Metal Silicon Content in the Blast Furnace

Since the hot metal silicon content simultaneously reflects the product quality and the thermal state of the blast furnace, its modeling is crucial and representative. In order to facilitate the realization of control, this paper proposes a Bayesian block structure sparse based Takagi–Sugeno (T–S) fuzzy modeling method, with which the main important fuzzy rules and the corresponding pivotal consequent parameters can be selected automatically to obtain a compact fuzzy model with good generalization performance. For being conjugate to the Gaussian likelihood that would lead to the associated Bayesian inference to be performed in closed form, a hierarchy of block structure sparse priori is adopted, and the variational Bayesian inference is used to solve it. The screening of model inputs and data processing appropriately consider the characteristics of the blast furnace process. The applicability and performance of the proposed method are demonstrated on no. 2 blast furnace of Liuzhou Steel in China.

Johansen’s Reduced Rank Estimator Is GMM

The generalized method of moments (GMM) estimator of the reduced-rank regression model is derived under the assumption of conditional homoscedasticity. It is shown that this GMM estimator is algebraically identical to the maximum likelihood estimator under normality developed by Johansen (1988). This includes the vector error correction model (VECM) of Engle and Granger. It is also shown that GMM tests for reduced rank (cointegration) are algebraically similar to the Gaussian likelihood ratio tests. This shows that normality is not necessary to motivate these estimators and tests.

Non-Gaussian quasi-likelihood estimation of SDE driven by locally stable Lévy process

We address estimation of parametric coefficients of a pure-jump Lévy driven univariate stochastic differential equation (SDE) model, which is observed at high frequency over a fixed time period. It is known from the previous study (Masuda, 2013) that adopting the conventional Gaussian quasi-maximum likelihood estimator then leads to an inconsistent estimator. In this paper, under the assumption that the driving Lévy process is locally stable, we extend the Gaussian framework into a non-Gaussian counterpart, by introducing a novel quasi-likelihood function formally based on the small-time stable approximation of the unknown transition density. The resulting estimator turns out to be asymptotically mixed normally distributed without ergodicity and finite moments for a wide range of the driving pure-jump Lévy processes, showing much better theoretical performance compared with the Gaussian quasi-maximum likelihood estimator. Extensive simulations are carried out to show good estimation accuracy. The case of large-time asymptotics under ergodicity is briefly mentioned as well, where we can deduce an analogous asymptotic normality result.

The skewed weak lensing likelihood: why biases arise, despite data and theory being sound

We derive the essentials of the skewed weak lensing likelihood via a simple Hierarchical Model. Our likelihood passes four objective and cosmology-independent tests which a standard Gaussian likelihood fails. We demonstrate that sound weak lensing analyses are naturally biased low, and this does not indicate any new physics such as deviations from $\Lambda$CDM. Mathematically, the biases arise because noisy two-point functions follow skewed distributions. This form of bias is already known from CMB analyses, where the low multipoles have asymmetric error bars. Weak lensing is more strongly affected by this asymmetry as galaxies form a discrete set of shear tracer particles, in contrast to a smooth shear field. We demonstrate that the biases can be up to 30 percent of the standard deviation per data point, dependent on the properties of the weak lensing survey. Our likelihood provides a versatile framework with which to address this bias in future weak lensing analyses.

Joint facies and rock properties Bayesian amplitude‐versus‐offset inversion using Markov random fields

ABSTRACTSeismic reflection pre‐stack angle gathers can be simultaneously inverted within a joint facies and elastic inversion framework using a hierarchical Bayesian model of elastic properties and categorical classes of rock and fluid properties. The Bayesian prior implicitly supplies low frequency information via a set of multivariate compaction trends for each rock and fluid type, combined with a Markov random field model of lithotypes, which carries abundance and continuity preferences. For the likelihood, we use a simultaneous, multi‐angle, convolutional model, which quantifies the data misfit probability using wavelets and noise levels inferred from well ties. Under Gaussian likelihood and facies‐conditional prior models, the posterior has simple analytic form, and the maximum a‐posteriori inversion problem boils down to a joint categorical/continuous non‐convex optimisation problem. To solve this, a set of alternative, increasingly comprehensive optimisation strategies is described: (i) an expectation–maximisation algorithm using belief propagation, (ii) a globalisation of method (i) using homotopy, and (iii) a discrete space approach using simulated annealing. We find that good‐quality inversion results depend on both sensible, elastically separable facies definitions, modest resolution ambitions, reasonably firm abundance and continuity parameters in the Markov random field, and suitable choice of algorithm. We suggest usually two to three, perhaps four, unknown facies per sample, and usage of the more expensive methods (homotopy or annealing) when the rock types are not strongly distinguished in acoustic impedance. Demonstrations of the technique on pre‐stack depth‐migrated field data from the Exmouth basin show promising agreements with lithological well data, including prediction accuracy improvements of 24% in and twofold in density, in comparison to a standard simultaneous inversion. Much clearer and extensive recovery of the thin Pyxis gas field was evident using stronger coupling in the Markov random field model and use of the homotopy or annealing algorithms.

Working correlation structure selection in generalized estimating equations

Selecting an appropriate correlation structure in analyzing longitudinal data can greatly improve the efficiency of parameter estimation, which leads to more reliable statistical inference. A number of such criteria have been proposed in the literature from different perspectives. However, little is known about the relative performance of these criteria. We review and evaluate these criteria by carrying out extensive simulation studies. Surprisingly, we find that the AIC and the BIC based on either the Gaussian working likelihood or the empirical likelihood outperform the others.

Structure-Adaptive Fuzzy Estimation for Random-Valued Impulse Noise Suppression

Noise detection accuracy is crucial in suppressing random-valued impulse noise. Both false and miss detections determine the final estimation performance. Deterministic detection methods, which distinctly classify pixels into noisy or uncorrupted pixels, tend to increase the estimation error because some uncorrupted edge points are hard to discriminate from the random-valued impulse noise points. This paper proposes an iterative structure-adaptive fuzzy estimation (SAFE) for random-valued impulse noise suppression. This SAFE method is developed in the framework of Gaussian maximum likelihood estimation. The structure-adaptive fuzziness is reflected by two structure-adaptive metrics based on pixel reliability and patch similarity, respectively. The reliability metric for each pixel (as noise free) is estimated via a novel-minimal-path-based structure propagation to give full consideration of the spatially varying image structures. A robust iteration stopping strategy is also proposed by evaluating the reestimation error of the uncorrupted intensity information. The comparative experimental results show that the proposed structure-adaptive fuzziness can lead to effective restoration. An efficient implementation of this SAFE method is also realized via graphics-processing-unit-based parallelization.

QML INFERENCE FOR VOLATILITY MODELS WITH COVARIATES

The asymptotic distribution of the Gaussian quasi-maximum likelihood estimator (QMLE) is obtained for a wide class of asymmetric GARCH models with exogenous covariates. The true value of the parameter is not restricted to belong to the interior of the parameter space, which allows us to derive tests for the significance of the parameters. In particular, the relevance of the exogenous variables can be assessed. The results are obtained without assuming that the innovations are independent, which allows conditioning on different information sets. Monte Carlo experiments and applications to financial series illustrate the asymptotic results. In particular, an empirical study demonstrates that the realized volatility can be a helpful covariate for predicting squared returns.

A general result on the estimation bias of ARMA models

A general result is derived on the finite-sample approximate bias of the Gaussian maximum likelihood estimator of the full vector of parameters in ARMA models when the error term may be nonnormally distributed and exogenous regressors may be included. It is found that there is typically estimation bias for parameters associated with the AR and MA terms and the bias depends only on these parameters themselves and the exogenous regressors. Parameters associated with the exogenous regressors are estimated approximately unbiased. The error variance is estimated with a downward bias, proportional to the number of exogenous regressors and the orders of AR and MA terms in the model. The distributional assumption on the error term does not affect the general bias result. Monte Carlo experiments are provided to illustrate the effectiveness of using analytical bias for the purpose of bias correction.

Comparing Structural Identification Methodologies for Fatigue Life Prediction of a Highway Bridge

Accurate measurement-data interpretation leads to increased understanding of structural behaviour and enhanced asset-management decision making. In this paper, four data-interpretation methodologies, residual minimization, traditional Bayesian model updating, modified Bayesian model updating (with an $L_\infty$-norm-based Gaussian likelihood function) and error-domain model falsification (EDMF), a method that rejects models that have unlikely differences between predictions and measurements, are compared. In the modified Bayesian model updating methodology, a correction is used in the likelihood function to account for the effect of a finite number of measurements on posterior probability-density functions. The application of these data-interpretation methodologies for condition assessment and fatigue-life prediction is illustrated on a highway steel-concrete composite bridge having four spans with a total length of 219m. A detailed 3D finite-element plate and beam model of the bridge and weigh-in-motion data are used to obtain the time-stress response at a fatigue critical location along the bridge span. The time stress-response, presented as a histogram, is compared to measured strain responses either to update prior knowledge of model parameters using residual minimization and Bayesian methodologies or to obtain candidate model instances using the EDMF methodology. It is concluded that the EDMF and modified Bayesian model updating methodologies provide robust prediction of fatigue-life compared with residual minimization and traditional Bayesian model updating in the presence of correlated non-Gaussian uncertainty. EDMF has additional advantages due to ease of understanding and applicability for practising engineers, thus enabling incremental asset-management decision making over long service lives. Finally, parallel implementations of EDMF using grid sampling have lower computations times than implementations using adaptive sampling.

Detecting Co-Movements in Noncausal Time Series

This paper introduces the notion of common noncausal features and proposes tools to detect them in multivariate time series models. We argue that the existence of co-movements might not be detected using the conventional stationary vector autoregressive (VAR) model as the common dynamics are present in the noncausal (i.e. forward-looking) component of the series. In particular, we show that the presence of a reduced rank structure allows to identify purely causal and noncausal VAR processes of order two and higher even in the Gaussian likelihood framework. Hence, usual test statistics and canonical correlation analysis can still be applied, where both lags and leads are used as instruments to determine whether the common features are present in either the backward- or forward-looking dynamics of the series. The proposed de finitions of co-movements also valid for the mixed causal-noncausal VAR, with the exception that an approximate non-Gaussian maximum likelihood estimator is necessary for these cases. This means however that one loses the bene fits of the simple tools proposed in this paper. An empirical analysis on European Brent and U.S. West Texas Intermediate oil prices illustrates the main fi ndings. Whereas we fail to fi nd any short run co-movements in a conventional causal VAR, they are detected in the growth rates of the series when considering a purely noncausal VAR.

A Frequency-Domain Approach to Dynamic Macroeconomic Models

This article is concerned with frequency-domain analysis of dynamic linear models under the hypothesis of rational expectations. We develop a unified framework for conveniently solving and estimating these models. Unlike existing strategies, our starting point is to obtain the model solution entirely in the frequency domain. This solution method is applicable to a wide class of models and permits straightforward construction of the spectral density for performing likelihood-based inference. To cope with potential model uncertainty, we also generalize the well-known spectral decomposition of the Gaussian likelihood function to a composite version implied by several competing models. Taken together, these techniques yield fresh insights into the model’s theoretical and empirical implications beyond what conventional time-domain approaches can offer. We illustrate the proposed framework using a prototypical new Keynesian model with fiscal details and two distinct monetary-fiscal policy regimes. The model is simple enough to deliver an analytical solution that makes the policy effects transparent under each regime, yet still able to shed light on the empirical interactions between U.S. monetary and fiscal policies along different frequencies.

Norm Discriminant Eigenspace Transform for Pattern Classification.

Most of the supervised dimensionality reduction (DR) methods design interclass scatter as the separability between the class means, which may force to assume unimodal Gaussian likelihoods and their projection space trends toward the class means. This paper presents a novel DR approach, norm discriminant eigenspace transform (NDET), in which average norms ( l2 ) of classes have been utilized to characterize the interclass separability and the within-class distance characterizes the intraclass compactness. NDET is intended to accommodate data distributions that may be multimodal and non-Gaussian. We derive an upper bound for NDET, and a specific solution space to attain this bound. Existence of the specific solution is very unwonted, thereby we have considered the solution space of upper bound to achieve better reduction of dimensionality and discrimination of classes. Also, a nonlinear version of NDET (kernel NDET) is developed to model nonlinear relationships between the features. We show, experimentally (on synthetic data) that NDET effectively overcomes the limitations, which arise due to unimodal and data distribution assumptions of the traditional algorithms. Extensive empirical studies are made; and the proposed method is compared with closely related state-of-the-art schemes on UCI machine learning repository and face recognition data sets, to establish its novelty.

Efficiency improvements for minimum distance estimation of causal and invertible ARMA models

In this note we analyze efficiency improvements over the Gaussian maximum likelihood (ML) estimator for frequency domain minimum distance (MD) estimation for causal and invertible autoregressive moving average (ARMA) models. The analysis complements Velasco and Lobato (2017) where optimal MD estimation, which employs information in higher order moments, is studied for the general possibly non causal or non-invertible case. We consider MD estimation that combines in two manners the information contained in second, third, and fourth moments. We show that for both MD estimators efficiency improvements over the Gaussian ML occur when the distribution of the innovations is platykurtic. In addition, we show that asymmetry alone is not associated with efficiency improvements.

Formula I(1) and I(2): Race Tracks for Likelihood Maximization Algorithms of I(1) and I(2) Cointegrated VAR Models

This paper provides some test cases, called circuits, for the evaluation of Gaussian likelihood maximization algorithms of the cointegrated vector autoregressive model. Both I(1) and I(2) models are considered. The performance of algorithms is compared first in terms of effectiveness, defined as the ability to find the overall maximum. The next step is to compare their efficiency and reliability across experiments. The aim of the paper is to commence a collective learning project by the profession on the actual properties of algorithms for cointegrated vector autoregressive model estimation, in order to improve their quality and, as a consequence, also the reliability of empirical research.

Recursive Computation for Block‐Nested Covariance Matrices

Covariance matrices corresponding to samples of multivariate time series or spatial random fields have a block‐Toeplitz structure that has a nested pattern. Also, non‐lattice data samples yield nested covariance matrices, although they are no longer block‐Toeplitz. The nested structure of such matrices facilitates the computation of their inverses, among other related quantities. Recursive algorithms, based upon this nested structure, are presented, yielding applications such as the simulation of vector time series, the evaluation of Gaussian likelihoods and Whittle likelihoods, the computation of spectral factorization, and the calculation of projections. Both multivariate time series applications and two‐dimensional random fields applications are discussed, as well as applications to non‐lattice data.