Numerical Standard Error Research Articles

Posterior-based Wald-type statistics for hypothesis testing

A new Wald-type statistic is proposed for hypothesis testing based on Bayesian posterior distributions under the correct model specification. The new statistic can be explained as a posterior version of the Wald statistic and has several nice properties. First, it is well-defined under improper prior distributions. Second, it avoids Jeffreys–Lindley–Bartlett’s paradox. Third, under the null hypothesis and repeated sampling, it follows a χ2 distribution asymptotically, offering an asymptotically pivotal test. Fourth, it only requires inverting the posterior covariance for parameters of interest. Fifth and perhaps most importantly, when a random sample from the posterior distribution (such as MCMC output) is available, the proposed statistic can be easily obtained as a by-product of posterior simulation. In addition, the numerical standard error of the estimated proposed statistic can be computed based on random samples. A robust version of the test statistic is developed under model misspecification and inherits many nice properties of the new posterior statistic. The finite sample performance of the statistics is examined in Monte Carlo studies. The method is applied to two latent variable models used in microeconometrics and financial econometrics.

Hierarchical Bayesian Estimation of Random Co-efficient Dynamic Panel Data Model

This research uses a Dynamic Panel Data Model (DPDM) to study how unobserved individual heterogeneity affects parameters of inference. The DPDM presumes correlation between lagged dependent variable and individual (unit) specific effects, resulting in heterogeneity among the units. The parameters of DPDM are usually estimated using the classical approach which has no control for heterogeneity of the error terms leading to non-consistent estimates of the parameters. Therefore, Hierarchical Bayesian Estimation (HBE) of unobserved individual heterogeneity of dynamic panel models was used in order to improve on a static panel model even for a panel with small, moderate, and large N. Three experiments for the individual (N) and time (T) were considered: (10, 15), (20, 20), and (100, 15). Theoretical findings are accompanied by extensive Markov Chain Monte Carlo experiments, which show that the estimator performs well and handled the complicated pattern exhibited by the data. The results show closed estimates of the parameters as the dimension of N>T. The performance of the HBE was assessed using Numerical Standard Error (NSE) as N>T produced the least NSE.

Partially censored posterior for robust and efficient risk evaluation

A novel approach to inference for a specific region of the predictive distribution is introduced. An important domain of application is accurate prediction of financial risk measures, where the area of interest is the left tail of the predictive density of logreturns. Our proposed approach originates from the Bayesian approach to parameter estimation and time series forecasting, however it is robust in the sense that it provides a more accurate estimation of the predictive density in the region of interest in case of misspecification. The first main contribution of the paper is the novel concept of the Partially Censored Posterior (PCP), where the set of model parameters is partitioned into two subsets: for the first subset of parameters we consider the standard marginal posterior, for the second subset of parameters (that are particularly related to the region of interest) we consider the conditional censored posterior. The censoring means that observations outside the region of interest are censored: for those observations only the probability of being outside the region of interest matters. This quasi-Bayesian approach yields more precise parameter estimation than a fully censored posterior for all parameters, and has more focus on the region of interest than a standard Bayesian approach. The second main contribution is that we introduce two novel methods for computationally efficient simulation: Conditional MitISEM, a Markov chain Monte Carlo method to simulate model parameters from the Partially Censored Posterior, and PCP-QERMit, an Importance Sampling method that is introduced to further decrease the numerical standard errors of the Value-at-Risk and Expected Shortfall estimators. The third main contribution is that we consider the effect of using a time-varying boundary of the region of interest. Extensive simulation and empirical studies show the ability of the introduced method to outperform standard approaches.

Bayesian inference for structural vector autoregressions identified by Markov-switching heteroskedasticity

In this study, Bayesian inference is developed for structural vector autoregressive models in which the structural parameters are identified via Markov-switching heteroskedasticity. In such a model, restrictions that are just-identifying in the homoskedastic case, become over-identifying and can be tested. A set of parametric restrictions is derived under which the structural matrix is globally or partially identified and a Savage–Dickey density ratio is used to assess the validity of the identification conditions. The latter is facilitated by analytical derivations that make the computations feasible and numerical standard errors small. As an empirical example, monetary models are compared using heteroskedasticity as an additional device for identification. The empirical results support an identified interest rate reaction function with money.

Partially Censored Posterior for Robust and Efficient Risk Evaluation

A novel approach to inference for a specific region of the predictive distribution is introduced. An important domain of application is accurate prediction of financial risk measures, where the area of interest is the left tail of the predictive density of logreturns. Our proposed approach originates from the Bayesian approach to parameter estimation and time series forecasting, however it is robust in the sense that it provides a more accurate estimation of the predictive density in the region of interest in case of misspecification. The first main contribution of the paper is the novel concept of the Partially Censored Posterior (PCP), where the set of model parameters is partitioned into two subsets: for the first subset of parameters we consider the standard marginal posterior, for the second subset of parameters (that are particularly related to the region of interest) we consider the conditional censored posterior. The censoring means that observations outside the region of interest are censored: for those observations only the probability of being outside the region of interest matters. This quasi-Bayesian approach yields more precise parameter estimation than a fully censored posterior for all parameters, and has more focus on the region of interest than a standard Bayesian approach. The second main contribution is that we introduce two novel methods for computationally efficient simulation: Conditional MitISEM, a Markov chain Monte Carlo method to simulate model parameters from the Partially Censored Posterior, and PCP-QERMit, an Importance Sampling method that is introduced to further decrease the numerical standard errors of the Value-at-Risk and Expected Shortfall estimators. The third main contribution is that we consider the effect of using a time-varying boundary of the region of interest, which may provide more information about the left tail of the distribution of the standardized innovations. Extensive simulation and empirical studies show the ability of the introduced method to outperform standard approaches.

Methods for Computing Numerical Standard Errors: Review and Application to Value-at-Risk Estimation

Abstract Numerical standard error (NSE) is an estimate of the standard deviation of a simulation result if the simulation experiment were to be repeated many times. We review standard methods for computing NSE and perform a Monte Carlo experiments to compare their performance in the case of high/extreme autocorrelation. In particular, we propose an application to risk management where we assess the precision of the value-at-risk measure when the underlying risk model is estimated by simulation-based methods. Overall, heteroscedasticity and autocorrelation estimators with prewhitening perform best in the presence of large/extreme autocorrelation.

Effects of varying substitution parameter (ρ) of the CES production function on the estimation methods: Bayesian and frequentist approaches

Intrinsically nonlinear models are models that cannot be made linear irrespective of the linearization method employed. Statisticians are often interested in estimating the parameters of nonlinear models but are faced with great difficulties since some nonlinear models cannot be solved analytically however, researchers have developed a way out of this difficulty using the Gauss-Newton Method via Kmenta approximation. This paper made use of classical and Bayesian approaches to estimate the Constant Elasticity of Substitution (CES) production function. The Metropolis-within Gibbs Algorithm was used to carry out the analysis as shown in the empirical illustrations and the result showed that the Numerical Standard Error (NSE) is minimal while the posterior estimates converged to the region of the true values making the Bayesian approach more preferred.

A Posterior-Based Wald-Type Statistic for Hypothesis Testing

A new Wald-type statistic is proposed for hypothesis testing based on Bayesian posterior distributions. The new statistic can be explained as a posterior version of Wald test and have several nice properties. First, it is well-defined under improper prior distributions. Second, it avoids Jeffreys-Lindley’s paradox. Third, under the null hypothesis and repeated sampling, it follows a χ2 distribution asymptotically, offering an asymptotically pivotal test. Fourth, it only requires inverting the posterior covariance for the parameters of interest. Fifth and perhaps most importantly, when a random sample from the posterior distribution (such as an MCMC output) is available, the proposed statistic can be easily obtained as a by-product of posterior simulation. In addition, the numerical standard error of the estimated proposed statistic can be computed based on the random sample. The finite sample performance of the statistic is examined in Monte Carlo studies. The method is applied to two latent variable models used in micro-econometrics and financial econometrics.

Nse: Computation of Numerical Standard Errors in R

NSE is an R package for computing the numerical standard error (NSE), an estimate of the standard deviation of a simulation result if the simulation experiment were to be repeated many times. The package provides a set of wrappers around several R packages, which give access to more than thirty estimators, including batch means estimators (Geyer (1992 Section 3.2), initial sequence estimators (Geyer (1992 Equation 3.3), spectrum at zero estimators (Heidelberger and Welch (1981),Flegal and Jones (2010), heteroskedasticity and autocorrelation consistent (HAC) kernel estimators (Newey and West (1987), Andrews (1991), Andrews and Monahan (1992), Newey and West (1994), Hirukawa (2010), and bootstrap estimators Politis and Romano (1992), Politis and Romano (1994), Politis and White (2004).

Comparison of Multiple Methods for Computing Numerical Standard Errors: An Extensive Monte Carlo Study

We perform extensive Monte Carlo experiments where we compare the performance of multiple methods for computing numerical standard errors (NSEs). NSE is an estimate of the standard deviation of a simulation result, if the simulation experiment were to be repeated many times. All considered methods fail in case of extreme serial correlation. On the other hand, we do need either extreme serial correlation or a rather short time series (with large serial correlation) to see a substantial failure of the initial sequence estimators, the spectral density estimator and the HAC kernel estimators. The bootstrap methods and batch means estimators perform relatively poorly, whereas prewhitening does make the HAC kernel estimators substantially better in experiments with large or extreme serial correlation.

A Bayesian chi-squared test for hypothesis testing

A new Bayesian test statistic is proposed to test a point null hypothesis based on a quadratic loss. The proposed test statistic may be regarded as the Bayesian version of the Lagrange multiplier test. Its asymptotic distribution is obtained based on a set of regular conditions and follows a chi-squared distribution when the null hypothesis is correct. The new statistic has several important advantages that make it appealing in practical applications. First, it is well-defined under improper prior distributions. Second, it avoids Jeffrey–Lindley’s paradox. Third, it always takes a non-negative value and is relatively easy to compute, even for models with latent variables. Fourth, its numerical standard error is relatively easy to obtain. Finally, it is asymptotically pivotal and its threshold values can be obtained from the chi-squared distribution. The method is illustrated using some real examples in economics and finance.

Marginal Likelihood Estimation with the Cross-Entropy Method

We consider an adaptive importance sampling approach to estimating the marginal likelihood, a quantity that is fundamental in Bayesian model comparison and Bayesian model averaging. This approach is motivated by the difficulty of obtaining an accurate estimate through existing algorithms that use Markov chain Monte Carlo (MCMC) draws, where the draws are typically costly to obtain and highly correlated in high-dimensional settings. In contrast, we use the cross-entropy (CE) method, a versatile adaptive Monte Carlo algorithm originally developed for rare-event simulation. The main advantage of the importance sampling approach is that random samples can be obtained from some convenient density with little additional costs. As we are generating independent draws instead of correlated MCMC draws, the increase in simulation effort is much smaller should one wish to reduce the numerical standard error of the estimator. Moreover, the importance density derived via the CE method is grounded in information theory, and therefore, is in a well-defined sense optimal. We demonstrate the utility of the proposed approach by two empirical applications involving women's labor market participation and U.S. macroeconomic time series. In both applications, the proposed CE method compares favorably to existing estimators.

Fast computation of the deviance information criterion for latent variable models

The deviance information criterion (DIC) has been widely used for Bayesian model comparison. However, recent studies have cautioned against the use of certain variants of the DIC for comparing latent variable models. For example, it has been argued that the conditional DIC–based on the conditional likelihood obtained by conditioning on the latent variables–is sensitive to transformations of latent variables and distributions. Further, in a Monte Carlo study that compares various Poisson models, the conditional DIC almost always prefers an incorrect model. In contrast, the observed-data DIC–calculated using the observed-data likelihood obtained by integrating out the latent variables–seems to perform well. It is also the case that the conditional DIC based on the maximum a posteriori (MAP) estimate might not even exist, whereas the observed-data DIC does not suffer from this problem. In view of these considerations, fast algorithms for computing the observed-data DIC for a variety of high-dimensional latent variable models are developed. Through three empirical applications it is demonstrated that the observed-data DICs have much smaller numerical standard errors compared to the conditional DICs. The corresponding Matlab code is available upon request.

Adaptive Sequential Posterior Simulators for Massively Parallel Computing Environments

Massively parallel desktop computing capabilities now well within the reach of individual academics modify the environment for posterior simulation in fundamental and potentially quite advantageous ways. But to fully exploit these benefits algorithms that conform to parallel computing environments are needed. Sequential Monte Carlo comes very close to this ideal whereas other approaches like Markov chain Monte Carlo do not. This paper presents a sequential posterior simulator well suited to this computing environment. The simulator makes fewer analytical and programming demands on investigators, and is faster, more reliable and more complete than conventional posterior simulators. The paper extends existing sequential Monte Carlo methods and theory to provide a thorough and practical foundation for sequential posterior simulation that is well suited to massively parallel computing environments. It provides detailed recommendations on implementation, yielding an algorithm that requires only code for simulation from the prior and evaluation of prior and data densities and works well in a variety of applications representative of serious empirical work in economics and finance. The algorithm is robust to pathological posterior distributions, generates accurate marginal likelihood approximations, and provides estimates of numerical standard error and relative numerical efficiency intrinsically. The paper concludes with an application that illustrates the potential of these simulators for applied Bayesian inference.

Bayesian Inference for Logistic Regression Models using Sequential Posterior Simulation

The logistic specification has been used extensively in non-Bayesian statistics to model the dependence of discrete outcomes on the values of specified covariates. Because the likelihood function is globally weakly concave estimation by maximum likelihood is generally straightforward even in commonly arising applications with scores or hundreds of parameters. In contrast Bayesian inference has proven awkward, requiring normal approximations to the likelihood or specialized adaptations of existing Markov chain Monte Carlo and data augmentation methods. This paper approaches Bayesian inference in logistic models using recently developed generic sequential posterior simulaton (SPS) methods that require little more than the ability to evaluate the likelihood function. Compared with existing alternatives SPS is much simpler, and provides numerical standard errors and accurate approximations of marginal likelihoods as by-products. The SPS algorithm for Bayesian inference is amenable to massively parallel implementation, and when implemented using graphical processing units it is more efficient than existing alternatives. The paper demonstrates these points by means of several examples.

Marginal Likelihood Estimation with the Cross-Entropy Method

We consider an adaptive importance sampling approach to estimating the marginal likelihood, a quantity that is fundamental in Bayesian model comparison and Bayesian model averaging. This approach is motivated by the difficulty of obtaining an accurate estimate through existing algorithms that use Markov chain Monte Carlo (MCMC) draws, where the draws are typically costly to obtain and highly correlated in high-dimensional settings. In contrast, we use the cross-entropy (CE) method, a versatile adaptive Monte Carlo algorithm originally developed for rare-event simulation. The main advantage of the importance sampling approach is that random samples can be obtained from some convenient density with little additional costs. As we are generating independent draws instead of correlated MCMC draws, the increase in simulation effort is much smaller should one wish to reduce the numerical standard error of the estimator. Moreover, the importance density derived via the CE method is grounded in information theory, and therefore, is in a well-defined sense optimal. We demonstrate the utility of the proposed approach by two empirical applications involving women’s labor market participation and U.S. macroeconomic time series. In both applications the proposed CE method compares favorably to existing estimators.

A comparative study of Monte Carlo methods for efficient evaluation of marginal likelihood

Strategic choices for efficient and accurate evaluation of marginal likelihoods by means of Monte Carlo simulation methods are studied for the case of highly non-elliptical posterior distributions. A comparative analysis is presented of possible advantages and limitations of different simulation techniques; of possible choices of candidate distributions and choices of target or warped target distributions; and finally of numerical standard errors. The importance of a robust and flexible estimation strategy is demonstrated where the complete posterior distribution is explored. Given an appropriately yet quickly tuned adaptive candidate, straightforward importance sampling provides a computationally efficient estimator of the marginal likelihood (and a reliable and easily computed corresponding numerical standard error) in the cases investigated, which include a non-linear regression model and a mixture GARCH model. Warping the posterior density can lead to a further gain in efficiency, but it is more important that the posterior kernel be appropriately wrapped by the candidate distribution than that it is warped.

Bayesian forecasting of Value at Risk and Expected Shortfall using adaptive importance sampling

An efficient and accurate approach is proposed for forecasting the Value at Risk (VaR) and Expected Shortfall (ES) measures in a Bayesian framework. This consists of a new adaptive importance sampling method for the Quick Evaluation of Risk using Mixture of t approximations (QERMit). As a first step, the optimal importance density is approximated, after which multi-step ‘high loss’ scenarios are efficiently generated. Numerical standard errors are compared in simple illustrations and in an empirical GARCH model with Student-t errors for daily S&P 500 returns. The results indicate that the proposed QERMit approach outperforms alternative approaches, in the sense that it produces more accurate VaR and ES estimates given the same amount of computing time, or, equivalently, that it requires less computing time for the same numerical accuracy.

Irregular Identification, Support Conditions, and Inverse Weight Estimation

In weighted moment condition models, we show a subtle link between identification and estimability that limits the practical usefulness of estimators based on these models. In particular, if it is necessary for (point) identification that the weights take arbitrarily large values, then the parameter of interest, though point identified, cannot be estimated at the regular (parametric) rate and is said to be irregularly identified. This rate depends on relative tail conditions and can be as slow in some examples as n−1/4. This nonstandard rate of convergence can lead to numerical instability and/or large standard errors. We examine two weighted model examples: (i) the binary response model under mean restriction introduced by Lewbel (1997) and further generalized to cover endogeneity and selection, where the estimator in this class of models is weighted by the density of a special regressor, and (ii) the treatment effect model under exogenous selection (Rosenbaum and Rubin (1983)), where the resulting estimator of the average treatment effect is one that is weighted by a variant of the propensity score. Without strong relative support conditions, these models, similar to well known “identified at infinity” models, lead to estimators that converge at slower than parametric rate, since essentially, to ensure point identification, one requires some variables to take values on sets with arbitrarily small probabilities, or thin sets. For the two models above, we derive some rates of convergence and propose that one conducts inference using rate adaptive procedures that are analogous to Andrews and Schafgans (1998) for the sample selection model.

Bayesian Forecasting of Value at Risk and Expected Shortfall Using Adaptive Importance Sampling

An efficient and accurate approach is proposed for forecasting Value at Risk [VaR] and Expected Shortfall [ES] measures in a Bayesian framework. This consists of a new adaptive importance sampling method for Quantile Estimation via Rapid Mixture of t approximations [QERMit]. As a first step the optimal importance density is approximated, after which multi-step 'high loss' scenarios are efficiently generated. Numerical standard errors are compared in simple illustrations and in an empirical GARCH model with Student-t errors for daily S&P 500 returns. The results indicate that the proposed QERMit approach outperforms several alternative approaches in the sense of more accurate VaR and ES estimates given the same amount of computing time, or equivalently requiring less computing time for the same numerical accuracy.