Finite Sample Inference Research Articles

A modern maximum-likelihood theory for high-dimensional logistic regression.

Students in statistics or data science usually learn early on that when the sample size n is large relative to the number of variables p, fitting a logistic model by the method of maximum likelihood produces estimates that are consistent and that there are well-known formulas that quantify the variability of these estimates which are used for the purpose of statistical inference. We are often told that these calculations are approximately valid if we have 5 to 10 observations per unknown parameter. This paper shows that this is far from the case, and consequently, inferences produced by common software packages are often unreliable. Consider a logistic model with independent features in which n and p become increasingly large in a fixed ratio. We prove that (i) the maximum-likelihood estimate (MLE) is biased, (ii) the variability of the MLE is far greater than classically estimated, and (iii) the likelihood-ratio test (LRT) is not distributed as a χ2 The bias of the MLE yields wrong predictions for the probability of a case based on observed values of the covariates. We present a theory, which provides explicit expressions for the asymptotic bias and variance of the MLE and the asymptotic distribution of the LRT. We empirically demonstrate that these results are accurate in finite samples. Our results depend only on a single measure of signal strength, which leads to concrete proposals for obtaining accurate inference in finite samples through the estimate of this measure.

Dividend Growth Predictability and the Price–Dividend Ratio

Asymptotic tests over-reject the null of no predictability in present-value models. We develop a nonparametric testing approach in state-space models, implying reliable finite sample inference under weak assumptions on price–dividend ratio and dividend shocks. We find sharp evidence of return predictability in postwar U.S. data but less consistent evidence of dividend predictability, which is significant only using cash-flow proxies reflecting information from mergers and acquisitions. These findings reconcile the diverging conclusions of present-value models and common predictive regressions in a way that is robust to the choice of the predictive variables, the sample period, and alternative cash-flow proxies. This paper was accepted by Gustavo Manso, finance.

Monte Carlo Inference on Two-Sided Matching Models

This paper considers two-sided matching models with nontransferable utilities, with one side having homogeneous preferences over the other side. When one observes only one or several large matchings, despite the large number of agents involved, asymptotic inference is difficult because the observed matching involves the preferences of all the agents on both sides in a complex way, and creates a complicated form of cross-sectional dependence across observed matches. When we assume that the observed matching is a consequence of a stable matching mechanism with homogeneous preferences on one side, and the preferences are drawn from a parametric distribution conditional on observables, the large observed matching follows a parametric distribution. This paper shows in such a situation how the method of Monte Carlo inference can be a viable option. Being a finite sample inference method, it does not require independence or local dependence among the observations which are often used to obtain asymptotic validity. Results from a Monte Carlo simulation study are presented and discussed.

Consistent Inference for Predictive Regressions in Persistent Economic Systems

We study standard predictive regressions in economic systems governed by persistent vector autoregressive dynamics for the state variables. In particular, all – or a subset – of the variables may be fractionally integrated, which induces a spurious regression problem. We propose a new inference and testing procedure – the Local speCtruM (LCM) approach – for joint significance of the regressors, that is robust against the variables having different integration orders and remains valid regardless of whether predictors are significant and if they induce cointegration. Specifically, the LCM procedure is based on fractional filtering and band-spectrum regression using a suitably selected set of frequency ordinates. Contrary to existing procedures, we establish a uniform Gaussian limit theory and a standard χ2-distributed test statistic. Using LCM inference and testing techniques, we explore predictive regressions for the realized return variation. Standard least squares inference indicates that popular financial and macroeconomic variables convey valuable information about future return volatility. In contrast, we find no significant evidence using our robust LCM procedure. If anything, our tests support a reverse chain of causality: rising financial volatility predates adverse innovations to macroeconomic variables. Simulations illustrate the relevance of the theoretical arguments for finite-sample inference. Institutional subscribers to the NBER working paper series, and residents of developing countries may download this paper without additional charge at www.nber.org.

Bayesian Estimation and Model Selection in Group-Based Trajectory Models

This paper makes three contributions to the literature on group-based trajectory models. First, although a small literature exists applying Bayesian methods to single trajectory models, their expositions are targeted toward biostatisticians. We bridge this gap by developing a model for normally distributed outcomes in congruence with standard social science references in order to facilitate more widespread adoption. Second, we extend this model to the first Bayesian dual trajectory model with a joint distribution specified over dual group memberships. Finally, we develop a Bayesian model averaging technique for efficient selection of the trajectories' functional forms. Model averaging provides additional flexibility in specifying trajectory shapes and saves researchers time by eliminating the need to fit functional forms separately and compare model fits. We also argue that the exact finite sample inference for model parameters and arbitrary functions thereof inherent in our Bayesian approach provides a compelling advantage over maximum likelihood methods.

Inference in Bayesian Proxy-SVARs

Motivated by the increasing use of external instruments to identify structural vector autoregressions (SVARs), we develop algorithms for exact finite sample inference in this class of time series models, commonly known as proxy-SVARs. Our algorithms make independent draws from the normal-generalized-normal family of conjugate posterior distributions over the structural parameterization of a proxy-SVAR. Importantly, our techniques can handle the case of set identification and hence they can be used to relax the additional exclusion restrictions unrelated to the external instruments often imposed to facilitate inference when more than one instrument are used to identify more than one equation, as in Mertens and Montiel-Olea (2018).

Nonparametric testing for smooth structural changes in panel data models

Detecting and modeling structural changes in time series models have attracted great attention. However, relatively little effort has been paid to the testing of structural changes in panel data models despite their increasing importance in economics and finance. In this paper, we propose a new approach to testing structural changes in panel data models. Unlike the bulk of the literature on structural changes, which focuses on detection of abrupt structural changes, we consider smooth structural changes for which model parameters are unknown deterministic smooth functions of time except for a finite number of time points. We use nonparametric local smoothing method to consistently estimate the smooth changing parameters and develop two consistent tests for smooth structural changes in panel data models. The first test is to check whether all model parameters are stable over time. The second test is to check potential time-varying interaction while allowing for a common trend. Both tests have an asymptotic N(0,1) distribution under the null hypothesis of parameter constancy and are consistent against a vast class of smooth structural changes as well as abrupt structural breaks with possibly unknown break points alternatives. Simulation studies show that the tests provide reliable inference in finite samples and two empirical examples with respect to a cross-country growth model and a capital structure model are discussed.

Bootstrap-based inferential improvements in beta autoregressive moving average model

We consider the issue of performing accurate small sample inference in beta autoregressive moving average model, which is useful for modeling and forecasting continuous variables that assume values in the interval (0, 1). The inferences based on conditional maximum likelihood estimation have good asymptotic properties, but their performances in small samples may be poor. This way, we propose bootstrap bias corrections of the point estimators and different bootstrap strategies for confidence interval improvements. Our Monte Carlo simulations show that finite sample inference based on bootstrap corrections is much more reliable than the usual inferences. We also presented an empirical application.

Inference in Regression Discontinuity Designs under Local Randomization

We introduce the rdlocrand package, which contains four commands to conduct finite-sample inference in regression discontinuity (RD) designs under a local randomization assumption, following the framework and methods proposed in Cattaneo, Frandsen, and Titiunik (2015, Journal of Causal Inference 3: 1–24) and Cattaneo, Titiunik, and Vazquez-Bare (2016, Working Paper, University of Michigan, http://www-personal.umich.edu/∼titiunik/papers/ CattaneoTitiunikVazquezBare2015_wp.pdf). Assuming a known assignment mechanism for units close to the RD cutoff, these functions implement a variety of procedures based on randomization inference techniques. First, the rdrandinf command uses randomization methods to conduct point estimation, hypothesis testing, and confidence interval estimation under different assumptions. Second, the rdwinselect command uses finite-sample methods to select a window near the cutoff where the assumption of randomized treatment assignment is most plausible. Third, the rdsensitivity command uses randomization techniques to conduct a sequence of hypothesis tests for different windows around the RD cutoff, which can be used to assess the sensitivity of the methods and to construct confidence intervals by inversion. Finally, the rdrbounds command implements Rosenbaum (2002, Observational Studies [Springer]) sensitivity bounds for the context of RD designs under local randomization. Companion R functions with the same syntax and capabilities are also provided.

Non parametric portmanteau tests for detecting non linearities in high dimensions

ABSTRACTWe propose two non parametric portmanteau test statistics for serial dependence in high dimensions using the correlation integral. One test depends on a cutoff threshold value, while the other test is freed of this dependence. Although these tests may each be viewed as variants of the classical Brock, Dechert, and Scheinkman (BDS) test statistic, they avoid some of the major weaknesses of this test. We establish consistency and asymptotic normality of both portmanteau tests. Using Monte Carlo simulations, we investigate the small sample properties of the tests for a variety of data generating processes with normally and uniformly distributed innovations. We show that asymptotic theory provides accurate inference in finite samples and for relatively high dimensions. This is followed by a power comparison with the BDS test, and with several rank-based extensions of the BDS tests that have recently been proposed in the literature. Two real data examples are provided to illustrate the use of the test procedure.

Likelihood Based Finite Sample Inference for Singly Imputed Synthetic Data Under the Multivariate Normal and Multiple Linear Regression Models

In this paper we develop likelihood-based finite sample inference based on singly imputed partially synthetic data, when the original data follow either a multivariate normal or a multiple linear regression model. We assume that the synthetic data are generated by using the plug-in sampling method, where unknown parameters in the data model are set equal to observed values of their point estimators based on the original data, and synthetic data are drawn from this estimated version of the model. Empirical studies are presented to show that the proposed methods do indeed perform as the theory predicts, and to compare the proposed methods for singly imputed synthetic data with the combining rules that are used to analyze multiply imputed partially synthetic data. Some theoretical comparisons between singly and multiply imputed partially synthetic data inference are also provided. A data analysis example and disclosure risk evaluation of singly and multiply imputed partially synthetic data is presented based on public use data from the Current Population Survey. We discuss the specific conditions under which the proposed methodology will yield valid inference, and evaluate the performance of the methodology when certain conditions do not hold. We outline some ways to extend the proposed methodology for certain scenarios where the required set of conditions do not hold.

A covariance correction that accounts for correlation estimation to improve finite-sample inference with generalized estimating equations: a study on its applicability with structured correlation matrices

ABSTRACTWhen generalized estimating equations (GEEs) incorporate an unstructured working correlation matrix, the variances of regression parameter estimates can inflate due to the estimation of the correlation parameters. In previous work, an approximation for this inflation that results in a corrected version of the sandwich formula for the covariance matrix of regression parameter estimates was derived. Use of this correction for correlation structure selection also reduces the over-selection of the unstructured working correlation matrix. In this manuscript, we conduct a simulation study to demonstrate that an increase in variances of regression parameter estimates can occur when GEE incorporates structured working correlation matrices as well. Correspondingly, we show the ability of the corrected version of the sandwich formula to improve the validity of inference and correlation structure selection. We also study the relative influences of two popular corrections to a different source of bias in the empirical sandwich covariance estimator.

Inference for Singly Imputed Synthetic Data Based on Posterior Predictive Sampling under Multivariate Normal and Multiple Linear Regression Models

Likelihood-based finite sample inference for singly imputed synthetic data generated via posterior predictive sampling is developed in this paper for multivariate normal and multiple linear regression models. Currently available methodology for drawing valid inference on population parameters using synthetic data is based on concepts of multiple imputation for missing data, and therefore requires the release of multiple synthetic datasets. The methodology developed in this paper demonstrates that, contrary to the usual belief, valid inference about meaningful model parameters can indeed be drawn based on a singly imputed synthetic dataset under the multivariate normal and multiple linear regression models, by fully utilizing the model structure.

DETECTING FOR SMOOTH STRUCTURAL CHANGES IN GARCH MODELS

Detecting and modeling structural changes in GARCH processes have attracted increasing attention in time series econometrics. In this paper, we propose a new approach to testing structural changes in GARCH models. The idea is to compare the log likelihood of a time-varying parameter GARCH model with that of a constant parameter GARCH model, where the time-varying GARCH parameters are estimated by a local quasi-maximum likelihood estimator (QMLE) and the constant GARCH parameters are estimated by a standard QMLE. The test does not require any prior information about the alternatives of structural changes. It has an asymptotic N(0,1) distribution under the null hypothesis of parameter constancy and is consistent against a vast class of smooth structural changes as well as abrupt structural breaks with possibly unknown break points. A consistent parametric bootstrap is employed to provide a reliable inference in finite samples and a simulation study highlights the merits of our test.

Bayesian Estimation and Clustering of Latent Attitudinal Parameters Using Cross-Sectional Survey Data: Application to an Online Banking Survey

Firms routinely collect cross-sectional survey data on attitudinal constructs such as satisfaction, brand awareness, behavioral intentions etc. from current and prospective customers. The single observation per individual in such survey data offers limited scope for estimation and inference of heterogeneous response parameters, and consequently, for downstream analyses that presuppose a heterogeneous response. We present a Bayesian treatment to this problem. We estimate these individual-specific latent attitudinal parameters, segment the respondents on the basis of these parameters, and finally, empirically validate our clustering solutions. Our approach considers both finite and infinite mixtures of component densities and enables exact finite sample inference on both individual level parameters and cluster level parameters. We examine the performance of the latent class, the mixture of normals and the semiparametric Bayesian estimators under varying conditions of latent data structure, cluster separation and individual-level idiosyncratic variance.

Efficient Factor Identification

Of the hundreds of published asset pricing anomalies, few have been properly tested to determine if they are admissible as true risk factors. Anomalies need not be risk factors, but they are often deployed in subsequent empirical tests as if they are. I leverage the equivalence between the linear factor (beta) model and the stochastic discount factor frameworks to show that the usual (Fama-MacBeth) statistical tests do not correspond to the null hypothesis of interest. These tests demonstrate only that the candidate risk factor is a viable trading strategy, in that it possesses a positive risk-adjusted return. I show that it is neither necessary nor sufficient for a risk factor to be a trading strategy, however. The sufficient statistic for whether a set of candidates are true risk factors is the multivariate analog of the Sharpe ratio, which accounts for the covariance structure of the factors. I present an MCMC procedure that efficiently estimates the multivariate Sharpe ratio and provides valid finite-sample inference.

Weak Inference for Dynamic Stochastic General Equilibrium Models with Time-Varying Parameters

This paper studies proper inference and asymptotically accurate structural break tests for parameters in Dynamic Stochastic General Equilibrium (DSGE) models in a maximum likelihood framework. Two empirically relevant issues may invalidate the conventional inference procedures and structural break tests for parameters in DSGE models: (i) weak identification and (ii) moderate parameter instability. DSGE literatures focus on dealing with weak identification issue, but ignore the impact of moderate parameter instability. This paper contributes to the literature via considering the joint impact of two issues in DSGE framework. The main results are: in a weakly identified DSGE model, (i) moderate instability from weakly identified parameters would not affect the validity of standard inference procedures or structural break tests; (ii) however, if strongly identified parameters are featured with moderate time-variation, the asymptotic distributions of test statistics would deviate from standard ones and would no longer be nuisance parameter free, which renders standard inference procedures and structural break tests invalid and provides practitioners misleading inference results; (iii) as long as I concentrate out strongly identified parameters, the instability impact of them would disappear as the sample size goes to infinity, which recovers the power of conventional inference procedure and structural break tests for weakly identified parameters. To illustrate my results, I simulate and estimate a modified version of the Hansen (1985) Real Business Cycle model and find that my theoretical results provide reasonable guidance for finite sample inference of the parameters in the model. I show that confidence intervals that incorporate weak identification and moderate parameter instability reduce the biases of confidence intervals that ignore those effects. While I focus on DSGE models in this paper, all of my theoretical results could be applied to any linear dynamic models or nonlinear GMM models.

Editors’ Introduction: Identification, Simulation and Finite-Sample Inference

Editors’ Introduction: Identification, Simulation and Finite-Sample Inference

Randomization Inference in the Regression Discontinuity Design: An Application to Party Advantages in the U.S. Senate

AbstractIn the Regression Discontinuity (RD) design, units are assigned a treatment based on whether their value of an observed covariate is above or below a fixed cutoff. Under the assumption that the distribution of potential confounders changes continuously around the cutoff, the discontinuous jump in the probability of treatment assignment can be used to identify the treatment effect. Although a recent strand of the RD literature advocates interpreting this design as a local randomized experiment, the standard approach to estimation and inference is based solely on continuity assumptions that do not justify this interpretation. In this article, we provide precise conditions in a randomization inference context under which this interpretation is directly justified and develop exact finite-sample inference procedures based on them. Our randomization inference framework is motivated by the observation that only a few observations might be available close enough to the threshold where local randomization is plausible, and hence standard large-sample procedures may be suspect. Our proposed methodology is intended as a complement and a robustness check to standard RD inference approaches. We illustrate our framework with a study of two measures of party-level advantage in U.S. Senate elections, where the number of close races is small and our framework is well suited for the empirical analysis.

Linear bootstrap methods for vector autoregressive moving-average models

We provide the theoretical justification of bootstrapping stationary invertible echelon vector autoregressive moving-average (VARMA) models using linear methods. The asymptotic validity of the bootstrap is established with strong white noise under parametric and nonparametric assumptions. Our methods are practical and useful for building reliable simulation-based inference and forecasting without implementing nonlinear estimation techniques such as ML which is usually burdensome, time demanding or impractical, particularly in big or highly persistent systems. The relevance of our procedures is more pronounced in the context of dynamic simulation-based techniques such as maximized Monte Carlo (MMC) tests [see Dufour J-M. Monte Carlo tests with nuisance parameters: a general approach to finite-sample inference and nonstandard asymptotics in econometrics. J Econom. 2006;133(2):443–477 and Dufour J-M, Jouini T. Finite-sample simulation-based tests in VAR models with applications to Granger causality testing. J Econom. 2006;135(1–2):229–254 for the VAR case]. Simulation evidence shows that, compared with conventional asymptotics, our bootstrap methods have good finite-sample properties in approximating the actual distribution of the studentized echelon VARMA parameter estimates, and in providing echelon parameter confidence sets with satisfactory coverage.