Small Sample Performance Research Articles

Inference without smoothing for large panels with cross-sectional and temporal dependence

This paper addresses inference in large panel data models in the presence of both cross-sectional and temporal dependence of unknown form. We are interested in making inferences that do not rely on the choice of any smoothing parameter as is the case with the often employed “HAC” estimator for the covariance matrix. To that end, we propose a cluster estimator for the asymptotic covariance of the estimators and valid bootstrap schemes that do not require the selection of a bandwidth or smoothing parameter and accommodate the nonparametric nature of both temporal and cross-sectional dependence. Our approach is based on the observation that the spectral representation of the fixed effect panel data model is such that the errors become approximately temporally uncorrelated. Our proposed bootstrap schemes can be viewed as wild bootstraps in the frequency domain. We present some Monte Carlo simulations to shed some light on the small sample performance of our inferential procedure.

Detection of units with pervasive effects in large panel data models

The importance of units that influence a large number of other units in a network has become increasingly recognized in the literature. In this paper we propose a new method to detect such pervasive units by basing our analysis on unit-specific residual error variances subject to suitable adjustments due to the multiple testing issues involved. Accordingly, a sequential multiple testing (SMT) procedure is proposed, which allows identification of pervasive units (if any) without a priori knowledge of the interconnections amongst cross-section units or availability of a short list of candidate units to search over. The proposed method is applicable even if the cross-section dimension exceeds the time series dimension, and most importantly it could end up with none of the units selected as pervasive when this is in fact the case. The SMT procedure exhibits satisfactory small-sample performance in Monte Carlo simulations and compares well relative to existing approaches. We apply the SMT detection method to sectoral indices of U.S. industrial production, U.S. house price changes by states, and the rates of change of real GDP and real equity prices across the world’s largest economies.

Bayesian MIDAS penalized regressions: Estimation, selection, and prediction

We propose a new approach to mixed-frequency regressions in a high-dimensional environment that resorts to Group Lasso penalization and Bayesian estimation and inference. In particular, to improve the prediction properties of the model and its sparse recovery ability, we consider a Group Lasso with a spike-and-slab prior. Penalty hyper-parameters governing the model shrinkage are automatically tuned via an adaptive MCMC algorithm. We establish good frequentist asymptotic properties of the posterior prediction error, we recover the optimal posterior contraction rate, and we show optimality of the posterior predictive density. Simulations show that the proposed models have good selection and forecasting performance in small samples, even when the design matrix presents cross-correlation. When applied to forecasting U.S. GDP, our penalized regressions can outperform many strong competitors. Results suggest that financial variables may have some, although very limited, short-term predictive content.

Generalized linear mixed quantile regression with panel data.

A new generalized linear mixed quantile model for panel data is proposed. This proposed approach applies GEE with smoothed estimating functions, which leads to asymptotically equivalent estimation of the regression coefficients. Random effects are predicted by using the best linear unbiased predictors (BLUP) based on the Tweedie exponential dispersion distributions which cover a wide range of distributions, including those widely used ones, such as the normal distribution, Poisson distribution and gamma distribution. A Taylor expansion of the quantile estimating function is used to linearize the random effects in the quantile process. The parameter estimation is based on the Newton-Raphson iteration method. Our proposed quantile mixed model gives consistent estimates that have asymptotic normal distributions. Simulation studies are carried out to investigate the small sample performance of the proposed approach. As an illustration, the proposed method is applied to analyze the epilepsy data.

Estimating scale-invariant directed dependence of bivariate distributions

Asymmetry of dependence is an inherent property of bivariate probability distributions. Being symmetric, commonly used dependence measures such as Pearson’s r or Spearman’s ρ mask asymmetry and implicitly assume that a random variable Y is equally dependent on a random variable X as vice versa. A copula-based, hence scale-invariant dependence measure called ζ1 overcoming the just mentioned problem was introduced in 2011. ζ1 attains values in [0,1], it is 0 if, and only if X and Y are independent, and 1 if, and only if Y is a measurable function of X. Working with so-called empirical checkerboard copulas allows to construct an estimator ζ1n for ζ1 which is strongly consistent in full generality, i.e., without any smoothness assumptions on the underlying copula. The R-package qad (short for quantification of asymmetric dependence) containing the estimator ζ1n is used both, to perform a simulation study illustrating the small sample performance of the estimator as well as to estimate the directed dependence between some global climate variables as well as between world development indicators.

A Bayesian stochastic approximation method

Motivated by the goal of improving the efficiency of a sequential design with small sample size, we propose a novel Bayesian stochastic approximation method to estimate the root of a regression function. The method features adaptive local modeling and nonrecursive iteration. Consistency of the Bayes estimator is established. Simulation studies show its superiority in small-sample performance to Robbins–Monro type procedures. Extension to a version of generalized multivariate quantile is presented.

A New Transformed t-test for Skewed Data: A Goodness-of-fit Approach

A new transformed two-sample t-test has been proposed for testing equality of two population means for skewed distributions by means of a univariate normal goodness of fit to the combined sample. The small sample performance of the proposed test is compared with untransformed t-test and the non-parametric analogue of t-test via Wilcoxon rank sum test using real-life examples and simulation from skewed distributions with varying values of skewness, empirically. It reveals that the proposed new test is appropriate for estimating the level of significance and is more powerful than the untransformed t-test and the Wilcoxon rank sum test for skewed distributions.

On the robustness of the pooled CCE estimator

Among the existing estimators of factor-augmented regressions, the CCE approach is the most popular. A major reason for this popularity is the simplicity and good small-sample performance of the approach, making it very attractive from an empirical point of view. The main drawback is that most of the available asymptotic theory is based on quite restrictive assumptions, such as that the common factor component should be independent of the regressors. The present paper can be seen as a reaction to this. The purpose is to study the asymptotic properties of the pooled CCE estimator under more realistic conditions. In particular, the common factor component may be correlated with the regressors, and the true number of common factors, r, can be larger than the number of estimated factors, which in CCE is given by k+1, where k is the number of regressors. The main conclusion is that while the estimator is generally consistent, asymptotic normality can fail when r>k+1.

Statistical Inference Based on Accelerated Failure Time Models Under Model Misspecification and Small Samples

–Accelerated failure time (AFT) models give an intuitive estimator for survival data analysis, but there is a risk of model misspecification. As a serious problem of model misspecification, the test size is likely to be far from the nominal level. Many researchers provided asymptotic corrections of various test statistics under model misspecification. However, their corrected statistics do not have good performance in small samples; in particular, they cause an inflation of test size. Although, the Bartlett adjustment is a popular approach for small-sample correction of the likelihood ratio statistic under the null hypothesis, it is impossible to derive the adjustment factor analytically under model misspecification. In this article, we proposed a robust test to model misspecification in small samples. Our proposed method is based on the Bartlett adjustment and we used the nonparametric bootstrap method to estimate the adjustment factor. We applied the proposed method to the AFT models when the error distribution and/or mean structure are misspecified in small samples. Our simulation results showed that the test size for the proposed method was close to the nominal level, although the existing methods resulted in substantial inflation of the test size. We illustrated our proposed method using two empirical examples.

Statistical Inference for the Inverted Scale Family under General Progressive Type-II Censoring

Two estimation problems are studied based on the general progressively censored samples, and the distributions from the inverted scale family (ISF) are considered as prospective life distributions. One is the exact interval estimation for the unknown parameter θ , which is achieved by constructing the pivotal quantity. Through Monte Carlo simulations, the average 90 % and 95 % confidence intervals are obtained, and the validity of the above interval estimation is illustrated with a numerical example. The other is the estimation of R = P ( Y < X ) in the case of ISF. The maximum likelihood estimator (MLE) as well as approximate maximum likelihood estimator (AMLE) is obtained, together with the corresponding R-symmetric asymptotic confidence intervals. With Bootstrap methods, we also propose two R-asymmetric confidence intervals, which have a good performance for small samples. Furthermore, assuming the scale parameters follow independent gamma priors, the Bayesian estimator as well as the HPD credible interval of R is thus acquired. Finally, we make an evaluation on the effectiveness of the proposed estimations through Monte Carlo simulations and provide an illustrative example of two real datasets.

Inference for Archimax copulas

Archimax copula models can account for any type of asymptotic dependence between extremes and at the same time capture joint risks at medium levels. An Archimax copula is characterized by two functional parameters: the stable tail dependence function $\ell $, and the Archimedean generator $\psi $ which distorts the extreme-value dependence structure. This article develops semiparametric inference for Archimax copulas: a nonparametric estimator of $\ell $ and a moment-based estimator of $\psi $ assuming the latter belongs to a parametric family. Conditions under which $\psi $ and $\ell $ are identifiable are derived. The asymptotic behavior of the estimators is then established under broad regularity conditions; performance in small samples is assessed through a comprehensive simulation study. The Archimax copula model with the Clayton generator is then used to analyze monthly rainfall maxima at three stations in French Brittany. The model is seen to fit the data very well, both in the lower and in the upper tail. The nonparametric estimator of $\ell $ reveals asymmetric extremal dependence between the stations, which reflects heavy precipitation patterns in the area. Technical proofs, simulation results and $\mathsf{R}$ code are provided in the Online Supplement.

An Augmented Anderson-Hsiao Estimator for Dynamic Short-T Panels

This paper introduces the idea of self-instrumenting endogenous regressors in settings when the correlation between these regressors and the errors can be derived and used to bias-correct the moment conditions. The resulting bias-corrected moment conditions are less likely to be subject to the weak instrument problem and can be used on their own and/or augmented with other available moment conditions (if any) to obtain more efficient estimators. This approach can be applied to estimation of a variety of models such as spatial and dynamic panel data models. This paper focuses on the latter, and proposes a new estimator for short-T dynamic panels by augmenting Anderson and Hsiao (AAH) estimator with bias-corrected quadratic moment conditions in first differences which substantially improve the small sample performance of the AH estimator without sacrificing on the generality of its underlying assumptions regarding the fixed effects, initial values, and heteroskedasticity of error terms. Using Monte Carlo experiments it is shown that the AAH estimator represents a substantial improvement over the AH estimator and more importantly it performs well even when compared to Arellano and Bond and Blundell and Bond (BB) estimators that are based on more restrictive assumptions, and continues to have satisfactory performance in cases where the standard GMM estimators are inconsistent. Finally, to decide between AAH and BB estimators, we also propose a Hausman type test which is shown to work well when T is small and n sufficiently large.

An Optimized Bayesian Hierarchical Two-Parameter Logistic Model for Small-Sample Item Calibration.

Accurate item calibration in models of item response theory (IRT) requires rather large samples. For instance, respondents are typically recommended for the two-parameter logistic (2PL) model. Hence, this model is considered a large-scale application, and its use in small-sample contexts is limited. Hierarchical Bayesian approaches are frequently proposed to reduce the sample size requirements of the 2PL. This study compared the small-sample performance of an optimized Bayesian hierarchical 2PL (H2PL) model to its standard inverse Wishart specification, its nonhierarchical counterpart, and both unweighted and weighted least squares estimators (ULSMV and WLSMV) in terms of sampling efficiency and accuracy of estimation of the item parameters and their variance components. To alleviate shortcomings of hierarchical models, the optimized H2PL (a) was reparametrized to simplify the sampling process, (b) a strategy was used to separate item parameter covariances and their variance components, and (c) the variance components were given Cauchy and exponential hyperprior distributions. Results show that when combining these elements in the optimized H2PL, accurate item parameter estimates and trait scores are obtained even in sample sizes as small as . This indicates that the 2PL can also be applied to smaller sample sizes encountered in practice. The results of this study are discussed in the context of a recently proposed multiple imputation method to account for item calibration error in trait estimation.

Estimation of parameters in random amplitude chirp signal

In this paper, we consider the random amplitude chirp signal model observed with independent and identically distributed additive error as proposed by Besson et al. [2]. The random amplitudes are also independent and identically distributed random variables with non-zero mean and some finite higher order moments. We consider the estimators proposed by Besson et al. [2] and study their theoretical properties. It is observed that the estimators are strongly consistent and asymptotically normally distributed. We propose a new multi-component random amplitude chirp model, which can be more useful in practice. Numerical experiments are conducted to see the small sample performances.

Bayesian nonparametric analysis of multivariate time series: A matrix Gamma Process approach

Many Bayesian nonparametric approaches to multivariate time series rely on Whittle’s Likelihood, involving the second order structure of a stationary time series by means of its spectral density matrix. In this work, we model the spectral density matrix by means of random measures that are constructed in such a way that positive definiteness is ensured. This is in line with existing approaches for the univariate case, where the normalized spectral density is modeled similar to a probability density, e.g. with a Dirichlet process mixture of Beta densities. We present a related approach for multivariate time series, with matrix-valued mixture weights induced by a Hermitian positive definite Gamma process. The latter has not been considered in the literature, allows to include prior knowledge and possesses a series representation that will be used in MCMC methods. We establish posterior consistency and contraction rates and small sample performance of the proposed procedure is shown in a simulation study and for real data.

A CLASS OF TWO-SAMPLE SCALE TESTS BASED ON U-STATISTICS

A class of tests based on U-statistic is proposed for two-sample scale problem. The U-statistic is function of extremes of subsamples taken from random samples of two absolutely continuous distributions. The asymptotic distribution, null distribution and efficacy of the class of tests are studied. A comparison of its performance with respect to other tests is studied in terms of Pitman ARE and small sample performance through its empirical power. Application of the class of tests is illustrated through an example.

Simultaneous inference for multiple marginal generalized estimating equation models.

Motivated by small-sample studies in ophthalmology and dermatology, we study the problem of simultaneous inference for multiple endpoints in the presence of repeated observations. We propose a framework in which a generalized estimating equation model is fit for each endpoint marginally, taking into account dependencies within the same subject. The asymptotic joint normality of the stacked vector of marginal estimating equations is used to derive Wald-type simultaneous confidence intervals and hypothesis tests for multiple linear contrasts of regression coefficients of the multiple marginal models. The small sample performance of this approach is improved by a bias adjustment to the estimate of the joint covariance matrix of the regression coefficients from multiple models. As a further small sample improvement a multivariate t-distribution with appropriate degrees of freedom is specified as reference distribution. In addition, a generalized score test based on the stacked estimating equations is derived. Simulation results show strong control of the family-wise type I error rate for these methods even with small sample sizes and increased power compared to a Bonferroni-Holm multiplicity adjustment. Thus, the proposed methods are suitable to efficiently use the information from repeated observations of multiple endpoints in small-sample studies.

Bayesian jackknife empirical likelihood

Summary Empirical likelihood is a very powerful nonparametric tool that does not require any distributional assumptions. Lazar (2003) showed that in Bayesian inference, if one replaces the usual likelihood with the empirical likelihood, then posterior inference is still valid when the functional of interest is a smooth function of the posterior mean. However, it is not clear whether similar conclusions can be obtained for parameters defined in terms of $U$-statistics. We propose the so-called Bayesian jackknife empirical likelihood, which replaces the likelihood component with the jackknife empirical likelihood. We show, both theoretically and empirically, the validity of the proposed method as a general tool for Bayesian inference. Empirical analysis shows that the small-sample performance of the proposed method is better than its frequentist counterpart. Analysis of a case-control study for pancreatic cancer is used to illustrate the new approach.

Estimating menarcheal age distribution from partially recalled data.

In a cross-sectional study, adolescent and young adult females were asked to recall the time of menarche, if experienced. Some respondents recalled the date exactly, some recalled only the month or the year of the event, and some were unable to recall anything. We consider estimation of the menarcheal age distribution from this interval-censored data. A complicated interplay between age-at-event and calendar time, together with the evident fact of memory fading with time, makes the censoring informative. We propose a model where the probabilities of various types of recall would depend on the time since menarche. For parametric estimation, we model these probabilities using multinomial regression function. Establishing consistency and asymptotic normality of the parametric maximum likelihood estimator requires a bit of tweaking of the standard asymptotic theory, as the data format varies from case to case. We also provide a non-parametric maximum likelihood estimator, propose a computationally simpler approximation, and establish the consistency of both these estimators under mild conditions. We study the small sample performance of the parametric and non-parametric estimators through Monte Carlo simulations. Moreover, we provide a graphical check of the assumption of the multinomial model for the recall probabilities, which appears to hold for the menarcheal data set. Our analysis shows that the use of the partially recalled part of the data indeed leads to smaller confidence intervals of the survival function.

Challenges and Opportunities in the Estimation of Dynamic Models

Interest in modeling longitudinal processes is increasing rapidly in organizational science. Organizational scholars often employ multilevel or hierarchical linear models (HLMs) to study such processes given that longitudinal data in organizational science typically consist of observations over a relatively small number of time intervals ( T) nested within a relatively large number of units ( N; e.g., people, teams, organizations). In this paper, we first distinguish change and dynamics as common research foci when modeling longitudinal processes and then demonstrate that a unique set of inferential hazards exists when investigating change or dynamics using multilevel models. Specifically, multilevel models that include one or more time-lagged values of the dependent variable as predictors often result in substantially biased estimates of the model parameters, inflated Type I error rates, and ultimately inaccurate inference. Using Monte Carlo simulations, we investigate the bias and Type I error rates for the standard centered/uncentered hierarchical linear model (HLM) and compare them with two alternative estimation methods: the Bollen and Brand structural equation modeling (SEM) approach and the Arrelano and Bond generalized method of moments using instrumental variables (GMM-IV) approach. We find that the commonly applied hierarchical linear model performs poorly, whereas the SEM and GMM-IV approaches generally perform well, with the SEM approach yielding slightly better performance in small samples with large autoregressive effects. We recommend the Bollen and Brand SEM approach for general use when studying change or dynamics in organizational science.