Annals Of Statistics Research Articles

Minimum profile Hellinger distance estimation for single-index models

Single-index models are the most commonly used covariate models and are widely used in statistical applications. Such models include linear models and generalised linear models as special cases. This article investigates efficient and robust estimates for single-index models. For this purpose, we employ the minimum distance approach which in general is automatically robust with respect to the stability of the quantity being estimated. In particular, the minimum Hellinger distance approach introduced by Beran [(1977), ‘Minimum Hellinger Distance Estimators for Parametric Models’, Annals of Statistics, 5, 445–463] produces estimators that are asymptotically efficient at the model density and simultaneously possess excellent robustness properties. In this paper, we construct a minimum profile Hellinger distance estimator (MPHDE) for single-index models. We prove the consistency of the proposed MPHDE and examine its finite-sample performance and robustness properties via Monte Carlo simulation studies and real data analysis.

Assessment of fit of the time-varying dynamic partial credit model using the posterior predictive model checking method.

Several new models based on item response theory have recently been suggested to analyse intensive longitudinal data. One of these new models is the time-varying dynamic partial credit model (TV-DPCM; Castro-Alvarez etal., Multivariate Behavioral Research, 2023, 1), which is a combination of the partial credit model and the time-varying autoregressive model. The model allows the study of the psychometric properties of the items and the modelling of nonlinear trends at the latent state level. However, there is a severe lack of tools to assess the fit of the TV-DPCM. In this paper, we propose and develop several test statistics and discrepancy measures based on the posterior predictive model checking (PPMC) method (PPMC; Rubin, The Annals of Statistics, 1984, 12, 1151) to assess the fit of the TV-DPCM. Simulated and empirical data are used to study the performance of and illustrate the effectiveness of the PPMC method.

Near-optimal estimation of linear functionals with log-concave observation errors

Abstract This note addresses the question of optimally estimating a linear functional of an object acquired through linear observations corrupted by random noise, where optimality pertains to a worst-case setting tied to a symmetric, convex and closed model set containing the object. It complements the article ‘Statistical Estimation and Optimal Recovery’ published in the Annals of Statistics in 1994. There, Donoho showed (among other things) that, for Gaussian noise, linear maps provide near-optimal estimation schemes relatively to a performance measure relevant in Statistical Estimation. Here, we advocate for a different performance measure arguably more relevant in Optimal Recovery. We show that, relatively to this new measure, linear maps still provide near-optimal estimation schemes even if the noise is merely log-concave. Our arguments, which make a connection to the deterministic noise situation and bypass properties specific to the Gaussian case, offer an alternative to parts of Donoho’s proof.

Editorial announcement

I am delighted to welcome Sam Astill, James Duffy and Liudas Giraitis to the editorial board of the Journal of Time Series Analysis. All three join as Associate Editors with effect from 1 March 2023. At the same time, I would like to thank Professor Konstantinos Fokianos, who steps down as an Associate Editor with effect from 1 March 2023, for his work for the journal in this capacity since 2013. Sam Astill is a Senior Lecturer in Econometrics at Essex Business School. His research interests include theoretical and applied time series econometrics and financial econometrics, in particular asset price bubble detection and predictive regression. He has published his research in Journal of Financial Econometrics, Journal of Time Series Analysis, The Econometrics Journal, among others. James Duffy is an Associate Professor in Economics at the University of Oxford. His research is focused on strongly dependent time series, particularly with respect to: non-parametric estimation and inference; nonlinear generalizations of cointegration; and the robustness of inferences to varying levels of persistence. He is also interested in problems of identification and inference in structural macroeconomic models. His research has been published in the Annals of Statistics, the Journal of Econometrics, Econometric Theory, among others. Liudas Giraitis is Professor of Econometrics at Queen Mary, University of London. His research interests cover long memory and integrated I(d) models, non-parametric methods for time series models with time-varying parameters, ARCH modeling, asymptotic theory for dependent variables and their statistical and econometric applications. He has published his research in Annals of Statistics, Annals of Probability, Journal of Time Series Analysis, Econometric Theory, Journal of Econometrics, among others.

Center-outward Rank- and Sign-based VARMA Portmanteau Tests: Chitturi, Hosking, and Li–McLeod revisited

The pseudo-Gaussian portmanteau tests of Chitturi, Hosking, and Li and McLeod for VARMA models are revisited from a Le Cam perspective, providing a precise and more rigorous description of the asymptotic behavior of the multivariate portmanteau test statistic, which depends on the dimension d of the observations, the number m of lags involved, and the length n of the observation period. Then, based on the concepts of center-outward ranks and signs recently developed (Hallin, del Barrio, Cuesta-Albertos, and Matrán, Annals of Statistics 49, 1139–1165, 2021), a class of multivariate rank- and sign-based portmanteau test statistics is proposed which, under the null hypothesis and under a broad family of innovation densities, can be approximated by an asymptotically chi-square variable. The asymptotic properties of these tests are derived; simulations demonstrate their advantages over their classical pseudo-Gaussian counterpart.

Renewable quantile regression for streaming data sets

Online updating is an important statistical method for the analysis of big data arriving in streams due to its ability to break the storage barrier and the computational barrier under certain circumstances. The quantile regression, as a widely used regression model in many fields, faces challenges in model fitting and variable selection with big data arriving in streams. Chen et al. (2019, Annals of Statistics) has proposed a quantile regression method for streaming data, but a strong additional condition is required. In this paper, renewable optimized objective functions for regression parameter estimation and variable selection in a quantile regression are proposed. The proposed methods are illustrated using current data and the summary statistics of historical data. Theoretically, the proposed statistics are shown to have the same asymptotic distributions as the standard version computed on an entire data stream with the data batches pooled into one data set, without additional condition. Both simulations and data analysis are conducted to illustrate the finite sample performance of the proposed methods.

Nonparametric inference for additive models estimated via simplified smooth backfitting

We investigate hypothesis testing in nonparametric additive models estimated using simplified smooth backfitting (Huang and Yu, Journal of Computational and Graphical Statistics, 28(2), 386–400, 2019). Simplified smooth backfitting achieves oracle properties under regularity conditions and provides closed-form expressions of the estimators that are useful for deriving asymptotic properties. We develop a generalized likelihood ratio (GLR) (Fan, Zhang and Zhang, Annals of statistics, 29(1),153–193, 2001) and a loss function (LF) (Hong and Lee, Annals of Statistics, 41(3), 1166–1203, 2013)-based testing framework for inference. Under the null hypothesis, both the GLR and LF tests have asymptotically rescaled chi-squared distributions, and both exhibit the Wilks phenomenon, which means the scaling constants and degrees of freedom are independent of nuisance parameters. These tests are asymptotically optimal in terms of rates of convergence for nonparametric hypothesis testing. Additionally, the bandwidths that are well suited for model estimation may be useful for testing. We show that in additive models, the LF test is asymptotically more powerful than the GLR test. We use simulations to demonstrate the Wilks phenomenon and the power of these proposed GLR and LF tests, and a real example to illustrate their usefulness.

Perturbation upper bounds for singular subspaces with a kind of heteroskedastic noise and its application in clustering

Perturbation bounds for singular subspaces have a wide range of applications, including high‐dimensional statistics, machine learning, and applied mathematics. This paper focuses on the perturbation bounds of singular subspaces under a signal matrix interfered by a special heteroscedastic noise matrix in which its same row or column shares similar variance. First of all, we extend homoscedastic results of T. Tony Cai and Anru Zhang (see Rate‐optimal perturbation bounds for singular subspaces with applications to high‐dimensional statistics, The Annals of Statistics, 2018, 46(1), 60–89) to heteroscedastic cases. Then we apply the developed tools to the heteroskedastic clustering model. We find out that our upper bound of clustering misclassification rate is better than the one of T. Tony Cai, Rungang Han, and Anru Zhang (see arXiv:2008.12434, 2020).

On sufficient dimension reduction via principal asymmetric least squares

Principal asymmetric least squares (PALS) is introduced as a novel method for sufficient dimension reduction with heteroscedastic error. Classical methods such as MAVE [Xia et al. (2002), ‘An Adaptive Estimation of Dimension Reduction Space’ (with discussion), Journal of the Royal Statistical Society Series B, 64, 363–410] and PSVM [Li et al. (2011), ‘Principal Support Vector Machines for Linear and Nonlinear Sufficient Dimension Reduction’, The Annals of Statistics, 39, 3182–3210] may not perform well in the presence of heteroscedasticity, while the new proposal addresses this limitation by synthesising different expectile levels. Through extensive numerical studies, we demonstrate the superior performance of PALS in terms of estimation accuracy over classical methods including MAVE and PSVM. For the asymptotic analysis of PALS, we develop new tools to compute the derivative of an expectation of a non-Lipschitz function.

Testing the volatility jumps based on the high frequency data

This article tests volatility jumps based on the high frequency data. Under the null hypothesis that the volatility process is a continuous semimartingale, our test statistic converges to a normal distribution, and under the alternative hypothesis where the volatility has jumps, the statistic diverges to infinity. Compared to the test statistic of Bibinger et al. (Bibinger et al. (2017). Annals of Statistics 45, 1542–1578), our proposed statistic diverges to infinity at a faster rate, and has a better power. Simulation studies confirm the theoretical results, and an empirical analysis shows that some real financial data possess volatility jumps.

Exponential tilted likelihood for stationary time series models

Depending on the asymptotical independence of periodograms, exponential tilted (ET) likelihood, as an effective nonparametric statistical method, is developed to deal with time series in this paper. Similar to empirical likelihood (EL), it still suffers from two drawbacks: the non-definition problem of the likelihood function and the under-coverage probability of confidence region. To overcome these two problems, we further proposed the adjusted ET (AET) likelihood. With a specific adjustment level, our simulation studies indicate that the AET method achieves a higher-order coverage precision than the unadjusted ET method. In addition, due to the good performance of ET under moment model misspecification [Schennach, S. M. (2007). Point estimation with exponentially tilted empirical likelihood. The Annals of Statistics, 35(2), 634–672. https://doi.org/10.1214/009053606000001208], we show that the one-order property of point estimate is preserved for the misspecified spectral estimating equations of the autoregressive coefficient of AR(1). The simulation results illustrate that the point estimates of the ET outperform those of the EL and their hybrid in terms of standard deviation. A real data set is analyzed for illustration purpose.

COMPLETE SUBSET AVERAGING FOR QUANTILE REGRESSIONS

We propose a novel conditional quantile prediction method based on complete subset averaging (CSA) for quantile regressions. All models under consideration are potentially misspecified, and the dimension of regressors goes to infinity as the sample size increases. Since we average over the complete subsets, the number of models is much larger than the usual model averaging method which adopts sophisticated weighting schemes. We propose to use an equal weight but select the proper size of the complete subset based on the leave-one-out cross-validation method. Building upon the theory of Lu and Su (2015, Journal of Econometrics 188, 40–58), we investigate the large sample properties of CSA and show the asymptotic optimality in the sense of Li (1987, Annals of Statistics 15, 958–975) We check the finite sample performance via Monte Carlo simulations and empirical applications.

Nonparametric Estimation of the Incubation Period of AIDS with Left Truncation and Right Censoring

In the natural history of Human Immunodeficiency Virus Type-1 (HIV-1) infection, many studies included the participants who were seropos itive at time of enrollment. Estimation of the unknown times since exposure to HIV-1 in the prevalent cohorts is of primary importance for estimation of the incubation period of Acquired Immunodeficiency Syndrome (AIDS). To estimate incubation period of AIDS we used prior distribution of incubation times, based on a external data as suggested by Bacchetti and Jewell (1991, Biometrics, 47,947-960). In the present study, our estimate was nonpara metric based on a method proposed by Wang, Jewell and Tsai (1986, Annals of Statistics, 14, 1597-1605).

On the Notion of Reproducibility and Its Full Implementation to Natural Exponential Families

Let F=Fθ:θ∈Θ⊂R be a family of probability distributions indexed by a parameter θ and let X1,⋯,Xn be i.i.d. r.v.’s with L(X1)=Fθ∈F. Then, F is said to be reproducible if for all θ∈Θ and n∈N, there exists a sequence (αn)n≥1 and a mapping gn:Θ→Θ,θ⟼gn(θ) such that L(αn∑i=1nXi)=Fgn(θ)∈F. In this paper, we prove that a natural exponential family F is reproducible iff it possesses a variance function which is a power function of its mean. Such a result generalizes that of Bar-Lev and Enis (1986, The Annals of Statistics) who proved a similar but partial statement under the assumption that F is steep as and under rather restricted constraints on the forms of αn and gn(θ). We show that such restrictions are not required. In addition, we examine various aspects of reproducibility, both theoretically and practically, and discuss the relationship between reproducibility, convolution and infinite divisibility. We suggest new avenues for characterizing other classes of families of distributions with respect to their reproducibility and convolution properties .

Asymptotic properties of GEE estimator for clustered ordinal data with high-dimensional covariates

Clustered ordinal data with high-dimensional covariates have become increasingly common in social and biomedical sciences. In this paper, we consider some asymptotic properties of generalized estimating equations (GEE) estimator in the case that the dimension of covariates goes to infinity as the sample size tends to infinity for such data. Under some regularity conditions, the existence, consistency and asymptotic normality of GEE estimator are proved. Besides, the effectiveness of asymptotic approximation is illustrated via numerical simulations. Our main result generalizes that of Wang [The Annals of Statistics, 39(1): 389–417, 2011] to the case of multinomial response variable.

On Optimal Rerandomization Designs

AbstractBlocking is commonly used in randomized experiments to increase efficiency of estimation. A generalization of blocking removes allocations with imbalance in covariate distributions between treated and control units, and then randomizes within the remaining set of allocations with balance. This idea of rerandomization was formalized by Morgan and Rubin (Annals of Statistics, 2012, 40, 1263–1282), who suggested using Mahalanobis distance between treated and control covariate means as the criterion for removing unbalanced allocations. Kallus (Journal of the Royal Statistical Society, Series B: Statistical Methodology, 2018, 80, 85–112) proposed reducing the set of balanced allocations to the minimum. Here we discuss the implication of such an ‘optimal’ rerandomization design for inferences to the units in the sample and to the population from which the units in the sample were randomly drawn. We argue that, in general, it is a bad idea to seek the optimal design for an inference because that inference typically only reflects uncertainty from the random sampling of units, which is usually hypothetical, and not the randomization of units to treatment versus control.

New exponential dispersion models for count data: the ABM and LM classes

In their fundamental paper on cubic variance functions (VFs), Letac and Mora (The Annals of Statistics, 1990) presented a systematic, rigorous and comprehensive study of natural exponential families (NEFs) on the real line, their characterization through their VFs and mean value parameterization. They presented a section that for some reason has been left unnoticed. This section deals with the construction of VFs associated with NEFs of counting distributions on the set of nonnegative integers and allows to find the corresponding generating measures. As EDMs are based on NEFs, we introduce in this paper two new classes of EDMs based on their results. For these classes, which are associated with simple VFs, we derive their mean value parameterization and their associated generating measures. We also prove that they have some desirable properties. Both classes are shown to be overdispersed and zero inflated in ascending order, making them as competitive statistical models for those in use in both, statistical and actuarial modeling. To our best knowledge, the classes of counting distributions we present in this paper, have not been introduced or discussed before in the literature. To show that our classes can serve as competitive statistical models for those in use (e.g., Poisson, Negative binomial), we include a numerical example of real data. In this example, we compare the performance of our classes with relevant competitive models.

ASYMPTOTIC ANALYSIS OF PERES’ ALGORITHM FOR RANDOM NUMBER GENERATION

von Neumann [(1951). Various techniques used in connection with random digits. National Bureau of Standards Applied Math Series 12: 36–38] introduced a simple algorithm for generating independent unbiased random bits by tossing a (possibly) biased coin with unknown bias. While his algorithm fails to attain the entropy bound, Peres [(1992). Iterating von Neumann's procedure for extracting random bits. The Annals of Statistics 20(1): 590–597] showed that the entropy bound can be attained asymptotically by iterating von Neumann's algorithm. Let $b(n,p)$ denote the expected number of unbiased bits generated when Peres’ algorithm is applied to an input sequence consisting of the outcomes of $n$ tosses of the coin with bias $p$. With $p=1/2$, the coin is unbiased and the input sequence consists of $n$ unbiased bits, so that $n-b(n,1/2)$ may be referred to as the cost incurred by Peres’ algorithm when not knowing $p=1/2$. We show that $\lim _{n\to \infty }\log [n-b(n,1/2)]/\log n =\theta =\log [(1+\sqrt {5})/2]$ (where $\log$ is the logarithm to base $2$), which together with limited numerical results suggests that $n-b(n,1/2)$ may be a regularly varying sequence of index $\theta$. (A positive sequence $\{L(n)\}$ is said to be regularly varying of index $\theta$ if $\lim _{n\to \infty }L(\lfloor \lambda n\rfloor )/L(n)=\lambda ^\theta$ for all $\lambda > 0$, where $\lfloor x\rfloor$ denotes the largest integer not exceeding $x$.) Some open problems on the asymptotic behavior of $nh(p)-b(n,p)$ are briefly discussed where $h(p)=-p\log p- (1-p)\log (1-p)$ denotes the Shannon entropy of a random bit with bias $p$.

Sparse Subspace Clustering via Two-Step Reweighted L1-Minimization: Algorithm and Provable Neighbor Recovery Rates

Sparse subspace clustering (SSC) relies on sparse regression for accurate neighbor identification. Inspired by recent progress in compressive sensing, this paper proposes a new sparse regression scheme for SSC via two-step reweighted $\ell _{1} $ -minimization, which also generalizes a two-step $\ell _{1} $ -minimization algorithm introduced by E. J. Candes et al. in [ The Annals of Statistics , vol. 42, no. 2, pp. 669–699, 2014] without incurring extra algorithmic complexity. To fully exploit the prior information offered by the computed sparse representation vector in the first step, our approach places a weight on each component of the regression vector, and solves a weighted LASSO in the second step. We propose a data weighting rule suitable for enhancing neighbor identification accuracy. Then, under the formulation of the dual problem of weighted LASSO, we study in depth the theoretical neighbor recovery rates of the proposed scheme. Specifically, an interesting connection between the locations of nonzeros of the optimal sparse solution to the weighted LASSO and the indexes of the active constraints of the dual problem is established. Afterwards, under the semi-random model, analytic probability lower/upper bounds for various neighbor recovery events are derived. Our analytic results confirm that, with the aid of data weighting and if the prior neighbor information is accurate enough, the proposed scheme with a higher probability can produce many correct neighbors and few incorrect neighbors as compared to the solution without data weighting. Computer simulations are provided to validate our analytic study and evidence the effectiveness of the proposed approach.

ASYMPTOTIC EXPANSION FOR THE TRANSITION DENSITIES OF STOCHASTIC DIFFERENTIAL EQUATIONS DRIVEN BY THE GAMMA PROCESSES

In this paper, enlightened by the asymptotic expansion methodology developed by Li [(2013). Maximum-likelihood estimation for diffusion processes via closed-form density expansions. Annals of Statistics 41: 1350–1380] and Li and Chen [(2016). Estimating jump-diffusions using closed-form likelihood expansions. Journal of Econometrics 195(1): 51–70], we propose a Taylor-type approximation for the transition densities of the stochastic differential equations (SDEs) driven by the gamma processes, a special type of Lévy processes. After representing the transition density as a conditional expectation of Dirac delta function acting on the solution of the related SDE, the key technical method for calculating the expectation of multiple stochastic integrals conditional on the gamma process is presented. To numerically test the efficiency of our method, we examine the pure jump Ornstein–Uhlenbeck model and its extensions to two jump-diffusion models. For each model, the maximum relative error between our approximated transition density and the benchmark density obtained by the inverse Fourier transform of the characteristic function is sufficiently small, which shows the efficiency of our approximated method.