Chapter 76 Large Sample Sieve Estimation of Semi-Nonparametric Models

We develop a general theory which provides a unified treatment for the asymptotic normality and efficiency of the maximum likelihood estimates (MLE's) in parametric, semiparametric and nonparametric models. We find that the asymptotic behavior of substitution estimates for estimating smooth functionals are essentially governed by two indices: the degree of smoothness of the functional and the local size of the underlying parameter space. We show that when the local size of the parameter space is not very large, the substitution standard (nonsieve), substitution sieve and substitution penalized MLE's are asymptotically efficient in the Fisher sense, under certain stochastic equicontinuity conditions of the log-likelihood. Moreover, when the convergence rate of the estimate is slow, the degree of smoothness of the functional needs to compensate for the slowness of the rate in order to achieve efficiency. When the size of the parameter space is very large, the standard and penalized maximum likelihood procedures may be inefficient, whereas the method of sieves may be able to overcome this difficulty. This phenomenon is particularly manifested when the functional of interest is very smooth, especially in the semiparametric case.

We obtain an improved approximation rate (in Sobolev norm) of r/sup -1/2-/spl alpha//(d+1)/ for a large class of single hidden layer feedforward artificial neural networks (ANN) with r hidden units and possibly nonsigmoid activation functions when the target function satisfies certain smoothness conditions. Here, d is the dimension of the domain of the target function, and /spl alpha//spl isin/(0, 1) is related to the smoothness of the activation function. When applying this class of ANNs to nonparametrically estimate (train) a general target function using the method of sieves, we obtain new root-mean-square convergence rates of Op([n/log(n)]/sup -/(1+2/spl alpha//(d+1))/[4(1+/spl alpha//(d+1))])=op(n/sup -1/4/) by letting the number of hidden units /spl tau//sub n/, increase appropriately with the sample size (number of training examples) n. These rates are valid for i.i.d. data as well as for uniform mixing and absolutely regular (/spl beta/-mixing) stationary time series data. In addition, the rates are fast enough to deliver root-n asymptotic normality for plug-in estimates of smooth functionals using general ANN sieve estimators. As interesting applications to nonlinear time series, we establish rates for ANN sieve estimators of four different multivariate target functions: a conditional mean, a conditional quantile, a joint density, and a conditional density. We also obtain root-n asymptotic normality results for semiparametric model coefficient and average derivative estimators.

Semi-nonparametric models are more flexible and robust than parametric models, but are more complex due to the presence of infinite dimensional unknown parameters. This article describes the method of sieve extremum estimation of semi-nonparametric models, which is a general method of optimizing an empirical criterion function over a sequence of approximating parameter spaces (that is, sieves). Widely used sieve spaces and criterion functions are presented as examples, including the sieve M-estimation, series estimation, and sieve minimum distance estimation as special cases. Existing results are cited on asymptotic properties and applications of the method.

Semi-nonparametric models are more flexible and robust than parametric models, but are more complex due to the presence of infinite dimensional unknown parameters. This article describes the method of sieve extremum estimation of semi-nonparametric models, which is a general method of optimizing an empirical criterion function over a sequence of approximating parameter spaces (that is, sieves). Widely used sieve spaces and criterion functions are presented as examples, including the sieve M-estimation, series estimation, and sieve minimum distance estimation as special cases. Existing results are cited on asymptotic properties and applications of the method.

This paper studies nonparametric estimation of conditional moment restrictions in which the generalized residual functions can be nonsmooth in the unknown functions of endogenous variables. This is a nonparametric nonlinear instrumental variables (IV) problem. We propose a class of penalized sieve minimum distance (PSMD) estimators, which are minimizers of a penalized empirical minimum distance criterion over a collection of sieve spaces that are dense in the infinite dimensional function parameter space. Some of the PSMD procedures use slowly growing finite dimensional sieves with flexible penalties or without any penalty; others use large dimensional sieves with lower semicompact and/or convex penalties. We establish their consistency and the convergence rates in Banach space norms (such as a sup-norm or a root mean squared norm), allowing for possibly non-compact infinite dimensional parameter spaces. For both mildly and severely ill-posed nonlinear inverse problems, our convergence rates in Hilbert space norms (such as a root mean squared norm) achieve the known minimax optimal rate for the nonparametric mean IV regression. We illustrate the theory with a nonparametric additive quantile IV regression. We present a simulation study and an empirical application of estimating nonparametric quantile IV Engel curves.

This paper studies nonparametric estimation of conditional moment restrictions in which the generalized residual functions can be nonsmooth in the unknown functions of endogenous variables. This is a nonparametric nonlinear instrumental variables (IV) problem. We propose a class of penalized sieve minimum distance (PSMD) estimators, which are minimizers of a penalized empirical minimum distance criterion over a collection of sieve spaces that are dense in the infinite dimensional function parameter space. Some of the PSMD procedures use slowly growing finite dimensional sieves with flexible penalties or without any penalty; others use large dimensional sieves with lower semicompact and/or convex penalties. We establish their consistency and the convergence rates in Banach space norms (such as a sup-norm or a root mean squared norm), allowing for possibly non-compact infinite dimensional parameter spaces. For both mildly and severely ill-posed nonlinear inverse problems, our convergence rates in Hilbert space norms (such as a root mean squared norm) achieve the known minimax optimal rate for the nonparametric mean IV regression. We illustrate the theory with a nonparametric additive quantile IV regression. We present a simulation study and an empirical application of estimating nonparametric quantile IV Engel curves.

Parametric estimation for the mean of a Gaussian process by the method of sieves

: The method of sieves is a technique of nonparametric estimation in which estimators are restricted by an increasing sequence of subsets of the parameter space with the subsets indexed by the sample size. The need for this technique arises in situations where the parameter space is too large for the existence or consistency of unconstrained maximum likelihood or least squares estimators. Grenander (10) developed the abstract theory of the method of sieves and provided a wealth of examples illustrating its use. This article is to appear under the entry SIEVES, METHOD OF in the Encylopedia of Statistical Sciences.

: The research project has built a theoretical foundation for using the method of sieves to adapt classical estimation principles such as maximum likelihood and least squares to problems with infinite dimensional parameter spaces. The first results about consistency of cross validated estimators of density functions have been obtained. The method of sieves and the principle of maximum likelihood have been used to develop algorithms for digital image processing. Specific applications include image segmentation, reconstruction methods for tomography, image registration methods for moving objects, and surface restoration algorithms. (Author)

In this paper, we develop a general theory for the convergence rate of sieve estimates, maximum likelihood estimates (MLE's) and related estimates obtained by optimizing certain empirical criteria in general parameter spaces. In many cases, especially when the parameter space is infinite dimensional, maximization over the whole parameter space is undesirable. In such cases, one has to perform maximization over an approximating space (sieve) of the original parameter space and allow the size of the approximating space to grow as the sample size increases. This method is called the method of sieves. In the case of the maximum likelihood estimation, an MLE based on a sieve is called a sieve MLE. We found that the convergence rate of a sieve estimate is governed by (a) the local expected values, variances and $L_2$ entropy of the criterion differences and (b) the approximation error of the sieve. A robust nonparametric regression problem, a mixture problem and a nonparametric regression problem are discussed as illustrations of the theory. We also found that when the underlying space is too large, the estimate based on optimizing over the whole parameter space may not achieve the best possible rates of convergence, whereas the sieve estimate typically does not suffer from this difficulty.

Flexible estimates of heterogeneity in crowding valuation in the New York City subway

This paper aims at better understanding passenger valuation of subway crowding in New York City. To this end, we conducted a stated preference survey with a discrete choice experiment where New Yorkers chose an alternative from a set of two hypothetical unlabeled subway routes based on occupancy levels and other attributes. We used the collected data to estimate crowding multipliers that quantify the trade-off between travel time and standee density. The previous studies have resorted to parametric heterogeneity distributions in analyzing preference variations in crowding multipliers, which can lead to misspecification issues. The contribution of this study is thus to estimate crowding multipliers using state-of-the-art semi-nonparametric models -- logit-mixed logit (LML) and mixture of normals multinomial logit (MON-MNL), and compare them across different parameter spaces. The estimated distribution of crowding multiplier of LML and MON-MNL coincide below median, but the former underestimates and the latter overestimates above median. Even though these flexible logit models can be useful for a comprehensive economic analysis of transit service improvements, these differences in estimates make model selection an important avenue for future research.

The estimation of the Lévy density, the infinite-dimensional parameter controlling the jump dynamics of a Lévy process, is considered here under a discrete-sampling scheme. In this setting, the jumps are latent variables, the statistical properties of which can be assessed when the frequency and time horizon of observations increase to infinity at suitable rates. Nonparametric estimators for the Lévy density based on Grenander’s method of sieves was proposed in Figueroa-López [IMS Lecture Notes 57 (2009) 117–146]. In this paper, central limit theorems for these sieve estimators, both pointwise and uniform on an interval away from the origin, are obtained, leading to pointwise confidence intervals and bands for the Lévy density. In the pointwise case, our estimators converge to the Lévy density at a rate that is arbitrarily close to the rate of the minimax risk of estimation on smooth Lévy densities. In the case of uniform bands and discrete regular sampling, our results are consistent with the case of density estimation, achieving a rate of order arbitrarily close to $\\log^{−1/2}(n) ⋅ n^{−1/3}$, where $n$ is the number of observations. The convergence rates are valid, provided that $s$ is smooth enough and that the time horizon $T_n$ and the dimension of the sieve are appropriately chosen in terms of $n$.

Introduction In this chapter, we review recent developments in large-sample theory for estimation of and inference on seminonparametric time-series models via the method of penalized sieves. To avoid confusion, we use the same terminology as in Chen (2007). An econometric (or statistical) model is a family of probability distributions indexed by unknown parameters. We call a model parametric if all of its unknown parameters belong to finite-dimensional Euclidean spaces. We call a model nonparametric if all of its unknown parameters belong to infinite-dimensional function spaces. Amodel is semiparametric if its parameters of interest belong to finite-dimensional spaces but its nuisance parameters are in infinite-dimensional spaces. Finally, a model is seminonparametric if it contains both finite-dimensional and infinite-dimensional unknown parameters of interest. Seminonparametric models and methods have become popular in much theoretical and empirical work in economics. This is partly because it often is the case that economic theory suggests neither parametric functional relationships among economic variables nor particular parametric forms for error distributions. Another reason for the rising popularity of semi-nonparametric models is rapidly declining costs of collecting and analyzing large datasets. The seminonparametric approach is very flexible in economic structural modeling and policy and welfare analysis. Compared to parametric and semiparametric approaches, seminonparametrics are more robust to functional-form misspecification and are better able to discover nonlinear economic relations. Compared to fully nonparametric methods, seminon-parametrics suffer less from the “curse of dimensionality” and allow for more accurate estimation of structural parameters of interest.

A bootstrap for stationary categorical time series based on the method of sieves is studied here. The data-generating process is approximated by the so-called variable-length Markov chain (VLMC), a flexible class of Markov models that allows for parsimonious structure. Then the resampling is given by simulating from the fitted model. It is shown that for a whole class of stationary categorical time series that is more general than VLMC, the VLMC sieve has faster rate of convergence for variance estimation than the more general block bootstrap. Results are illustrated from a theoretical and empirical perspective. For the latter, a real data application about (in-) homogeneity classification of a DNA strand is also presented. Finally, the VLMC sieve scheme enjoys an implementational advantage of using the plug-in rule for bootstrapping a statistical procedure, which generally is not the case for the block method.