Infinite-dimensional Parameter Space Research Articles

Estimation of Nonparametric Conditional Moment Models with Possibly Nonsmooth Generalized Residuals

This paper studies nonparametric estimation of conditional moment restrictions in which the generalized residual functions can be nonsmooth in the unknown functions of endogenous variables. This is a nonparametric nonlinear instrumental variables (IV) problem. We propose a class of penalized sieve minimum distance (PSMD) estimators, which are minimizers of a penalized empirical minimum distance criterion over a collection of sieve spaces that are dense in the infinite dimensional function parameter space. Some of the PSMD procedures use slowly growing finite dimensional sieves with flexible penalties or without any penalty; others use large dimensional sieves with lower semicompact and/or convex penalties. We establish their consistency and the convergence rates in Banach space norms (such as a sup-norm or a root mean squared norm), allowing for possibly non-compact infinite dimensional parameter spaces. For both mildly and severely ill-posed nonlinear inverse problems, our convergence rates in Hilbert space norms (such as a root mean squared norm) achieve the known minimax optimal rate for the nonparametric mean IV regression. We illustrate the theory with a nonparametric additive quantile IV regression. We present a simulation study and an empirical application of estimating nonparametric quantile IV Engel curves.

Estimation of Nonparametric Conditional Moment Models with Possibly Nonsmooth Generalized Residuals

This paper studies nonparametric estimation of conditional moment restrictions in which the generalized residual functions can be nonsmooth in the unknown functions of endogenous variables. This is a nonparametric nonlinear instrumental variables (IV) problem. We propose a class of penalized sieve minimum distance (PSMD) estimators, which are minimizers of a penalized empirical minimum distance criterion over a collection of sieve spaces that are dense in the infinite dimensional function parameter space. Some of the PSMD procedures use slowly growing finite dimensional sieves with flexible penalties or without any penalty; others use large dimensional sieves with lower semicompact and/or convex penalties. We establish their consistency and the convergence rates in Banach space norms (such as a sup-norm or a root mean squared norm), allowing for possibly non-compact infinite dimensional parameter spaces. For both mildly and severely ill-posed nonlinear inverse problems, our convergence rates in Hilbert space norms (such as a root mean squared norm) achieve the known minimax optimal rate for the nonparametric mean IV regression. We illustrate the theory with a nonparametric additive quantile IV regression. We present a simulation study and an empirical application of estimating nonparametric quantile IV Engel curves.

Dynamics of Bayesian updating with dependent data and misspecified models

Much is now known about the consistency of Bayesian updating on infinite-dimensional parameter spaces with independent or Markovian data. Necessary conditions for consistency include the prior putting enough weight on the correct neighborhoods of the data-generating distribution; various sufficient conditions further restrict the prior in ways analogous to capacity control in frequentist nonparametrics. The asymptotics of Bayesian updating with mis-specified models or priors, or non-Markovian data, are far less well explored. Here I establish sufficient conditions for posterior convergence when all hypotheses are wrong, and the data have complex dependencies. The main dynamical assumption is the asymptotic equipartition (Shannon-McMillan-Breiman) property of information theory. This, along with Egorov’s Theorem on uniform convergence, lets me build a sieve-like structure for the prior. The main statistical assumption, also a form of capacity control, concerns the compatibility of the prior and the data-generating process, controlling the fluctuations in the log-likelihood when averaged over the sieve-like sets. In addition to posterior convergence, I derive a kind of large deviations principle for the posterior measure, extending in some cases to rates of convergence, and discuss the advantages of predicting using a combination of models known to be wrong. An appendix sketches connections between these results and the replicator dynamics of evolutionary theory.

Gluing an Infinite Number of Instantons

AbstractThis paper is one step toward infinite energy gauge theory and the geometry of infinite dimensional moduli spaces. We generalize a gluing construction in the usual Yang-Mills gauge theory to an “infinite energy” situation. We show that we can glue an infinite number of instantons, and that the resulting ASD connections have infinite energy in general. Moreover they have an infinite dimensional parameter space. Our construction is a generalization of Donaldson’s “alternating method”.

Bifurcation structures in maps of Hénon type

We construct a series of n-unimodal approximations to maps of the Hénon type and utilize the associated symbolic dynamics to describe the possible bifurcation structures for such maps. We construct the bifurcation surfaces of the short periodic orbits in the topological parameter space and check numerically that the Hénon map parameter plane (a,b) is topologically equivalent to a two-dimensional section through the infinite-dimensional parameter space characterizing a generic map of the Hénon type.

Large parameter spaces of quasiconformally distinct hyperbolic structures

SupposeS is a surface of infinite type with pants decomposition ⌆. We construct a real analytic embedding of an infinite-dimensional parameter space into the Fenchel-Nielsen space ofS with respect to ⌆, whose image is made up of topologically conjugate Fuchsian groups for which no two groups in the image are quasiconformally conjugate. Moreover, all of the Fuchsian groups in this parameter space have the same Fenchel—Nielsen twist parameters. As a consequence, arbitrarily close (in the Fenchel-Nielsen topology) to a hyperbolic structure forS there is an infinite-dimensional set of disjoint Teichmuller spaces.

Nonparametric maximum likelihood estimation in a non locally compact setting

Maximum likelihood estimation is considered in the context of infinite dimensional parameter spaces. It is shown that in some locally convex parameter spaces sequential compactness of the bounded sets ensures the existence of minimizers of objective functions and the consistency of maximum likelihood estimators in an appropriate topology. The theory is applied to revisit some classical problems of nonparametric maximum likelihood estimation, to study location parameters in Banach spaces, and finally to obtain Varadarajan’s theorem on the convergence of empirical measures in the form of a consistency result for a sequence of maximum likelihood estimators. Several parameter spaces sharing the crucial compactness property are identified.

Principal component decomposition of non-parametric tests

Let ϕ denote an arbitrary non-parametric unbiased test for a Gaussian shift given by an infinite dimensional parameter space. Then it is shown that the curvature of its power function has a principal component decomposition based on a Hilbert-Schmidt operator. Thus every test has reasonable curvature only for a finite number of orthogonal directions of alternatives. As application the two-sided Kolmogorov-Smirnov goodnessof-fit test is treated. We obtain lower bounds for their local asymptotic relative efficiency. They converge to one as α↓0 for the directionh0(u)=sign(2u−1) of the gradient of the median test. These results are analogous to earlier results of Hajek and Sidak for one-sided Kolmogorov-Smirnov tests.

Efficiency. of infinite dimensional M‐ estimators

It is well‐known that maximum likelihood estimators are asymptotically normal with covariance equal to the inverse Fisher information in smooth, finite dimensional parametric models. Thus they are asymptotically efficient. A similar phenomenon has been observed for certain infinite dimensional parameter spaces. We give a simple proof of efficiency, starting from a theorem on asymptotic normality of infinite dimensional M‐estimators. The proof avoids the explicit calculation of the Fisher information. We also address Hadamard differentiability of the corresponding M‐functionals.

Toward a Theory of Evolution Strategies: Some Asymptotical Results from the (1,+ λ)-Theory

A method for the determination of the progress rate and the probability of success for the Evolution Strategy (ES) is presented. The new method is based on the asymptotical behavior of the χ-distribution and yields exact results in the case of infinite-dimensional parameter spaces. The technique is demonstrated for the (l,+ λ) ES using a spherical model including noisy quality functions. The results are used to discuss the convergence behavior of the ES.