Nonparametric Problems Research Articles

Nonparametric inference for stochastic feedforward networks based on cross-spectral analysis of point processes

In this paper we study a nonparametric estimation problem for stochastic feedforward systems with nodes of type G/G/$\infty.$ We assume that we have observations of the external arrival and external departure processes of customers of the system, but no information about the movements of the indistinguishable customers in the network. Our aim is the construction of estimators for the characteristic functions and the densities of the service time distributions at the nodes as well as for the routing probabilities. Since the system is only partly observed we have to clarify first if the parameters are identifiable from the given data. The crucial point in our approach is the observation that in our stochastic networks under study the influence of the arrival processes on the departure processes can be described in a linear and time-invariant model. This makes it possible to apply cross-spectral techniques for multivariate point processes. The construction of the estimators is then based on smoothed periodograms. We prove asymptotic normality for the estimators. We present the statistical analysis for a tandem system of two nodes in full details and show afterwards how the results can be generalized to feedforward systems of three or more nodes and to systems with positive feedback probabilities at the nodes.

Estimation for the invariant law of an ergodic diffusion process based on high-frequency data

Let a one-dimensional ergodic diffusion process X be observed at time points such that and , where , with p∈(0, 1) being a constant depending also on some conditions on X. We consider the nonparametric estimation problems for the invariant distribution and the invariant density. In both problems, we propose some estimators which are asymptotically normal and asymptotically efficient in some functional senses.

The central limit theorem for degenerate variable U-statistics under dependence

The central limit theorem (CLT) for degenerate U-statistics with a variable symmetric kernel function has been studied under dependence by many authors, since it has many important applications in nonparametric estimation and testing problems [see, e.g. Takahata, H., and Yoshihara, K. (1987), ‘Central Limit Theorems for Integrated Square Error of Nonparametric Density Estimators Based on a Absolutely Regular Random Sequences’, Yokohama Mathematical Journal, 35, 95–111; Yoshihara, K. (1989), ‘Limiting Behavior of Generalized Quadratic Forms Generated by Absolutely Regular Sequences II’, Yokohama Mathematical Journal, 37, 109–123. Yoshihara, K. (1992), ‘Limiting Behavior of Generalized Quadratic Forms Generated by Absolutely Regular Sequences III’, Yokohama Mathematical Journal, 40, 1–9; Fan, J., and Li, Q. (1999), ‘Central Limit Theorem for Degenerate U-Statistics of Absolutely Regular Processes with Applications to Model Specification Testing’, Journal of Nonparametric Statistics, 10, 245–271; Gao, J., and King, M.L. (2004), ‘Adaptive Testing in Continuous-time Diffusion Models’, Econometric Theory, 20, 844–882; Gao, J. (2007), Nonlinear Time Series: Semiparametric and Nonparametric Methods, Chapman & Hall/CRC; Gao, J., and Hong, Y. (2008), ‘Central Limit Theorem for Generalized U-statistics with Applications in Nonparametric Specification’, Journal of Nonparametric Statistics, 20, 61–76]. In this paper, we provide an improved version with the asymmetric kernel method which is quite useful for application to nonparametric methods in various situations. As an illustration of the usefulness of our result, CLTs for quadratic errors of a nonparametric density estimator are developed under dependency, which is meaningful in its own right.

Nonparametric estimation of genewise variance for microarray data

Estimation of genewise variance arises from two important applications in microarray data analysis: selecting significantly differentially expressed genes and validation tests for normalization of microarray data. We approach the problem by introducing a two-way nonparametric model, which is an extension of the famous Neyman-Scott model and is applicable beyond microarray data. The problem itself poses interesting challenges because the number of nuisance parameters is proportional to the sample size and it is not obvious how the variance function can be estimated when measurements are correlated. In such a high-dimensional nonparametric problem, we proposed two novel nonparametric estimators for genewise variance function and semiparametric estimators for measurement correlation, via solving a system of nonlinear equations. Their asymptotic normality is established. The finite sample property is demonstrated by simulation studies. The estimators also improve the power of the tests for detecting statistically differentially expressed genes. The methodology is illustrated by the data from MicroArray Quality Control (MAQC) project.

Exact linear rank tests for two-sample equivalence problems with continuous data

The present paper introduces a methodology for the semiparametric or non-parametric two-sample equivalence problem when the effects are specified by statistical functionals. The mean relative risk functional of two populations is given by the average of the time-dependent risk. This functional is a meaningful non-parametric quantity, which is invariant under strictly monotone transformations of the data. In the case of proportional hazard models, the functional determines just the proportional hazard risk factor. It is shown that an equivalence test of the type of the two-sample Savage rank test is appropriate for this functional. Under proportional hazards, this test can be carried out as an exact level α test. It also works quite well under other semiparametric models. Similar results are presented for a Wilcoxon rank-sum test for equivalence based on the Mann–Whitney functional given by the relative treatment effect.

Data-Driven k-Sample Tests

A new solution of the general nonparametric k-sample problem for independent continuous random variables is developed. Our approach to solving the nonparametric problem is via introducing a net of semiparametric models, solving the testing problem for members of this approximating net, and then combining the resulting statistics via model selection rules which mimic the rule proposed recently in Inglot and Ledwina (2006). We call such a solution a data-driven test. Since we in fact use a class of model selection criteria, we also obtain a class of k-sample data-driven tests. The class is flexible and allows us to construct omnibus as well as more specialized solutions. Simulations show that the new omnibus test has power comparable to existing k-sample tests in case of changes of location or scale, and is more powerful for much more complex changes. A variant of the new solution focused on detecting high-frequency alternations is also briefly discussed. Some theoretical support is given. This article is accompanied by supplemental materials available online at the Technometrics website.

Nonparametric Prediction in Measurement Error Models

Predicting the value of a variable Y corresponding to a future value of an explanatory variable X, based on a sample of previously observed independent data pairs (X1, Y1), …, (Xn, Yn) distributed like (X, Y), is very important in statistics. In the error-free case, where X is observed accurately, this problem is strongly related to that of standard regression estimation, since prediction of Y can be achieved via estimation of the regression curve E(Y|X). When the observed Xis and the future observation of X are measured with error, prediction is of a quite different nature. Here, if T denotes the future (contaminated) available version of X, prediction of Y can be achieved via estimation of E(Y|T). In practice, estimating E(Y|T) can be quite challenging, as data may be collected under different conditions, making the measurement errors on Xi and X nonidentically distributed. We take up this problem in the nonparametric setting and introduce estimators which allow a highly adaptive approach to smoothing. Reflecting the complexity of the problem, optimal rates of convergence of estimators can vary from the semiparametric n−1/2 rate to much slower rates that are characteristic of nonparametric problems. Nevertheless, we are able to develop highly adaptive, data-driven methods that achieve very good performance in practice. This article has the supplementary materials online.Acknowledgments: Carroll's research was supported by grants from the National CancerInstitute (CA57030, CA104620).Delaigle's work was partially supported by a fellowship from the MauriceBelz foundation.Hall's work was partially supported by the Australian Reserach Council andby a grant from the National Science Foundation (DMS 0604698).

Progressive censoring under inverse sampling for nonparametric multi-sample location problem

In this study, we propose a class of distribution-free progressive censoring test procedures for multi-sample location problem using a stage-wise partially sequential sampling technique. For this, we draw a fixed number of sample observations from one of the populations (say, control) and random number of observations from other populations (say, treatments) using suitable stopping rules. At each stage, two types of control groups, termed as fixed control group (FCG) and updated control group (UCG), are considered. Suitable stopping rules are constructed based on quantiles of the FCG and UCG observations separately. At each stage, FCG consists of initial control observations only; while UCG consists of stage-wise updated combined control observations and previous treatment observations. We examined different large sample results of the proposed tests. We also performed numerical studies to compare the performance of the tests based on FCG and UCG procedures. In addition, we compared the performance of the progressive censoring test procedures with the corresponding competitive terminal test procedure numerically.

Model selection for quantum homodyne tomography

This paper deals with a non-parametric problem coming from physics, namely quantum tomography. That consists in determining the quantum state of a mode of light through a homodyne measurement. We apply several model selection procedures: penalized projection estimators, where we may use pattern functions or wavelets, and penalized maximum likelihood estimators. In all these cases, we get oracle inequalities. In the former we also have a polynomial rate of convergence for the non-parametric problem. We finish the paper with applications of similar ideas to the calibration of a photocounter, a measurement apparatus counting the number of photons in a beam. Here the mathematical problem reduces similarly to a non-parametric missing data problem. We again get oracle inequalities, and better speed if the photocounter is good.

Estimation of additive quantile regression

We consider the nonparametric estimation problem of conditional regression quantiles with high-dimensional covariates. For the additive quantile regression model, we propose a new procedure such that the estimated marginal effects of additive conditional quantile curves do not cross. The method is based on a combination of the marginal integration technique and non-increasing rearrangements, which were recently introduced in the context of estimating a monotone regression function. Asymptotic normality of the estimates is established with a one-dimensional rate of convergence and the finite sample properties are studied by means of a simulation study and a data example.

Discussion: Nonparametric estimation of noisy integral equations of the second kind

Mammen and Yu (MY) give an excellent account of various statistical estimation problems in terms of noisy integral equations of the second kind. In particular, the authors shed a new light on the smooth backfittingmethod for fitting additive models, which provides us with another way of understanding the existing theory and also with a useful tool for developing new theory for other statistical methods. The smooth backfitting method was first introduced by Mammen, Linton, and Nielsen (1999) as a way of fitting the additive regression model. The method is known to possess very nice theoretical properties as discussed in Section 4 of MY. Its success stories in theoretical aspects have been developed recently in several other nonparametric estimation problems, including those mentioned in Sections 6 and 7 of MY. There are two other main kernel methods for fitting additive models. One is themarginal integration technique of Linton and Nielsen (1995), and the other is the ordinary backfitting procedure of Buja, Hastie, and Tibshirani (1989). It is generally accepted that, in the case of the additive regression model, the smooth backfitting method is better than the other two, not only in theoretical performance but also in numerical stability; see Nielsen and Sperlich (2005). But there are few reported results comparing the performance of the threemethods for other regressionmodels. A particular case that occurs tome, but was not discussed in MY, is the varying coefficient additive model, originally introduced by Hastie and Tibshirani (1993). This model has received much attention recently. It inherits the simplicity and easy interpretation of the classical linear model, yet is intrinsically nonparametric, so it is flexible enough to accommodate various complicated relationships between the response and predictor variables. In this model, the mean regression function is expressed as

Advances on asymptotic normality in non-parametric functional time series analysis

We consider a stationary process and wish to predict future values from previous ones. Instead of considering the process in its discretized form, we choose to see it as a sample of dependent curves. Then, we cut the process into N successive curves. Obviously, the N curves are not independent. The prediction issue can be translated into a non-parametric functional regression problem from dependent functional variables. This paper aims to revisit and complete two recent works on this topic. This article extends recent literature and provides asymptotic law with explicit constants under α-mixing assumptions. Then we establish pointwise confidence bands for the regression function. To conclude, we present how our results behave on a simulation and on a real time series.

On some treatment analyses

Fawzy et al. [1] and Jamigorn et al. [2] illustrate systemic incorrect statistical analysis, in their use of two-sample t tests, paired or independent. These tests compare normal population means assuming unknown equal variances. The central limit theorem fully justiWes normality for inferences on means, but unknown variances need not be equal. This makes two-sample t tests, together with analysis of variance, unsuitable for general mean inferences. Since Linnik [3] has shown that two-sample t-statistics for the diVerence of means based on their suYcient statistics do not exist when variances are unknown, this Behrens–Fisher problem is not avoided by uselessly [4] testing for the equality of variances. Nor is it bypassed with rank tests such as the Mann–Whitney test. Being a comparison of distributions, these rank tests say nothing about means if signiWcant, and are biased [5] to one side in a two-sided test. Tsakok [6] has solved the Behrens–Fisher problem of comparing the means of normal distributions with unknown variances at exact signiWcant levels and shown that the Tsakok solution is more eVective in detecting signiWcant mean diVerences even with unknown equal variances. There is an indication [7] that the Tsakok solution applies to dependent samples. Its exposition [8] is available. The software GSP implements the Tsakok technique, and is used to compare means at 0.02 signiWcance level (one signiWcant Wgure) per pair. For Table 2 [1], there are signiWcant mean diVerences between the enoxaparin group and placebo in their estimated gestational age (EGA) at birth, EGA at spontaneous vaginal delivery (SVD) and EGA at caesarean section. For Table 4 [1], there is signiWcant mean diVerence between placebo and the enoxaparin group in their EGA at loss. For Table 2 [2], there is signiWcant mean diVerence between the acupressure group and the vitamin B6 group in their weight at the end of the trial. After the attention given to the data, they deserve correct analysis. The Tsakok technique is extended to the non-parametric two-sample problem with the article by Tsakok [9] on constructing exact uniformly most powerful unbiased tests, superseding above rank tests. The Tsakok articles are reprinted [10] with further results.

Far Casting Cross-Validation

Cross-validation has long been used for choosing tuning parameters and other model selection tasks. It generally performs well provided the data are independent, or nearly so. Improvements have been suggested which address ordinary cross-validation’s (OCV) shortcomings in correlated data. Whereas these techniques have merit, they can still lead to poor model selection in correlated data or are not readily generalizable to high-dimensional data.The proposed solution, far casting cross-validation (FCCV), addresses these problems. FCCV withholds correlated neighbors in every aspect of the cross-validation procedure. The result is a technique that stresses a fitted model’s ability to extrapolate rather than interpolate. This generally leads to better model selection in correlated datasets.Whereas FCCV is less than optimal in the independence case, our improvement of OCV applies more generally to higher dimensional error processes and to both parametric and nonparametric model selection problems. To facilitate introduction, we consider only one application, namely estimating global bandwidths for curve estimation with local linear regression. We provide theoretical motivation and report some comparative results from a simulation experiment and on a time series of annual global temperature deviations. For such data, FCCV generally has lower average squared error when disturbances are correlated.Supplementary materials are available online.

Nonparametric Interval Estimation in One-Way Random-Effects Models

The structural method provided by Hannig et al. (2006) has proved to be a useful tool for constructing confidence intervals. However, it is difficult to apply this method to nonparametric problems since the pivotal quantity required in using it exists only in some special parametric models. Based on an extended structural method, this article discusses nonparametric interval estimation for smooth functions of the variances in one-way random-effects models. We use the bootstrap distribution estimator of a statistic to construct an approximate pivotal equation, and prove that the confidence interval derived by the approximate pivotal equation has asymptotically correct coverage probability. Simulation results are presented and show that the normal fiducial interval is not robust against non normality and that the proposed confidence interval has better finite-sample behaviors than the naive interval based on normal approximation.

Sufficient conditions for Bayesian consistency

We introduce the Hausdorff α -entropy to study the strong Hellinger consistency of posterior distributions. We obtain general Bayesian consistency theorems which extend the well-known results of Barron et al. [1999. The consistency of posterior distributions in nonparametric problems. Ann. Statist. 27, 536–561] and Ghosal et al. [1999. Posterior consistency of Dirichlet mixtures in density estimation. Ann. Statist. 27, 143–158] and Walker [2004. New approaches to Bayesian consistency. Ann. Statist. 32, 2028–2043]. As an application we strengthen previous results on Bayesian consistency of the (normal) mixture models.

Nonparametric inference in multiplicative intensity model by discrete time observation

This paper deals with nonparametric inference problems in the multiplicative intensity model for counting processes. We propose a Nelson–Aalen type estimator based on discrete observation. The functional asymptotic normality of the estimator is proved. The limit process is the same as that in the continuous observation case, thus the proposed estimator based on discrete observation has the same properties as the Nelson–Aalen estimator based on continuous observation. For example, the asymptotic efficiency of proposed estimator is valid based on less information than the continuous observation case. A Kaplan–Meier type estimator is also discussed. Nonparametric goodness of fit test is considered, and an asymptotically distribution free test is proposed.

Cross-validation for comparing multiple density estimation procedures

We demonstrate the consistency of cross-validation for comparing multiple density estimators using simple inequalities on the likelihood ratio. In nonparametric problems, the splitting of data does not require the domination of test data over the training/estimation data, contrary to Shao [Shao, J., 1993. Linear model selection by cross-validation. J. Amer. Statist. Assoc. 88, 486–494]. The result is complementary to that of Yang [Yang, Y., 2007. Consistency of cross-validation for comparing regression procedures, Ann. Statist. 35, 2450–2473; Yang, Y., 2006. Comparing learning methods for classification. Statist. Sinica 16, 635–657].

Inferences from biased samples with a memory effect

Biased sampling occurs often in observational studies. With one biased sample, the problem of nonparametrically estimating both a target density function and a selection bias function is unidentifiable. This paper studies the nonparametric estimation problem when there are two biased samples that have some overlapping observations (i.e. recaptures) from a finite population. Since an intelligent subject sampled previously may experience a memory effect if sampled again, two general 2-stage models that incorporate both a selection bias and a possible memory effect are proposed. Nonparametric estimators of the target density, selection bias, and memory functions, as well as the population size are developed. Asymptotic properties of these estimators are studied and confidence bands for the selection function and memory function are provided. Our procedures are compared with those ignoring the memory effect or the selection bias in finite sample situations. A nonparametric model selection procedure is also given for choosing a model from the two 2-stage models and a mixture of these two models. Our procedures work well with or without a memory effect, and with or without a selection bias. The paper concludes with an application to a real survey data set.

Nonparametric nonlinear regression using polynomial and neural approximators: a numerical comparison

The solution of nonparametric regression problems is addressed via polynomial approximators and one-hidden-layer feedforward neural approximators. Such families of approximating functions are compared as to both complexity and experimental performances in finding a nonparametric mapping that interpolates a finite set of samples according to the empirical risk minimization approach. The theoretical background that is necessary to interpret the numerical results is presented. Two simulation case studies are analyzed to fully understand the practical issues that may arise in solving such problems. The issues depend on both the approximation capabilities of the approximating functions and the effectiveness of the methodologies that are available to select the tuning parameters, i.e., the coefficients of the polynomials and the weights of the neural networks. The simulation results show that the neural approximators perform better than the polynomial ones with the same number of parameters. However, this superiority can be jeopardized by the presence of local minima, which affects the neural networks but does not regard the polynomial approach.