Nonparametric Function Estimation Research Articles

Stock market prices and Dividends in the US: Bubbles or Long-run equilibria relationships?

This paper presents a novel approach to identifying potential bubbles in the US stock market by employing alternative time series methods based on long memory, including fractional integration and cointegration, as well as duration dependence non-parametric models. To test for duration dependence, the paper employs a unique non-parametric hazard function estimation method, using monotonic P-splines, which enables the estimation of a coherent and sophisticated Bayesian confidence interval for the function. By analyzing historical data of real US stock index from January 1871 to November 2022, the results reveal strong evidence of bubbles in the US stock market. It is worth noting that this is the first time a fractional integration approach and a novel non-parametric hazard function have been utilized to test for rational speculative bubbles in the US stock market, making this paper a significant contribution to the field.

Spectral density estimation for random processes with stationary increments

AbstractSpectral density analysis plays an important role in studying a stationary random process on a real line. In this paper, we extend this discussion for the random process with stationary increments. We investigate the properties of the method of moments structure function estimation, and propose a nonparametric spectral density function estimator. Our numerical results show that the proposed spectral density estimator performs comparable with the parametric counterpart when the underlying process is assumed to be band‐limited. Additionally, this method is applied to analyze US Housing Starts Data, where the hidden periodicities are detected, providing consistent conclusions with previous economic studies.

Statistical Spatially Inhomogeneous Diffusion Inference

Inferring a diffusion equation from discretely observed measurements is a statistical challenge of significant importance in a variety of fields, from single-molecule tracking in biophysical systems to modeling financial instruments. Assuming that the underlying dynamical process obeys a d-dimensional stochastic differential equation of the form dx_t = b(x_t)dt + \Sigma(x_t)dw_t, we propose neural network-based estimators of both the drift b and the spatially-inhomogeneous diffusion tensor D = \Sigma\Sigma^T/2 and provide statistical convergence guarantees when b and D are s-Hölder continuous. Notably, our bound aligns with the minimax optimal rate N^{-\frac{2s}{2s+d}} for nonparametric function estimation even in the presence of correlation within observational data, which necessitates careful handling when establishing fast-rate generalization bounds. Our theoretical results are bolstered by numerical experiments demonstrating accurate inference of spatially-inhomogeneous diffusion tensors.

Abstract P329: Lifetime Risk of Dementia: The Atherosclerosis Risk in Communities (ARIC) Study

Introduction: Prior studies suggest that 15-20% of adults will develop dementia after midlife. However, current lifetime risk estimates are based on limited dementia ascertainment, potentially resulting in underestimation. Methods: We conducted a prospective cohort analysis using data from the Atherosclerosis Risk in Communities Study (ARIC) (baseline 1990-1992). We used non-parametric cumulative incidence function estimation to calculate lifetime risk of dementia while accounting for the competing risk of death, overall and stratified by demographics and APOE ε 4 status. Dementia was ascertained until December 31, 2019 by: (1) cognitive testing at study visits, with diagnoses generated using a standardized computer algorithm and confirmed by a panel of experts; (2) semiannual telephone cognitive testing and Clinical Dementia Rating interviews with informants; and (3) ICD codes from hospital records and death certificates. Results: We included 13,358 participants (mean age: 57 years; 25% of Black race; 55% female sex; 31% carried at least one APOE ε 4 allele). Over a median follow-up of 24 (IQR: 20, 29) years, there were 2,812 incident cases of dementia. At age 55, the lifetime risk of dementia (up to age 95) was 43% (Fig A) . Adults with two copies of the APOE ε 4 allele had substantially higher lifetime risk (61%) of dementia compared to those with one copy (48%) and those none (40%), with differences beginning at age ~70 years (Fig B) . Lifetime risk was higher in women and Black adults (Fig C-D) . Conclusion: More than 4 in 10 adults will develop dementia after midlife. Comprehensive capture of dementia in the ARIC study results in lifetime risk estimate substantially exceeding prior research.

Lévy Adaptive B-spline Regression via Overcomplete Systems

The estimation of functions with varying degrees of smoothness is a challenging problem in the nonparametric function estimation. In this paper, we propose the LABS (L\'{e}vy Adaptive B-Spline regression) model, an extension of the LARK models, for the estimation of functions with varying degrees of smoothness. LABS model is a LARK with B-spline bases as generating kernels. The B-spline basis consists of piecewise k degree polynomials with k-1 continuous derivatives and can express systematically functions with varying degrees of smoothness. By changing the orders of the B-spline basis, LABS can systematically adapt the smoothness of functions, i.e., jump discontinuities, sharp peaks, etc. Results of simulation studies and real data examples support that this model catches not only smooth areas but also jumps and sharp peaks of functions. The proposed model also has the best performance in almost all examples. Finally, we provide theoretical results that the mean function for the LABS model belongs to the certain Besov spaces based on the orders of the B-spline basis and that the prior of the model has the full support on the Besov spaces.

A Sequential Stein's Method for Faster Training of Additive Index Models.

The additive index models (AIMs) can be viewed as a kind of artificial neural networks based on nonparametric activation or so-called ridge functions. Recently, they are shown to achieve enhanced explainability after incorporating various interpretability constraints. However, the training of AIMs by either the backfitting algorithm or the joint stochastic optimization is known to be very slow for especially high dimensional inputs. In this article, we propose a novel sequential approach based on the celebrated Stein's lemma. The proposed SeqStein method can successfully decouple the training of AIMs into two separable steps, namely, the following: 1) Stein's estimation of the projection indices and 2) nonparametric estimation of ridge functions using the smoothing splines. We show through numerical experiments that the SeqStein algorithm is not only more efficient for training AIMs, but also inclined to produce more interpretable models that have smooth ridge functions with sparse and nearly orthogonal projection indices.

Nonparametric path function estimation of Fourier series at low oscillations for modelling timely paying credit

Nonparametric path function estimation of Fourier series at low oscillations for modelling timely paying credit

Nonparametric path function estimation of Fourier series at low oscillations for modelling timely paying credit

Nonparametric path function estimation of Fourier series at low oscillations for modelling timely paying credit

Semi-parametric single-index predictive regression models with cointegrated regressors

This paper considers the estimation of a semi-parametric single-index regression model that allows for nonlinear predictive relationships. This model is useful for predicting financial asset returns, whose observed behaviour resembles a stationary process, if the multiple nonstationary predictors are cointegrated. The presence of cointegrated regressors imposes a single-index structure in the model, and this structure not only balances the nonstationarity properties of the multiple predictors with the stationarity properties of asset returns but also avoids the curse of dimensionality associated with nonparametric regression function estimation. An orthogonal series expansion is used to approximate the unknown link function for the single-index component. We consider the constrained nonlinear least squares estimator of the single-index (or the cointegrating) parameters and the plug-in estimator of the link function, and derive their asymptotic properties. In an empirical application, we find some evidence of in-sample nonlinear predictability of U.S. stock returns using cointegrated predictors. We also find that the single-index model in general produces better out-of-sample forecasts than both the historical average benchmark and the linear predictive regression model.

NONPARAMETRIC FUNCTIONAL HAZARD WITH k NEAREST NEIGHBORS ESTIMATION WHERE THE OBSERVATIONS ARE M.A.R. AND RELATED VIA A FUNCTIONAL SINGLE-INDEX STRUCTURE

The nonparametric local linear hazard functional estimation analyzed by the k.N.N method of the scalar response variable B, not totally observed provided the functional variable A. The purpose of this paper is to illustrate, under some general conditions, the almost complete convergence (with rates) of the constructed estimator.

Boundary-adaptive kernel density estimation: the case of (near) uniform density

We consider nonparametric kernel estimation of density functions in the bounded-support setting having known support [ a , b ] using a boundary-adaptive kernel function and data-driven bandwidth selection, where a and b are finite and known prior to estimation. We observe, theoretically and in finite sample settings, that when bounds are known a priori this kernel approach is capable of outperforming even correctly specified parametric models, in the case of the uniform distribution. We demonstrate that this result has implications for modelling a range of densities other than the uniform case. Furthermore, when bounds [ a , b ] are unknown and the empirical support (i.e. [ min ( x i ) , max ( x i ) ] ) is used in their place, similar behaviour surfaces.

Nonparametric estimation of the shape functions and related asymptotic results

AbstractArriaza et al (Metrika 82:99–124, 2019) introduced the right and left shape functions, which enjoy interesting properties in terms of describing the global form of a distribution. This paper proposes and studies nonparametric estimators of those functions. The estimators involve nonparametric estimation of the quantile and density functions. Pointwise and uniform consistency are proved under general regularity assumptions, as well as the limit in law. Simulations are included to study the practical performance of the proposed estimators. The analysis of a real data set illustrates the methodology.

Neural Networks for Partially Linear Quantile Regression

Deep learning has enjoyed tremendous success in a variety of applications but its application to quantile regression remains scarce. A major advantage of the deep learning approach is its flexibility to model complex data in a more parsimonious way than nonparametric smoothing methods. However, while deep learning brought breakthroughs in prediction, it is not well suited for statistical inference due to its black box nature. In this article, we leverage the advantages of deep learning and apply it to quantile regression where the goal is to produce interpretable results and perform statistical inference. We achieve this by adopting a semiparametric approach based on the partially linear quantile regression model, where covariates of primary interest for statistical inference are modeled linearly and all other covariates are modeled nonparametrically by means of a deep neural network. In addition to the new methodology, we provide theoretical justification for the proposed model by establishing the root-n consistency and asymptotically normality of the parametric coefficient estimator and the minimax optimal convergence rate of the neural nonparametric function estimator. Across several simulated and real data examples, the proposed model empirically produces superior estimates and more accurate predictions than various alternative approaches.

A spline-assisted semiparametric approach to nonparametric measurement error models

A spline-assisted approach is proposed to handle the measurement error problem in treating the pollution and asthma data. It is well known that the minimax rate of convergence in nonparametric regression function estimation of a random variable measured with error is much slower than the rate in the error free case. A different problem is considered. It is shown that if one is willing to impose a relatively mild assumption in requiring that the error-prone variable has a compact support, then standard nonparametric results are obtainable for measurement error models. New and constructive methods to take full advantage of the compact support assumption via spline-assisted semiparametric methods are proposed. It is proven that the new estimator achieves the usual nonparametric rate as if there were no measurement error. Furthermore it is shown that similar spline approach can be used to assess the probability density function on a compact set, using observations with error, and the resulting estimator differs from the true density function only by a constant scale. In addition, it retains the nonparametric density convergence properties of the error free case. The performance of the new methods is demonstrated through simulations and the methods are implemented to analyze the relation between asthma and pollution.

Penalized Whittle likelihood for spatial data

Inference for spatial data is challenging because fitting an appropriate parametric model is often difficult. The penalized likelihood-type approach has been successfully developed for various nonparametric function estimation problems in time series analysis. However, it has not been well developed in spatial analysis. In this paper, a penalized Whittle likelihood approach is developed for nonparametric estimation of spectral density functions for regularly spaced spatial data. In particular, the estimated spectral density is the minimizer of a criterion which is developed based on the Whittle likelihood and a penalty for roughness. This approach aggregates several popular nonparametric density estimation methods into a coherent framework. Asymptotic properties of the proposed estimator are derived under mild assumptions without assuming Gaussianity. In addition, a computationally efficient method is developed to optimize the penalized likelihood function. Simulation results and real data examples are also provided to illustrate the finite sample performances of the methodology.

On a new higher order kernel for density estimation

Higher order kernels have been widely implemented for nonparametric function estimation, mainly due to their faster asymptotic rates of convergence and interpretability. This article constructs a novel higher order kernel of any pre‐specified even order by using Shannon's formula. The proposed kernels have a closed form expression and are easy to implement. We compare the constructed kernel with several popular higher order kernels in the context of density estimation. Further, we observe that the mean integrated squared error (MISE) of the density estimator corresponding to the proposed kernel is comparable with that of the popular kernels. The constructed kernel performs somewhat better for the Fejér‐de la Vallée Poussin and the ‐symmetric densities. The MISE of the corresponding kernel density estimator and a few related theoretical results are also presented.

Estimation and Testing of Random Effects Semiparametric Regression Model with Separable Space-Time Filters

This paper focuses on studying a random effects semiparametric regression model (RESPRM) with separable space-time filters. The model cannot only capture the linearity and nonlinearity existing in a space-time dataset, but also avoid the inefficient estimators caused by ignoring spatial correlation and serial correlation in the error term of a space-time data regression model. Its profile quasi-maximum likelihood estimators (PQMLE) for parameters and nonparametric functions, and a generalized F-test statistic for checking the existence of nonlinear relationships are constructed. The asymptotic properties of estimators and asymptotic distribution of test statistic are derived. Monte Carlo simulations imply that our estimators and test statistic have good finite sample performance. The Indonesian rice farming data are used to illustrate our methods.

Wavelet Density and Regression Estimators for Continuous Time Functional Stationary and Ergodic Processes

In this study, we look at the wavelet basis for the nonparametric estimation of density and regression functions for continuous functional stationary processes in Hilbert space. The mean integrated squared error for a small subset is established. We employ a martingale approach to obtain the asymptotic properties of these wavelet estimators. These findings are established under rather broad assumptions. All we assume about the data is that they are ergodic, but beyond that, we make no assumptions. In this paper, the mean integrated squared error findings in the independence or mixing setting were generalized to the ergodic setting. The theoretical results presented in this study are (or will be) valuable resources for various cutting-edge functional data analysis applications. Applications include conditional distribution, conditional quantile, entropy, and curve discrimination.

Kernel density estimation for undirected dyadic data

We study nonparametric estimation of density functions for undirected dyadic random variables (i.e., random variables defined for all n≡defN2 unordered pairs of agents/nodes in a weighted network of order N). These random variables satisfy a local dependence property: any random variables in the network that share one or two indices may be dependent, while those sharing no indices in common are independent. In this setting, we show that density functions may be estimated by an application of the kernel estimation method of Rosenblatt (1956) and Parzen (1962). We suggest an estimate of their asymptotic variances inspired by a combination of (i) Newey’s (1994) method of variance estimation for kernel estimators in the “monadic” setting and (ii) a variance estimator for the (estimated) density of a simple network first suggested by Holland and Leinhardt (1976). More unusual are the rates of convergence and asymptotic (normal) distributions of our dyadic density estimates. Specifically, we show that they converge at the same rate as the (unconditional) dyadic sample mean: the square root of the number, N, of nodes. This differs from the results for nonparametric estimation of densities and regression functions for monadic data, which generally have a slower rate of convergence than their corresponding sample mean.

Nonparametric Testing for Hammerstein Systems

We examine the problem of testing a parametric form of the nonlinearity of Hammerstein systems against a nonparametric alternative. This article proposes test statistics relying on nonparametric orthogonal series regression function estimates. An asymptotic theory of the proposed tests is established including the asymptotic normality under the null hypothesis that the parametric form of the system nonlinearity is correct. Moreover, it is shown that under nonparametric alternatives, the probability of detecting that the parametric model is incorrect tends to one. The rate of detecting local alternatives is also established.