Nonparametric Function Estimation Research Articles

Analysis of Variance of Tensor Product Reproducing Kernel Hilbert Spaces on Metric Spaces

Many methods have been developed to analyze complex data, such as non-Euclidean shape, network, and manifold data. However, there is a lack of methods for studying interactions among complex data. In this paper, we first propose a novel kernel function for a metric space and construct its associated reproducing kernel Hilbert space. The new nonstationary kernel function provides a flexible and powerful tool for learning complex structures in non-Euclidean data. We then construct an analysis of variance (ANOVA) decomposition of the nonparametric regression function defined on metric space, which provides a hierarchical structure for investigating the main effects and interactions. We develop estimation and computational methods for a semi-parametric model with a multivariate function on a product of metric spaces modeled by the ANOVA decomposition. We establish the convergence rates of parameter and nonparametric function estimates. The application of the proposed methods to the Alzheimer’s Disease Neuroimaging Initiative hippocampus shape data confirms some existing and suggests some new interactions among hippocampal regions. Simulations indicate that the proposed methods work well.

Nonparametric estimation of the cumulative incidence function for doubly-truncated and interval-censored competing risks data.

Interval sampling is widely used for collection of disease registry data, which typically report incident cases during a certain time period. Such sampling scheme induces doubly truncated data if the failure time can be observed exactly and doubly truncated and interval censored (DTIC) data if the failure time is known only to lie within an interval. In this article, we consider nonparametric estimation of the cumulative incidence functions (CIF) using doubly-truncated and interval-censored competing risks (DTIC-C) data obtained from interval sampling scheme. Using the approach of Shen (Stat Methods Med Res 31:1157-1170, 2022b), we first obtain the nonparametric maximum likelihood estimator (NPMLE) of the distribution function of failure time ignoring failure types. Using the NPMLE, we proposed nonparametric estimators of the CIF with DTIC-C data and establish consistency of the proposed estimators. Simulation studies show that the proposed estimator performs well for finite sample size.

Calibrated Equilibrium Estimation and Double Selection for High-dimensional Partially Linear Measurement Error Models

In practice, measurement error data is frequently encountered and needs to be handled appropriately. As a result of additional bias induced by measurement error, many existing estimation methods fail to achieve satisfactory performances. This paper studies high-dimensional partially linear measurement error models. It proposes a calibrated equilibrium (CARE) estimation method to calibrate the bias caused by measurement error and overcomes the technical difficulty of the objective function unbounded from below in high-dimensional cases due to non-convexity. To facilitate the applications of the CARE estimation method, a bootstrap approach for approximating the covariance matrix of measurement errors is introduced. For the high-dimensional or ultra-high dimensional partially linear measurement error models, a novel multiple testing method, the calibrated equilibrium multiple double selection (CARE–MUSE) algorithm, is proposed to control the false discovery rate (FDR). Under certain regularity conditions, we derive the oracle inequalities for estimation error and prediction risk, along with an upper bound on the number of falsely discovered signs for the CARE estimator. We further establish the convergence rate of the nonparametric function estimator. In addition, FDR and power guarantee for the CARE–MUSE algorithm are investigated under a weaker minimum signal condition, which is insufficient for the CARE estimator to achieve sign consistency. Extensive simulation studies and a real data application demonstrate the satisfactory finite sample performance of the proposed methods.

A Sequential Stein's Method for Faster Training of Additive Index Models.

The additive index models (AIMs) can be viewed as a kind of artificial neural networks based on nonparametric activation or so-called ridge functions. Recently, they are shown to achieve enhanced explainability after incorporating various interpretability constraints. However, the training of AIMs by either the backfitting algorithm or the joint stochastic optimization is known to be very slow for especially high dimensional inputs. In this article, we propose a novel sequential approach based on the celebrated Stein's lemma. The proposed SeqStein method can successfully decouple the training of AIMs into two separable steps, namely, the following: 1) Stein's estimation of the projection indices and 2) nonparametric estimation of ridge functions using the smoothing splines. We show through numerical experiments that the SeqStein algorithm is not only more efficient for training AIMs, but also inclined to produce more interpretable models that have smooth ridge functions with sparse and nearly orthogonal projection indices.

Nonparametric conditional survival function estimation and plug-in bandwidth selection with multiple covariates

Nonparametric conditional survival function estimation and plug-in bandwidth selection with multiple covariates

Active mutual conjoint estimation of multiple contrast sensitivity functions.

Recent advances in nonparametric contrast sensitivity function (CSF) estimation have yielded a new tradeoff between accuracy and efficiency not available to classical parametric estimators. An additional advantage of this new framework is the ability to independently tune multiple aspects of the estimator to seek further improvements. Machine learning CSF estimation with Gaussian processes allows for design optimization in the kernel, acquisition function, and underlying task representation, to name a few. This article describes a novel kernel for CSF estimation that is more flexible than a kernel based on strictly functional forms. Despite being more flexible, it can result in a more efficient estimator. Further, trial selection for data acquisition that is generalized beyond pure information gain can also improve estimator quality. Finally, introducing latent variable representations underlying general CSF shapes can enable simultaneous estimation of multiple CSFs, such as from different eyes, eccentricities, or luminances. The conditions under which the new procedures perform better than previous nonparametric estimation procedures are presented and quantified.

Mortality prediction using survival energy models with functional data analysis

The survival energy model (SEM), as originally introduced by Shimizu et al. (ASTIN Bull 51(1):191–219, 2020), is designed to characterize human bioenergetics by employing diffusion processes or inverse Gaussian processes. While parametric models have been employed to articulate the SEM, they exhibit inherent sensitivity in their parameters and hyperparameters, which in turn introduces issues of instability in estimation and prediction. In this paper, we demonstrate that the utilization of functional data analysis techniques for nonparametric estimation and prediction of critical functions within the SEM leads to a substantial enhancement in prediction performance.

Stock market prices and Dividends in the US: Bubbles or Long-run equilibria relationships?

This paper presents a novel approach to identifying potential bubbles in the US stock market by employing alternative time series methods based on long memory, including fractional integration and cointegration, as well as duration dependence non-parametric models. To test for duration dependence, the paper employs a unique non-parametric hazard function estimation method, using monotonic P-splines, which enables the estimation of a coherent and sophisticated Bayesian confidence interval for the function. By analyzing historical data of real US stock index from January 1871 to November 2022, the results reveal strong evidence of bubbles in the US stock market. It is worth noting that this is the first time a fractional integration approach and a novel non-parametric hazard function have been utilized to test for rational speculative bubbles in the US stock market, making this paper a significant contribution to the field.

Spectral density estimation for random processes with stationary increments

AbstractSpectral density analysis plays an important role in studying a stationary random process on a real line. In this paper, we extend this discussion for the random process with stationary increments. We investigate the properties of the method of moments structure function estimation, and propose a nonparametric spectral density function estimator. Our numerical results show that the proposed spectral density estimator performs comparable with the parametric counterpart when the underlying process is assumed to be band‐limited. Additionally, this method is applied to analyze US Housing Starts Data, where the hidden periodicities are detected, providing consistent conclusions with previous economic studies.

Statistical Spatially Inhomogeneous Diffusion Inference

Inferring a diffusion equation from discretely observed measurements is a statistical challenge of significant importance in a variety of fields, from single-molecule tracking in biophysical systems to modeling financial instruments. Assuming that the underlying dynamical process obeys a d-dimensional stochastic differential equation of the form dx_t = b(x_t)dt + \Sigma(x_t)dw_t, we propose neural network-based estimators of both the drift b and the spatially-inhomogeneous diffusion tensor D = \Sigma\Sigma^T/2 and provide statistical convergence guarantees when b and D are s-Hölder continuous. Notably, our bound aligns with the minimax optimal rate N^{-\frac{2s}{2s+d}} for nonparametric function estimation even in the presence of correlation within observational data, which necessitates careful handling when establishing fast-rate generalization bounds. Our theoretical results are bolstered by numerical experiments demonstrating accurate inference of spatially-inhomogeneous diffusion tensors.

Abstract P329: Lifetime Risk of Dementia: The Atherosclerosis Risk in Communities (ARIC) Study

Introduction: Prior studies suggest that 15-20% of adults will develop dementia after midlife. However, current lifetime risk estimates are based on limited dementia ascertainment, potentially resulting in underestimation. Methods: We conducted a prospective cohort analysis using data from the Atherosclerosis Risk in Communities Study (ARIC) (baseline 1990-1992). We used non-parametric cumulative incidence function estimation to calculate lifetime risk of dementia while accounting for the competing risk of death, overall and stratified by demographics and APOE ε 4 status. Dementia was ascertained until December 31, 2019 by: (1) cognitive testing at study visits, with diagnoses generated using a standardized computer algorithm and confirmed by a panel of experts; (2) semiannual telephone cognitive testing and Clinical Dementia Rating interviews with informants; and (3) ICD codes from hospital records and death certificates. Results: We included 13,358 participants (mean age: 57 years; 25% of Black race; 55% female sex; 31% carried at least one APOE ε 4 allele). Over a median follow-up of 24 (IQR: 20, 29) years, there were 2,812 incident cases of dementia. At age 55, the lifetime risk of dementia (up to age 95) was 43% (Fig A) . Adults with two copies of the APOE ε 4 allele had substantially higher lifetime risk (61%) of dementia compared to those with one copy (48%) and those none (40%), with differences beginning at age ~70 years (Fig B) . Lifetime risk was higher in women and Black adults (Fig C-D) . Conclusion: More than 4 in 10 adults will develop dementia after midlife. Comprehensive capture of dementia in the ARIC study results in lifetime risk estimate substantially exceeding prior research.

Lévy Adaptive B-spline Regression via Overcomplete Systems

The estimation of functions with varying degrees of smoothness is a challenging problem in the nonparametric function estimation. In this paper, we propose the LABS (L\'{e}vy Adaptive B-Spline regression) model, an extension of the LARK models, for the estimation of functions with varying degrees of smoothness. LABS model is a LARK with B-spline bases as generating kernels. The B-spline basis consists of piecewise k degree polynomials with k-1 continuous derivatives and can express systematically functions with varying degrees of smoothness. By changing the orders of the B-spline basis, LABS can systematically adapt the smoothness of functions, i.e., jump discontinuities, sharp peaks, etc. Results of simulation studies and real data examples support that this model catches not only smooth areas but also jumps and sharp peaks of functions. The proposed model also has the best performance in almost all examples. Finally, we provide theoretical results that the mean function for the LABS model belongs to the certain Besov spaces based on the orders of the B-spline basis and that the prior of the model has the full support on the Besov spaces.

Nonparametric path function estimation of Fourier series at low oscillations for modelling timely paying credit

Nonparametric path function estimation of Fourier series at low oscillations for modelling timely paying credit

Nonparametric path function estimation of Fourier series at low oscillations for modelling timely paying credit

Nonparametric path function estimation of Fourier series at low oscillations for modelling timely paying credit

Semi-parametric single-index predictive regression models with cointegrated regressors

This paper considers the estimation of a semi-parametric single-index regression model that allows for nonlinear predictive relationships. This model is useful for predicting financial asset returns, whose observed behaviour resembles a stationary process, if the multiple nonstationary predictors are cointegrated. The presence of cointegrated regressors imposes a single-index structure in the model, and this structure not only balances the nonstationarity properties of the multiple predictors with the stationarity properties of asset returns but also avoids the curse of dimensionality associated with nonparametric regression function estimation. An orthogonal series expansion is used to approximate the unknown link function for the single-index component. We consider the constrained nonlinear least squares estimator of the single-index (or the cointegrating) parameters and the plug-in estimator of the link function, and derive their asymptotic properties. In an empirical application, we find some evidence of in-sample nonlinear predictability of U.S. stock returns using cointegrated predictors. We also find that the single-index model in general produces better out-of-sample forecasts than both the historical average benchmark and the linear predictive regression model.

NONPARAMETRIC FUNCTIONAL HAZARD WITH k NEAREST NEIGHBORS ESTIMATION WHERE THE OBSERVATIONS ARE M.A.R. AND RELATED VIA A FUNCTIONAL SINGLE-INDEX STRUCTURE

The nonparametric local linear hazard functional estimation analyzed by the k.N.N method of the scalar response variable B, not totally observed provided the functional variable A. The purpose of this paper is to illustrate, under some general conditions, the almost complete convergence (with rates) of the constructed estimator.

Boundary-adaptive kernel density estimation: the case of (near) uniform density

We consider nonparametric kernel estimation of density functions in the bounded-support setting having known support [ a , b ] using a boundary-adaptive kernel function and data-driven bandwidth selection, where a and b are finite and known prior to estimation. We observe, theoretically and in finite sample settings, that when bounds are known a priori this kernel approach is capable of outperforming even correctly specified parametric models, in the case of the uniform distribution. We demonstrate that this result has implications for modelling a range of densities other than the uniform case. Furthermore, when bounds [ a , b ] are unknown and the empirical support (i.e. [ min ( x i ) , max ( x i ) ] ) is used in their place, similar behaviour surfaces.

Nonparametric estimation of the shape functions and related asymptotic results

Arriaza et al (Metrika 82:99–124, 2019) introduced the right and left shape functions, which enjoy interesting properties in terms of describing the global form of a distribution. This paper proposes and studies nonparametric estimators of those functions. The estimators involve nonparametric estimation of the quantile and density functions. Pointwise and uniform consistency are proved under general regularity assumptions, as well as the limit in law. Simulations are included to study the practical performance of the proposed estimators. The analysis of a real data set illustrates the methodology.

Neural Networks for Partially Linear Quantile Regression

Deep learning has enjoyed tremendous success in a variety of applications but its application to quantile regression remains scarce. A major advantage of the deep learning approach is its flexibility to model complex data in a more parsimonious way than nonparametric smoothing methods. However, while deep learning brought breakthroughs in prediction, it is not well suited for statistical inference due to its black box nature. In this article, we leverage the advantages of deep learning and apply it to quantile regression where the goal is to produce interpretable results and perform statistical inference. We achieve this by adopting a semiparametric approach based on the partially linear quantile regression model, where covariates of primary interest for statistical inference are modeled linearly and all other covariates are modeled nonparametrically by means of a deep neural network. In addition to the new methodology, we provide theoretical justification for the proposed model by establishing the root-n consistency and asymptotically normality of the parametric coefficient estimator and the minimax optimal convergence rate of the neural nonparametric function estimator. Across several simulated and real data examples, the proposed model empirically produces superior estimates and more accurate predictions than various alternative approaches.

A spline-assisted semiparametric approach to nonparametric measurement error models

A spline-assisted approach is proposed to handle the measurement error problem in treating the pollution and asthma data. It is well known that the minimax rate of convergence in nonparametric regression function estimation of a random variable measured with error is much slower than the rate in the error free case. A different problem is considered. It is shown that if one is willing to impose a relatively mild assumption in requiring that the error-prone variable has a compact support, then standard nonparametric results are obtainable for measurement error models. New and constructive methods to take full advantage of the compact support assumption via spline-assisted semiparametric methods are proposed. It is proven that the new estimator achieves the usual nonparametric rate as if there were no measurement error. Furthermore it is shown that similar spline approach can be used to assess the probability density function on a compact set, using observations with error, and the resulting estimator differs from the true density function only by a constant scale. In addition, it retains the nonparametric density convergence properties of the error free case. The performance of the new methods is demonstrated through simulations and the methods are implemented to analyze the relation between asthma and pollution.