Nonparametric Density Estimation Research Articles

A class of Birnbaum–Saunders type kernel density estimators for nonnegative data

Nonparametric density estimation using a class of deformed skew Birnbaum–Saunder (BS) type kernels is suggested for nonnegative data. A remarkable feature of new skew BS type kernel density estimators lies in its general formulation via asymmetry parameter as well as density generator. Mean integrated squared errors of the proposed estimators are investigated, together with strong consistency and asymptotic normality. Simulation studies and applications to real data sets are presented.

Short-Term Forecasting and Uncertainty Analysis of Wind Power

Abstract Accurate forecasting is the key factor in promoting wind power consumption and improving the stable operation of power systems. A short-term wind power forecasting (WPF) and uncertainty analysis method based on whale optimization algorithm (WOA), least squares support vector machine (LSSVM), and nonparametric kernel density estimation (NPKDE) was proposed in this paper. The advantages of WOA (fast convergence speed and high convergence accuracy) were used to optimize the penalty factor and kernel function width of the LSSVM model, and the calculation speed and forecasting accuracy of the LSSVM model were improved. The training sample set is classified according to the wind speed interval, and the WOA-LSSVM forecasting model is trained by subclass after classification to further improve the accuracy of short-term WPF. The NPKDE method is used to accurately calculate the probability density distribution characteristics of the forecasting error of wind power, and the confidence interval of the WPF is accurately calculated based on the probability density distribution characteristics. The calculation results show that the forecasting accuracy of the WOA-LSSVM model is higher than those of the LSSVM, long short-term memory (LSTM), and particle swarm optimization and least squares support vector machine (PSO-LSSVM) models, and the forecasting accuracy of the WOA-LSSVM model can be further improved after classifying the training sample set. The coverage of the confidence intervals in different time scales is higher than the corresponding confidence level, indicating that the NPKDE method can accurately describe the probability density distribution characteristics of the WPF errors.

Weighted rank estimation of nonparametric transformation models with case-1 and case-2 interval-censored failure time data

Case-1 and case-2 interval-censored failure time data commonly occur in medical research as well as other fields and many methods have been developed for their analysis under different frameworks. In this paper, we consider regression analysis of such data and present a general class of nonparametric transformation models. One major advantage of these models is their flexibility and generality as they include the linear transformation model as a special case. For estimation of regression parameters, we propose a weighted rank (WR) estimation procedure and establish the consistency and asymptotic normality of the resulting estimator. Furthermore, to estimate the asymptotic covariance matrix of the proposed estimator, a resampling technique, which does not involve nonparametric density estimation or numerical derivatives, is developed. A numerical study is also conducted and suggests that the proposed methodology works well in practice. Finally an application is provided.

Recursive asymmetric kernel density estimation for nonnegative data

Recursive nonparametric density estimation for nonnegative data is considered, using an asymmetric kernel with nonnegative support. It has a computational advantage in a situation where a huge number of data are sequentially collected. The recursive asymmetric kernel estimator keeps desirable asymptotic properties similar to the ordinary non-recursive asymmetric kernel estimator. Also, simulation studies and a real data analysis are performed for illustration.

High-dimensional nonparametric density estimation via symmetry and shape constraints

We tackle the problem of high-dimensional nonparametric density estimation by taking the class of log-concave densities on Rp and incorporating within it symmetry assumptions, which facilitate scalable estimation algorithms and can mitigate the curse of dimensionality. Our main symmetry assumption is that the super-level sets of the density are K-homothetic (i.e., scalar multiples of a convex body K⊆Rp). When K is known, we prove that the K-homothetic log-concave maximum likelihood estimator based on n independent observations from such a density achieves the minimax optimal rate of convergence with respect to, for example, squared Hellinger loss, of order n−4/5, independent of p. Moreover, we show that the estimator is adaptive in the sense that if the data generating density admits a special form, then a nearly parametric rate may be attained. We also provide worst case and adaptive risk bounds in cases where K is only known up to a positive definite transformation, and where it is completely unknown and must be estimated nonparametrically. Our estimation algorithms are fast even when n and p are on the order of hundreds of thousands, and we illustrate the strong finite-sample performance of our methods on simulated data.

Extrapolation-based tuning parameters selection in massive data analysis

Many statistical modeling procedures involve one or more tuning parameters tocontrol the model complexity. These tuning parameters can be the bandwidth in thekernel smoothing method in the nonparametric regression and density estimation orbe the regularization parameter in the regularization method for feature selectionin the high dimensional modeling. Tuning parameter selection plays critical rolesin the statistical modeling and machine learning. For the massive data analysis,commonly-used methods such as grid-point search with information criteriabecome prohibitively costly in computation. Their feasibility isquestionable even with modern parallel computing platforms.This paper aims to develop a fast algorithm to efficientlyapproximate the best tuning parameters. The algorithm entails (a) assuming aparametric model to describe the trend between the best tuning parameters andsample sizes, (b) establishing the trend via fitting the model with subsamplingdata, and (c) extrapolating this trend to the case of huge sample size. Todetermine the subsampling sample sizes to be taken, we derive optimaldesigns for settings that allow a constraint on the budget of total computational cost.We show that the proposed designs possess an asymptotic optimality 性质.Our numerical studies demonstrate that with a simple two-parameter polynomial model, the proposed algorithm performsalmost equivalently to the procedure using the full data setin several different statistical settings, while ithas a significant reduction in computing time and storage.

Nonparametric Estimation of Multivariate Density and its Derivative by Dependent Data Using Gamma Kernels

We consider the nonparametric estimation of the multivariate probability density function and its partial derivative with a support on nonnegative axis by dependent data. We use the class of kernel estimators with asymmetric gamma kernel functions. The gamma kernels are nonnegative; they may change their shape depending on the position on the semi-axis and possess good boundary properties for a wide class of densities. Asymptotic estimates of the multivariate density and of its partial derivatives such as biases, variances, and covariances are derived. The optimal bandwidth of both estimates is obtained as a minimum of the mean integrated squared error (MISE) by dependent data with a strong mixing. Optimal convergence rates of the MISE both for the density and its derivative are found.

Nonparametric density estimation and bandwidth selection with B-spline bases: A novel Galerkin method

A general and efficient nonparametric density estimation procedure for local bases, including B-splines, is proposed, which employs a novel statistical Galerkin method combined with basis duality theory. To select the bandwidth, an efficient cross-validation procedure is introduced, based on closed-form expressions in terms of the primal and dual B-spline basis. By utilizing a closed-form expression for the dual basis, the least-squares cross validation formula is calculated in closed-form, enabling an efficient estimation of the optimal bandwidth. The full computational procedure achieves optimal complexity, and is very accurate in comparison with existing estimation procedures, including state-of-the-art kernel density estimators. The presented theoretical results are supported by extensive numerical experiments, which demonstrate the efficiency and accuracy of the new methodology. This new approach provides a complete and optimally efficient framework for density estimation with a B-spline basis, based on simple and elegant closed-form estimators with theoretical convergence results that are substantiated in numerical experiments.

The Novel Successive Variational Mode Decomposition and Weighted Regularized Extreme Learning Machine for Fault Diagnosis of Automobile Gearbox

In order to improve the diagnosis accuracies of the current diagnosis methods, a novel fault diagnosis method of automobile gearbox based on novel successive variational mode decomposition and weighted regularized extreme learning machine is presented for fault diagnosis of gearbox in this paper. The novel successive variational mode decomposition (SVMD) is presented to improve the traditional variational mode decomposition, which finds modes one after the other, and this succession helps increase convergence rate and also not extract the unwanted modes; weighted regularized extreme learning machine (WRELM) is presented to improve the traditional extreme learning machine, which uses the weight of each sample with the nonparametric kernel density estimation and can find the optimal weight for each sample. The test results indicate that the diagnosis accuracy of SVMD-WRELM for gearbox is better than that of VMD-WRELM, VMD-ELM.

Density estimation using bootstrap quantile variance and quantile-mean covariance

We evaluate two density estimators based on the quantile variance and the quantile-mean covariance estimated by bootstrap. We review previous developments on density estimation related to quantiles. Monte Carlo simulations for different data generating processes, sample sizes, and other parameters show that the estimators perform well in comparison to the benchmark non-parametric kernel density estimator. Some of the explored smoothing techniques present lower bias and mean integrated squared errors, which indicates that the proposed estimator is a promising strategy.

A robust estimation method for the linear regression model parameters with correlated error terms and outliers

Independence of error terms in a linear regression model, often not established. So a linear regression model with correlated error terms appears in many applications. According to the earlier studies, this kind of error terms, basically can affect the robustness of the linear regression model analysis. It is also shown that the robustness of the parameters estimators of a linear regression model can stay using the M-estimator. But considering that, it acquires this feature as the result of establishment of its efficiency. Whereas, it has been shown that the minimum Matusita distance estimators, has both features robustness and efficiency at the same time. On the other hand, because the Cochrane and Orcutt adjusted least squares estimators are not affected by the dependence of the error terms, so they are efficient estimators. Here we are using of a non-parametric kernel density estimation method, to give a new method of obtaining the minimum Matusita distance estimators for the linear regression model with correlated error terms in the presence of outliers. Also, simulation and real data study both are done for the introduced estimation method. In each case, the proposed method represents lower biases and mean squared errors than the other two methods.

Near-Linear Time Local Polynomial Nonparametric Estimation with Box Kernels

Summary of Contribution: Big data analytics has become essential for modern operations research and operations management applications. Statistics methods, such as nonparametric density and function estimation, play important roles in predictive and exploratory data analysis for economics and operations management problems. In this paper, we concentrate on efficiently computing local polynomial regression estimates. We significantly accelerate the computation of such local polynomial estimates by novel applications of multidimensional binary indexed trees ( Fenwick 1994 ) and lazy memory allocation via hashing. Both time and space complexity of our proposed algorithm are nearly linear in the number of inputs. Simulation results confirm the efficiency and effectiveness of our proposed methods.

Automatic topography of high-dimensional data sets by non-parametric density peak clustering

Data analysis in high-dimensional spaces aims at obtaining a synthetic description of a data set, revealing its main structure and its salient features. We here introduce an approach providing this description in the form of a topography of the data, namely a human-readable chart of the probability density from which the data are harvested. The approach is based on an unsupervised extension of Density Peak clustering and on a non-parametric density estimator that measures the probability density in the manifold containing the data. This allows finding automatically the number and the height of the peaks of the probability density, and the depth of the “valleys” separating them. Importantly, the density estimator provides a measure of the error, which allows distinguishing genuine density peaks from density fluctuations due to finite sampling. The approach thus provides robust and visual information about the density peaks height, their statistical reliability and their hierarchical organization, offering a conceptually powerful extension of the standard clustering partitions. We show that this framework is particularly useful in the analysis of complex data sets.

Multiscale stick-breaking mixture models

Bayesian nonparametric density estimation is dominated by single-scale methods, typically exploiting mixture model specifications, exception made for Pólya trees prior and allied approaches. In this paper we focus on developing a novel family of multiscale stick-breaking mixture models that inherits some of the advantages of both single-scale nonparametric mixtures and Pólya trees. Our proposal is based on a mixture specification exploiting an infinitely deep binary tree of random weights that grows according to a multiscale generalization of a large class of stick-breaking processes; this multiscale stick-breaking is paired with specific stochastic processes generating sequences of parameters that induce stochastically ordered kernel functions. Properties of this family of multiscale stick-breaking mixtures are described. Focusing on a Gaussian specification, a Markov Chain Monte Carlo algorithm for posterior computation is introduced. The performance of the method is illustrated analyzing both synthetic and real datasets consistently showing competitive results both in scenarios favoring single-scale and multiscale methods. The results suggest that the method is well suited to estimate densities with varying degree of smoothness and local features.

Nonparametric Density Estimation and Bandwidth Selection with B-spline bases: a Novel Galerkin Method

We study a general and efficient nonparametric density estimation procedure for local bases, including B-splines, using a novel statistical Galerkin method, combined with basis duality theory. We provide an efficient cross-validation procedure to select the bandwidth, based on closed-form expressions in terms of the primal and dual basis. By utilizing a closed-form expression for the dual basis, we are able to compute the least-squares cross validation formula in closed-form, which allows us to effectively determine the bandwidth. The full computational procedure achieves optimal complexity, and is very accurate in comparisons with existing estimation procedures, including state-of-the-art kernel density estimators. Our theoretical results are supported by extensive numerical experiments, which demonstrate the efficiency and accuracy of the new methodology.

Consider the Risk Constraints of Hydro-Thermal Power Generation in Real-Time Strategy of Control

Based on the problem of abandoned water and insufficient output caused by runoff randomness, the short-term runoff inflow interval prediction method combined with nonparametric kernel density estimation (NKDE)and Monte Carlo (MC)sampling is proposed. The method analyzes the impact of hydropower station abandonment risk and insufficient output risk caused by inflow uncertainty on hydropower station output and thermal power output in the power grid, and the corresponding output adjustment strategy is given. This paper mainly carries out these studies: real-time load distribution, short-term runoff interval prediction and real-time output adjustment of hydro-thermal power plant under uncertain incoming water: (1) Risk factors of water and electricity may be involved in the scheduling process, the process of considering runoff uncertainty on hydropower unit output, analyze the impact of hydropower abandoned water risk, estimate based on short-term nonparametric kernel density estimation (NKDE) runoff forecasting methods, given the range of the forecast runoff, reducing forecast error runoff, combine Monte Carlo (MC) sampling and propose a new real-time power control adjustment strategy. (2) Taking Qingjiang cascade power station as a case study, considering the increase of incoming water and the decrease of incoming water, the impact of water abandonment risk and insufficient output risk on short-term power generation plan is verified. The results show that the proposed hydropower and thermal power output adjustment can effectively deal with the risk of abandoned water and the risk of insufficient output. It is of positive significance to make full use of hydropower resources. It is helpful for the safe and stable operation of Hubei Power Grid.

A Suggested Nonparametric Bivariate Logistic Density Estimator with Application on the Productivity of Egyptian Wheat during 2019/2020

In this study, the nonparametric standard logistic density estimator, introduced by Abo-El-Hadid (2018), is extended to the bivariate case. The multiplicative standard logistic distribution is used as a kernel function to derive the bivariate kernel estimator. The statistical properties of the resulting estimator are studied, which are: The asymptotic bias, variance, Mean Squared Error (MSE) and Integrated Mean Squared Error (IMSE); also, the optimal bandwidth is obtained. A simulation study is introduced to investigate the performance of the proposed estimator with other estimators. We also apply the proposed estimator to a real data set to estimate the bivariate density of the planted and productive areas of wheat in Egypt.

On density estimation at a fixed point under local differential privacy

We consider non-parametric density estimation in the framework of local, both pure and approximate, differential privacy. In contrast to centralized privacy scenarios with a trusted curator, in the local setup anonymization must be guaranteed already on the individual data owners’ side and must therefore precede any data mining tasks. Thus, the published anonymized data should be compatible with as many statistical procedures as possible. We consider different mechanisms to establish pure and approximate differential privacy, respectively. We obtain minimax type results over Sobolev classes indexed by a smoothness parameter s>1∕2 for the mean squared error at a fixed point. In particular, we show that appropriately defined kernel density estimators can attain the optimal rate of convergence if the bandwidth parameter is correctly specified. Notably, the optimal convergence rate in terms of the sample size n is n−(2s−1)∕(2s+1) under pure differential privacy and thus deteriorated to the rate n−(2s−1)∕(2s) which holds both without privacy restrictions and under approximate differential privacy. Since the optimal choice of the bandwidth parameter depends on the smoothness s and is thus not accessible in practise, adaptive methods for bandwidth selection are necessary and must, in the local privacy framework, be performed based on the anonymized data only. We address this problem by means of variants of Lepski’s method tailored to the privacy setups at hand and obtain general oracle inequalities for private kernel density estimators. In the Sobolev case, the resulting adaptive estimators attain the optimal rates of convergence at least up to logarithmic factors. On the side, we discuss some critical issues related with the notion of approximate differential privacy.

Asymptotic normality of conditional density and conditional mode in the functional single index model

The main objective of this paper is to investigate the nonparametric estimation of the conditional density of a scalar response variable Y, given the explanatory variable X taking value in a Hilbert space when the sample of observations is considered as an independent random variables with identical distribution (i.i.d) and are linked with a single functional index structure. First of all, a kernel type estimator for the conditional density function (cond-df) is introduced. Afterwards, the asymptotic properties are stated for a conditional density estimator when the observations are linked with a single-index structure from which one derives a central limit theorem (CLT) of the conditional density estimator to show the asymptotic normality of the kernel estimate of this model. As an application the conditional mode in functional single-index model is presented, and the asymptotic (1 – ζ) confidence interval of the conditional mode function is given for 0

Spectral cut-off regularisation for density estimation under multiplicative measurement errors

We study the non-parametric estimation of an unknown density f with support on R+ based on an i.i.d. sample with multiplicative measurement errors. The proposed fully-data driven procedure is based on the estimation of the Mellin transform of the density f, a regularisation of the inverse of the Mellin transform by a spectral cut-off and a data-driven model selection in order to deal with the upcoming bias-variance trade-off. We introduce and discuss further Mellin-Sobolev spaces which characterize the regularity of the unknown density f through the decay of its Mellin transform. Additionally, we show minimax-optimality over Mellin-Sobolev spaces of the data-driven density estimator and hence its adaptivity.