Nonparametric Probability Density Estimation Research Articles

Demand forecasting in power distribution systems using nonparametric probability density estimation

Customer demand data are required by power flow programs to accurately simulate the behavior of electric distribution systems. At present, economic constraints limit widespread customer monitoring, resulting in a need to forecast these demands for distribution system analysis. This paper presents the application of nonparametric probability density estimation to the problem of customer demand forecasting using information readily available at most utilities. The method utilizes demand survey information, including weather conditions, to build a probabilistic demand model that expresses both the random nature of demand and its temperature dependence. The paper describes a procedure for developing such a model and its application for demand forecasting based on customer energy usage and outside temperature.

Function estimation using partitions of unity

We present a new nonparametric function estimate based on partitions of unity. These density estimates are applicable not only to subsets of R n but to arbitrary metric spaces. They can be used for estimating conditional expectations, and for constructing both nonparametric energy density and nonparametric probability density estimates. We show that under reasonable conditions that the nonparametric density estimates converge in mean square error and under slightly more restrictive conditions that they converge in integrated mean square error. A computer graphics application is presented.

Forecasting Abundance from Habitat Measures Using Nonparametric Density Estimation Methods

The relationships between abundance of animals and characteristics of habitats are important for theoretical and management reasons. However, data on populations and habitats frequently are unsuitable for model-based statistical analyses. The shape of the functional relation between abundance and habitat and the form of the error distribution are rarely known accurately, and noise in the data often is large relative to the signal. Kernel estimators, a type of nonparametric probability density estimation technique, can use available data to estimate the probability density function (pdf) of abundance, given specified habitat conditions. One nonparametric estimator, related to kernel estimators and based on the Cauchy distribution, is described in detail. It is applied to data from three studies of influences of habitat on populations: stream characteristics on salmon and trout biomasses, depth of the 2 °C isotherm on codtrap catches, and bottom temperature on capelin densities. Despite the differences among the data sets, the Cauchy kernel could be applied without modification in all cases. The pdf's of abundance were interpretable readily, captured dominant features of the biology of each species, and were suitable for management applications and for tests of hypotheses.

Smoothing Spline Density Estimation: A Dimensionless Automatic Algorithm

Abstract As a sequel to an earlier article by Gu and Qiu, this article describes and illustrates a dimensionless automatic algorithm for nonparametric probability density estimation using smoothing splines. The algorithm is designed to calculate an adaptive finite dimensional solution to the penalized likelihood problem, which was shown by Gu and Qiu to share the same asymptotic convergence rates as the nonadaptive infinite dimensional solution. The smoothing parameter is updated jointly with the estimate in a performance-oriented iteration via a cross-validation performance estimate, where the performance is measured by proxies of the symmetrized Kullback-Leibler distance between the true density and the estimate. Simulations of limited scale are conducted to examine the relative effectiveness of the automatic smoothing parameter selection procedure and to assess the practical statistical performance of the methodology in general. The method is also applied to some real data sets. The algorithm is implemented in a few Ratfor routines, which will be included in later versions of RKPACK for public access.

Smoothing Spline Density Estimation: A Dimensionless Automatic Algorithm

Abstract As a sequel to an earlier article by Gu and Qiu, this article describes and illustrates a dimensionless automatic algorithm for nonparametric probability density estimation using smoothing splines. The algorithm is designed to calculate an adaptive finite dimensional solution to the penalized likelihood problem, which was shown by Gu and Qiu to share the same asymptotic convergence rates as the nonadaptive infinite dimensional solution. The smoothing parameter is updated jointly with the estimate in a performance-oriented iteration via a cross-validation performance estimate, where the performance is measured by proxies of the symmetrized Kullback-Leibler distance between the true density and the estimate. Simulations of limited scale are conducted to examine the relative effectiveness of the automatic smoothing parameter selection procedure and to assess the practical statistical performance of the methodology in general. The method is also applied to some real data sets. The algorithm is implemented in a few Ratfor routines, which will be included in later versions of RKPACK for public access.

Discussion of the Paper by Donoho, Johnstone, Hoch and Stern

Discussion of the Paper by Donoho, Johnstone, Hoch and Stern

Unsupervised tissue type segmentation of 3D dual-echo MR head data

The visualization of 3D phenomena and the extraction of quantitative information from magnetic resonance (MR) image data require efficient semiautomated or automated segmentation techniques. The application of multivariate statistical classification to the segmentation of dual-echo volume data of the human head into tissue types (grey matter, white matter and fluid spaces) is studied in this paper. Tests of the radiometric variability of tissue classes within the data volume demonstrate the improvement of the image acquisition technology and the suitability of statistical methods to perform brain tissue segmentation. Supervised classification is successfully applied to a study of 16 MR volume images of the human head, illustrating the robustness of this method in segmenting brain (white and grey matter) and cerebrospinal fluid (CSF). To avoid subjective criteria involved in the supervised approach, ISODATA clustering as well as clustering based on nonparametric probability density estimation were tested. Both methods performed well (success rates 93.8% and 87.5%, respectively), indicating that the classification procedure can be completely automated. The reproducibility and reliability of supervised and unsupervised classfication were studied by comparing results of segmentation performed by five expert operators. Results suggest that the interoperator and intraoperator variations could be reduced using automated clustering techniques. The accuracy of the volume calculations was quantified by applying the MR imaging and segmentation process to a phantom resembling shape and tissue characteristics of brain tissue. The segmented brain objects are displayed using 3D surface rendering.

Nonparametric probability density estimation: Improvements to the histogram for laboratory data

The histogram has long been used in the clinical laboratory for the depiction and manipulation of frequency data. We present recent results of refinements to the usual histogram procedures along with modern alternative methods of estimating frequency distributions, including the kernel and discrete maximum penalized likelihood estimation (DMPLE) approaches. We compared these nonparametric methods on 15 different types of simulated distributions, and on several sets (>1000 subjects/set) of real data, including alanine aminotransferase, aspartate aminotransferase, and lactate dehydrogenase levels. Each frequency curve estimation technique was evaluated by measuring the integrated mean square error between each technique's prediction and the true underlying distribution, using Monte Carlo techniques on sample sets with size 49 and 119. The kernel methos was the clear method of choice, both in performance (best in 22 36 cases) and in practical usage.

Strong consistency and rates for recursive nonparametric conditional probability density estimates under (α, β)-mixing conditions

Let { X j : j ⩾ 1} be a real-valued stationary process. Recursive kernel estimators of the joint probability density functions, and of conditional probability density functions of X j , given past behavior, are considered. Their strong consistency, along with rates, are given for process { X j ; j ⩾ 1} satisfying (α, β)-mixing conditions. Here, we improve the rates of a.s. convergence in Masry (1987, 1989) without imposing considerably faster rate of decay on the mixing coefficients.

Convergence rates in density estimation for data from infinite-order moving average processes

The effect of long-range dependence in nonparametric probability density estimation is investigated under the assumption that the observed data are a sample from a stationary, infinite-order moving average process. It is shown that to first order, the mean integrated squared error (MISE) of a kernel estimator for moving average data may be expanded as the sum of MISE of the kernel estimator for a same-sizerandom sample, plus a term proportional to the variance of the moving average sample mean. The latter term does not depend on bandwidth, and so imposes a ceiling on the convergence rate of a kernel estimator regardless of how bandwidth is chosen. This ceiling can be quite significant in the case of long-range dependence. We show thatall density estimators have the convergence rate ceiling possessed by kernel estimators.

A Nonparametric Approach to Generating Inseason Forecasts of Salmon Returns

Variations in the size and timing of salmon runs have frustrated attempts to develop models for updating forecasts of returns inseason. The relationship between run size and inseason predictor variables is often nonlinear and changes over time. The nature of these nonlinearities is generally not known and analysts have elected to approximate observed patterns in the data with ad hoc parametric models. Although conceptually attractive, the performance of these models is frequently degraded by their inflexibility and failure to satisfy key model assumptions. In this paper, nonparametric probability density estimation techniques are employed to calculate the total run size conditional on the observed inseason data. Unlike the parametric techniques, the nonparametric approach allows the data to speak for themselves instead of having to merely conform to some arbitrary mathematical model. The approach is easily adapted to handle information from either terminal or guantlet fisheries and can be generalized to compute conditional expectations of various quantities of interest including run size, run timing, or both. Weekly estimates of the number of sockeye salmon (Oncorhynchus nerka) returning to the Skeena River, British Columbia, are employed to demonstrate the performance of the model.

Non-parametric learning algorithms in time-varying environments

A non-parametric recursive algorithm is investigated from the viewpoint of learning properties in a time-varying environment. The algorithm is based on the kernels used in the non-parametric estimation of probability density and regression functions. The procedure possesses tracking properties as the sample size grows large and the conditions for the mean square error convergence and the almost sure convergence are given. Applications to non-parametric learning of time-varying probability densities and non-parametric prediction of time-varying conditional means are discussed. An illustrative simulation example is presented to show that the algorithm converges even if a time-varying function to be tracked diverges. A great advantage of the approach is that it does not require a finite parametrization of unknown time-varying functions. The method is applicable for relatively slow non-stationary changes.

Hypothesis Testing in Semiparametric and Nonparametric Models for Econometric Time Series

A restriction on a semiparametric or nonparametric econometric time series model determines the value of a finite-dimensional functional τ of an infinite-dimensional nuisance function. The estimate of τ and its estimated covariance matrix use nonparametric probability and spectral density estimation. A consequent test of the restriction is given approximate large sample justification under absolute regularity on the time series and other conditions. The methodology relates closely to recent proposals of Powell, Stock, Stoker and Robinson in cross-sectional applications, but serial dependence generally affects the test statistic's form, as well as statistical theory.

Automatic smoothing parameter selection: A survey

This is a survey of recent developments in smoothing parameter selection for curve estimation. The first goal of this paper is to provide an introduction to the methods available, with discussion at both a practical and also a nontechnical level, including comparison of methods. The second goal is to provide access to the literature, especially on smoothing parameter selection, but also on curve estimation in general. The two main settings considered here are nonparametric regression and probability density estimation, although the points made apply to other settings as well. These points also apply to many different estimators, although the focus is on kernel estimators, because they are the most easily understood and motivated, and have been at the heart of the development in the field.

Nonparametric Probability Density Estimation by Discrete Maximum Penalized- Likelihood Criteria

A nonparametric probability density estimator is proposed that is optimal -with respect to a discretized form of a continuous penalized-likelihood criterion functional. Approximation results relating the discrete estimator to the estimate obtained by solving the corresponding infinite-dimensional problem are presented. The discrete estimator is shown to be consistent. The numerical implementation of this discrete estimator is outlined and examples displayed. A simulation study compares the integrated mean square error of the discrete estimator with that of the well-known kernel estimators. Asymptotic rates of convergence of the discrete estimator are also investigated.

Nonparametric detection scheme for myocardial infarction

Bernstein et al.1 have suggested a method for the detection of myocardial infarction using the combined measurement of serum LD activity and inhibition of LD by pyruvate (which depends on the amount of LD from damaged myocardium). This is another in the growing number of applications of discriminant analysis in medical diagnosis. As is often the case, the true underlying distribution of the data is not known. In this case, in particular, an attempt is made at defining the distribution to more accurately assess those patients among whom the diagnosis of myocardial infarction is suspect but is not clearly identified. Tsokos and Welch2 have shown that discriminant procedures based on incorrect assumptions of the underlying distribution led to substantially higher error rates. In this paper, we consider the application of a nonparametric probability density estimator recently developed by David W. Scott.3 This leads to a rather accurate discriminant procedure that is applicable to many other types of data.

A nonparametric classification scheme with mean squared error criterion

A statistical classification scheme for a given set of data requires knowledge of the probability distribution of the observations. Traditional approaches to this problem have revolved around chosen various parametric forms for the probability distribution and evaluating these by goodness of fit methods. Among the difficulties with this method are that it is time consuming, it may not lead to satisfactory results and it may lie beyond the statistical expertise of many practitioners. In this paper, the author's consider the use of a recently developed nonparametric probability density estimator in classification schemes with mean squared error loss criterion. Classical parametric approaches are compared to the nonparametric method on simulated data on the basis of the misclassification probability. Real data from the medical and biological sciences are also used to illustrate the usefulness of the nonparametric method.

Nonparametric Probability Density Estimation: I. A Summary of Available Methods

Click to increase image sizeClick to decrease image sizeKey Words: Nonparametric EstimationDensity EstimationNonparametricPearson SystemGram-CharlierConsistencyKernel EstimatesOrthogonal Series EstimatesMaximum Likelihood EstimatesHistogram