Problem Of Density Estimation Research Articles

Nonparametric Mixtures Based on Skew-normal Distributions: An Application to Density Estimation

This article addresses the density estimation problem using nonparametric Bayesian approach. It is considered hierarchical mixture models where the uncertainty about the mixing measure is modeled using the Dirichlet process. The main goal is to build more flexible models for density estimation. Extensions of the normal mixture model via Dirichlet process previously introduced in the literature are twofold. First, Dirichlet mixtures of skew-normal distributions are considered, say, in the first stage of the hierarchical model, the normal distribution is replaced by the skew-normal one. We also assume a skew-normal distribution as the center measure in the Dirichlet mixture of normal distributions. Some important results related to Bayesian inference in the location-scale skew-normal family are introduced. In particular, we obtain the stochastic representations for the full conditional distributions of the location and skewness parameters. The algorithm introduced by MacEachern and Müller in 1998 is used to sample from the posterior distributions. The models are compared considering simulated data sets. Finally, the well-known Old Faithful Geyser data set is analyzed using the proposed models and the Dirichlet mixture of normal distributions. The model based on Dirichlet mixture of skew-normal distributions captured the data bimodality and skewness shown in the empirical distribution.

The Lower Bound of Density Estimation for Biased Data in Sobolev Spaces

In this paper, we consider the density estimation problem from independent and identically distributed (i.i.d.) biased observations. We study the lower bound of convergence rates of density estimation over Sobolev spaces WrN(NN+) under the Lp risk by using Fanos lemma.

Modeling with Normalized Random Measure Mixture Models

The Dirichlet process mixture model and more general mixtures based on discrete random probability measures have been shown to be flexible and accurate models for density estimation and clustering. The goal of this paper is to illustrate the use of normalized random measures as mixing measures in nonparametric hierarchical mixture models and point out how possible computational issues can be successfully addressed. To this end, we first provide a concise and accessible introduction to normalized random measures with independent increments. Then, we explain in detail a particular way of sampling from the posterior using the Ferguson-Klass representation. We develop a thorough comparative analysis for location-scale mixtures that considers a set of alternatives for the mixture kernel and for the nonparametric component. Simulation results indicate that normalized random measure mixtures potentially represent a valid default choice for density estimation problems. As a byproduct of this study an R package to fit these models was produced and is available in the Comprehensive R Archive Network (CRAN).

On adaptive minimax density estimation on $$R^d$$

We address the problem of adaptive minimax density estimation on $$\mathbb{R }^d$$ with $$\mathbb{L }_p$$ -loss on the anisotropic Nikol’skii classes. We fully characterize behavior of the minimax risk for different relationships between regularity parameters and norm indexes in definitions of the functional class and of the risk. In particular, we show that there are four different regimes with respect to the behavior of the minimax risk. We develop a single estimator which is (nearly) optimal in order over the complete scale of the anisotropic Nikol’skii classes. Our estimation procedure is based on a data-driven selection of an estimator from a fixed family of kernel estimators.

Nonlinear wavelet density estimation for biased data in Sobolev spaces

In this paper, we consider the density estimation problem from independent and identically distributed (i.i.d.) biased observations. We develop an adaptive wavelet hard thresholding rule and evaluate its performance by considering risk over Sobolev balls. We prove that our estimation attains a sharp rate of convergence and show the optimality. MSC:49K40, 90C29, 90C31.

Nonparametric Least Squares Mixture Density Estimation

In this paper, we consider using nonparametric mixtures for density estimation. The mixture density estimation problem simply reduces to the problem of estimating a mixing distribution in the nonparametric mixture model. We focus on the least squares method for mixture density estimation problem. In a simulation experiment, the performance of the least squares mixture density estimator (MDE) and the kernel density estimator (KDE) is assessed by the mean integrated squared error. The performance improvement of MDE over KDE for some common densities is achieved by using cross-validation method for bandwidth selection.

Probability density estimation with surrogate data and validation sample

The probability density estimation problem with surrogate data and validation sample is considered. A regression calibration kernel density estimator is defined to incorporate the information contained in both surrogate variates and validation sample. Also, we define two weighted estimators which have less asymptotic variances but have bigger biases than the regression calibration kernel density estimator. All the proposed estimators are proved to be asymptotically normal. And the asymptotic representations for the mean squared error and mean integrated square error of the proposed estimators are established, respectively. A simulation study is conducted to compare the finite sample behaviors of the proposed estimators.

Estimation of a Density Using Real and Artificial Data

Let <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">X</i> , <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">X</i> <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sub> , <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">X</i> <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sub> , ... be independent and identically distributed R <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">d</i> -valued random variables and let <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">m</i> : R <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">d</i> → R be a measurable function such that a density <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">f</i> of <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Y</i> = <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">m</i> ( <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">X</i> ) exists. Given a sample of the distribution of ( <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">X</i> , <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Y</i> ) and additional independent observations of <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">X</i> , we are interested in estimating <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">f</i> . We apply a regression estimate to the sample of ( <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">X</i> , <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Y</i> ) and use this estimate to generate additional artificial observations of <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Y</i> . Using these artificial observations together with the real observations of <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Y</i> , we construct a density estimate of <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">f</i> by using a convex combination of two kernel density estimates. It is shown that if the bandwidths satisfy the usual conditions and if in addition the supremum norm error of the regression estimate converges almost surely faster toward zero than the bandwidth of the kernel density estimate applied to the artificial data, then the convex combination of the two density estimates is <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">L</i> <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sub> -consistent. The performance of the estimate for finite sample size is illustrated by simulated data, and the usefulness of the procedure is demonstrated by applying it to a density estimation problem in a simulation model.

Posterior consistency in conditional distribution estimation

A wide variety of priors have been proposed for nonparametric Bayesian estimation of conditional distributions, and there is a clear need for theorems providing conditions on the prior for large support, as well as posterior consistency. Estimation of an uncountable collection of conditional distributions across different regions of the predictor space is a challenging problem, which differs in some important ways from density and mean regression estimation problems. Defining various topologies on the space of conditional distributions, we provide sufficient conditions for posterior consistency focusing on a broad class of priors formulated as predictor-dependent mixtures of Gaussian kernels. This theory is illustrated by showing that the conditions are satisfied for a class of generalized stick-breaking process mixtures in which the stick-breaking lengths are monotone, differentiable functions of a continuous stochastic process. We also provide a set of sufficient conditions for the case where stick-breaking lengths are predictor independent, such as those arising from a fixed Dirichlet process prior.

Asymptotic Behavior of the Kernel Density Estimator from a Geometric Viewpoint

From the view of a geometric approach, we consider the problem of density estimation on the m-dimensional unit sphere by using the kernel method. The definition of the kernel estimator is motivated from the concept of the exponential map. This article shows that the asymptotic behavior of the estimator contains a geometric quantity (the sectional curvature) on the unit sphere. This implies that the behavior depends on whether the sectional curvature is positive or negative. Using observed data on normals to the orbital planes of long-period comets, numerical examples on the two-dimensional unit sphere are given.

On density estimation with superkernels

In this article, we consider the problem of nonparametric density estimation in the case, when the original sample has a large size, but the data are given in a binned form, i.e. in the form of a histogram. Such situations are typical for many physical problems, in particular, in scanning electron microscopy and electron beam lithography. We study how superkernels can be used in such situations. It is shown that superkernels can be essentially superior over conventional kernels not only for very smooth densities. The problem of bandwidth and bin width selection is also considered.

Vehicular Traffic Density State Estimation Based on Cumulative Road Acoustics

This paper considers the problem of vehicular traffic density estimation, utilizing the information cues present in the cumulative acoustic signal acquired from a roadside-installed single microphone. This cumulative signal comprises several noise signals such as tire noise, engine noise, engine-idling noise, occasional honks, and air turbulence noise of multiple vehicles. The occurrence and mixture weightings of these noise signals are determined by the prevalent traffic density conditions on the road segment. For instance, under a free-flowing traffic condition, the vehicles typically move with medium to high speeds and thereby produce mainly tire noise and air turbulence noise and less engine-idling noise and honks. For slow-moving congested traffic, the cumulative signal will largely be dominated by engine-idling noise and honks; air turbulence and tire noises will be inconspicuous. Furthermore, these various noise signals have spectral content that are very different from each other and, hence, can be used to discriminate between the different traffic density states that lead to them. Therefore, in this work, we extract the short-term spectral envelope features of the cumulative acoustic signals and model their class-conditional probability distributions, conditioned on one of the three broad traffic density states, i.e., Jammed (0-10 km/h), Medium-Flow (10-40 km/h), and Free-Flow (40 km/h and above) traffic. While these states are coarse measures of the average traffic speed, they nevertheless can provide useful traffic density information in the often-chaotic and nonlane-driven traffic conditions of the developing geographies, where other techniques (magnetic loop detectors) are inapplicable. Based on these learned distributions, we use a Bayes' classifier to classify the acoustic signal segments spanning a duration of 5-30 s, which results in a high classification accuracy of ~95%. Using a discriminative classifier such as a support vector machine (SVM) results in further classification accuracy gains over the Bayes' classifier.

Smooth and Semismooth Newton Methods for Constrained Approximation and Estimation

In the article, we show that the constrained L 2 approximation problem, the positive polynomial interpolation, and the density estimation problems can all be reformulated as a system of smooth or semismooth equations by using Lagrange duality theory. The obtained equations contain integral functions of the same form. The differentiability or (strong) semismoothness of the integral functions and the Hölder continuity of the Jacobian of the integral function were investigated. Then a globalized Newton-type method for solving these problems was introduced. Global convergence and numerical tests for estimating probability density functions with wavelet basis were also given. The research in this article not only strengthened the theoretical results in literatures but also provided a possibility for solving the probability density function estimation problem by Newton-type method.

Catching up Faster by Switching Sooner: A Predictive Approach to Adaptive Estimation with an Application to the AIC–BIC Dilemma

SummaryPrediction and estimation based on Bayesian model selection and model averaging, and derived methods such as the Bayesian information criterion BIC, do not always converge at the fastest possible rate. We identify the catch-up phenomenon as a novel explanation for the slow convergence of Bayesian methods, which inspires a modification of the Bayesian predictive distribution, called the switch distribution. When used as an adaptive estimator, the switch distribution does achieve optimal cumulative risk convergence rates in non-parametric density estimation and Gaussian regression problems. We show that the minimax cumulative risk is obtained under very weak conditions and without knowledge of the underlying degree of smoothness. Unlike other adaptive model selection procedures such as the Akaike information criterion AIC and leave-one-out cross-validation, BIC and Bayes factor model selection are typically statistically consistent. We show that this property is retained by the switch distribution, which thus solves the AIC–BIC dilemma for cumulative risk. The switch distribution has an efficient implementation. We compare its performance with AIC, BIC and Bayesian model selection and averaging on a regression problem with simulated data.

Reconstructing High Dimensional Dynamic Distributions from Distributions of Lower Dimension

We propose a new procedure for estimating a dynamic joint distribution of a group of assets in a sequential manner starting from univariate marginals, continuing with pairwise bivariate distributions, then with triplewise trivariate distributions, etc., until the joint distribution for the whole group is constructed. The procedure uses principles and ideas from the copula theory in how at each step to arrive at a higher dimensional distribution utilizing the results from previous steps. The proposed procedure trades the dimensionality of the parameter space for numerous simpler estimations: even though there are more optimization problems to solve, each is of much lower dimension than the joint density estimation problem; in addition, the parameterization tends to be much more flexible. The paper demonstrates how to apply the new sequential technique to model a dynamic distribution of five DJIA constituents.

Nonparametric estimation of density under bias and multiplicative censoring via wavelet methods

The density estimation problem under bias and multiplicative censoring is considered. Adopting the wavelet approach, we construct a linear nonadaptive estimator and a nonlinear adaptive estimator. The adaptive one belongs to the family of the hard thresholding estimators. We evaluate their performances by determining upper bounds of the mean integrated squared error over a wide range of functions. Sharp upper bounds are obtained.

Adaptive density estimation in the pile-up model involving measurement errors

Motivated by fluorescence lifetime measurements, this paper considers the problem of nonparametric density estimation in the pile-up model, where observations suffer also from measurement errors. In the pile-up model, an observation is defined as the minimum of a random number of i.i.d. variables following the target distribution. Adaptive nonparametric estimators are proposed for this pile-up model with measurement errors. Furthermore, oracle type risk bounds for the mean integrated squared error (MISE) are provided. Finally, the estimation method is assessed by a simulation study and the application to real fluorescence lifetime data.

A Greedy Algorithm for Unimodal Kernel Density Estimation by Data Sharpening

We consider the problem of nonparametric density estimation where estimates are constrained to be unimodal. Though several methods have been proposed to achieve this end, each of them has its own drawbacks and none of them have readily-available computer codes. The approach of Braun and Hall (2001), where a kernel density estimator is modified by data sharpening, is one of the most promising options, but optimization difficulties make it hard to use in practice. This paper presents a new algorithm and MATLAB code for finding good unimodal density estimates under the Braun and Hall scheme. The algorithm uses a greedy, feasibility-preserving strategy to ensure that it always returns a unimodal solution. Compared to the incumbent method of optimization, the greedy method is easier to use, runs faster, and produces solutions of comparable quality. It can also be extended to the bivariate case.

Adaptive normal reference bandwidth based on quantile for kernel density estimation

Bandwidth selection is an important problem of kernel density estimation. Traditional simple and quick bandwidth selectors usually oversmooth the density estimate. Existing sophisticated selectors usually have computational difficulties and occasionally do not exist. Besides, they may not be robust against outliers in the sample data, and some are highly variable, tending to undersmooth the density. In this paper, a highly robust simple and quick bandwidth selector is proposed, which adapts to different types of densities.

A particle swarm algorithm with broad applicability in shape-constrained estimation

In nonparametric function estimation, the inclusion of shape constraints can confer several advantages, including improved estimation accuracy, reduced sensitivity to smoothing parameters, and control over the qualitative appearance of the estimate. Finding shape-restricted estimates may require solving a difficult optimization problem, however, making these advantages hard to realize. A particle swarm algorithm is proposed to overcome this barrier and expand the possibilities for shape-constrained estimation. The algorithm uses a cooperative search strategy with two swarms, one focused on global exploration and one focused on local exploitation. The new heuristic has the added advantage of being a general tool, applicable without modification to a variety of estimators, constraints, and objective functions. The algorithm is demonstrated on a number of density estimation and regression problems, and the potential for further improvement is discussed. Supplementary materials, including source code, are available online.