Adaptive Density Estimation Research Articles

Adaptive density estimation for clustering with Gaussian mixtures

Gaussian mixture models are widely used to study clustering problems. These model-based clustering methods require an accurate estimation of the unknown data density by Gaussian mixtures. In Maugis and Michel (2009), a penalized maximum likelihood estimator is proposed for automatically selecting the number of mixture components. In the present paper, a collection of univariate densities whose logarithm is locally {\beta}-H\"older with moment and tail conditions are considered. We show that this penalized estimator is minimax adaptive to the {\beta} regularity of such densities in the Hellinger sense.

Locally Adaptive Density Estimation on the Unit Sphere Using Needlets

The problem of estimating a probability density function f on the (d-1)-dimensional unit sphere S^{d-1} from directional data using the needlet frame is considered. It is shown that the decay of needlet coefficients supported near a point of a function f depends only on local H\"{o}lder continuity properties of f at x. This is then used to show that the thresholded needlet estimator introduced in Baldi, Kerkyacharian, Marinucci and Picard adapts to the local regularity properties of f. Moreover an adaptive confidence interval for f based on the thresholded needlet estimator is proposed, which is asymptotically honest over suitable classes of locally H\"{o}lderian densities.

Adaptive density estimation in the pile-up model involving measurement errors

Motivated by fluorescence lifetime measurements, this paper considers the problem of nonparametric density estimation in the pile-up model, where observations suffer also from measurement errors. In the pile-up model, an observation is defined as the minimum of a random number of i.i.d. variables following the target distribution. Adaptive nonparametric estimators are proposed for this pile-up model with measurement errors. Furthermore, oracle type risk bounds for the mean integrated squared error (MISE) are provided. Finally, the estimation method is assessed by a simulation study and the application to real fluorescence lifetime data.

Regularized tessellation density estimation with bootstrap aggregation and complexity penalization

Locally adaptive density estimation presents challenges for parametric or non-parametric estimators. Several useful properties of tessellation density estimators (TDEs), such as low bias, scale invariance and sensitivity to local data morphology, make them an attractive alternative to standard kernel techniques. However, simple TDEs are discontinuous and produce highly unstable estimates due to their susceptibility to sampling noise. With the motivation of addressing these concerns, we propose applying TDEs within a bootstrap aggregation algorithm, and incorporating model selection with complexity penalization. We implement complexity reduction of the TDE via sub-sampling, and use information-theoretic criteria for model selection, which leads to an automatic and approximately ideal bias/variance compromise. The procedure yields a stabilized estimator that automatically adapts to the complexity of the generating distribution and the quantity of information at hand, and retains the highly desirable properties of the TDE. Simulation studies presented suggest a high degree of stability and sensitivity can be obtained using this approach.

SPADES and mixture models

This paper studies sparse density estimation via ℓ1 penalization (SPADES). We focus on estimation in high-dimensional mixture models and nonparametric adaptive density estimation. We show, respectively, that SPADES can recover, with high probability, the unknown components of a mixture of probability densities and that it yields minimax adaptive density estimates. These results are based on a general sparsity oracle inequality that the SPADES estimates satisfy. We offer a data driven method for the choice of the tuning parameter used in the construction of SPADES. The method uses the generalized bisection method first introduced in [10]. The suggested procedure bypasses the need for a grid search and offers substantial computational savings. We complement our theoretical results with a simulation study that employs this method for approximations of one and two-dimensional densities with mixtures. The numerical results strongly support our theoretical findings.

Adaptive density estimation: A curse of support?

This paper deals with the classical problem of density estimation on the real line. Most of the existing papers devoted to minimax properties assume that the support of the underlying density is bounded and known. But this assumption may be very difficult to handle in practice. In this work, we show that, exactly as a curse of dimensionality exists when the data lie in R d , there exists a curse of support as well when the support of the density is infinite. As for the dimensionality problem where the rates of convergence deteriorate when the dimension grows, the minimax rates of convergence may deteriorate as well when the support becomes infinite. This problem is not purely theoretical since the simulations show that the support-dependent methods are really affected in practice by the size of the density support, or by the weight of the density tail. We propose a method based on a biorthogonal wavelet thresholding rule that is adaptive with respect to the nature of the support and the regularity of the signal, but that is also robust in practice to this curse of support. The threshold, that is proposed here, is very accurately calibrated so that the gap between optimal theoretical and practical tuning parameters is almost filled.

Boundary kernels for adaptive density estimators on regions with irregular boundaries

In some applications of kernel density estimation the data may have a highly non-uniform distribution and be confined to a compact region. Standard fixed bandwidth density estimates can struggle to cope with the spatially variable smoothing requirements, and will be subject to excessive bias at the boundary of the region. While adaptive kernel estimators can address the first of these issues, the study of boundary kernel methods has been restricted to the fixed bandwidth context. We propose a new linear boundary kernel which reduces the asymptotic order of the bias of an adaptive density estimator at the boundary, and is simple to implement even on an irregular boundary. The properties of this adaptive boundary kernel are examined theoretically. In particular, we demonstrate that the asymptotic performance of the density estimator is maintained when the adaptive bandwidth is defined in terms of a pilot estimate rather than the true underlying density. We examine the performance for finite sample sizes numerically through analysis of simulated and real data sets.

GPU‐Assisted High Quality Particle Rendering

AbstractVisualizing dynamic participating media in particle form by fully solving equations from the light transport theory is a computationally very expensive process. In this paper, we present a computational pipeline for particle volume rendering that is easily accelerated by the current GPU. To fully harness its massively parallel computing power, we transform input particles into a volumetric density field using a GPU‐assisted, adaptive density estimation technique that iteratively adapts the smoothing length for local grid cells. Then, the volume data is visualized efficiently based on the volume photon mapping method where our GPU techniques further improve the rendering quality offered by previous implementations while performing rendering computation in acceptable time. It is demonstrated that high quality volume renderings can be easily produced from large particle datasets in time frames of a few seconds to less than a minute.

Adaptive density estimation of stationary β-mixing and τ-mixing processes

We propose an algorithm to estimate the common density $s$ of a stationary process $X_1,...,X_n$. We suppose that the process is either $\beta$ or $\tau$-mixing. We provide a model selection procedure based on a generalization of Mallows' $C_p$ and we prove oracle inequalities for the selected estimator under a few prior assumptions on the collection of models and on the mixing coefficients. We prove that our estimator is adaptive over a class of Besov spaces, namely, we prove that it achieves the same rates of convergence as in the i.i.d framework.

A variable bandwidth selector in multivariate kernel density estimation

Based on a random sample of size n from an unknown d-dimensional density f, the problem of selecting the variable (or adaptive) bandwidth in kernel estimation of f is investigated. The common strategy is to express the variable bandwidth at each observation as the product of a local bandwidth factor and a global smoothing parameter. For selecting the local bandwidth factor a method based on cluster analysis is proposed. This method is direct and intuitively appealing. For selecting the global smoothing parameter a method that is an adaptation of the frequency domain approach of selecting the fixed bandwidth in Wu and Tsai [2004. Root n bandwidths selectors in multivariate kernel density estimation. Probab. Theory Related Fields 129, 537–558] is used. For d = 1 and 2, extensive simulation studies have been done to compare the performance of our selector with the selectors of Abramson [1982. On bandwidth variation in kernel estimates—a square root law. Ann. Statist. 10, 1217–1223] and Sain and Scott [1996. On locally adaptive density estimation. J. Amer. Statist. Assoc. 91, 1525–1534] and Sain [2002. Multivariate locally adaptive density estimation. Comput. Statist. Data Anal. 39, 165–186], and the excellent performance of our selector at practical sample sizes is clearly demonstrated.

Efficient adaptive density estimation per image pixel for the task of background subtraction

We analyze the computer vision task of pixel-level background subtraction. We present recursive equations that are used to constantly update the parameters of a Gaussian mixture model and to simultaneously select the appropriate number of components for each pixel. We also present a simple non-parametric adaptive density estimation method. The two methods are compared with each other and with some previously proposed algorithms.

On Sequential Data-Driven Density Estimation

The theory and methods of minimax and sequential inferences, pioneered by Abraham Wald in 1940's, shaped the way statisticians see the statistics today. This article employs the Wald approaches together with the modern oracle analysis to develop the theory and methods of a sharp minimax adaptive sequential density estimation. In particular, it proves a long-standing conjecture about a sufficient condition for a sharp adaptive sequential estimation with an assigned mean integrated squared error. It also suggests, and then studies via intensive Monte-Carlo simulations, a data-driven sequential density estimator that can be recommended for practical applications.

Adaptive variable location kernel density estimators with good performance at boundaries

This paper introduces new adaptive versions of the variable location density estimator which, for the first time, achieve bias improvement by an order of magnitude at the boundaries, as well as affording the usual higher order bias in the interior of the density support. We develop a general theoretical framework into which both these and earlier versions of adaptive variable location density estimators fit. This enables us to provide a single formula for the higher order biases and variances of these estimators. Numerical work for the comparison of these estimators with each other and with the conventional kernel density estimator reveals good properties of the proposed estimators.

Kernel density estimation with adaptive varying window size

A new method of kernel density estimation with a varying adaptive window size is proposed. It is based on the so-called intersection of confidence intervals (ICI) rule. Several examples of the proposed method are given for different types of densities and the quality of the adaptive density estimate is assessed by means of numerical simulations.

Multivariate locally adaptive density estimation

Multivariate versions of variable bandwidth kernel density estimators can lead to improvement over kernel density estimators using global bandwidth choices. These estimators are more flexible and better able to model complex (multimodal) densities. In this work, two variable bandwidth estimators are discussed: the balloon estimator which varies the smoothing matrix with each estimation point and the sample point estimator which uses a different smoothing matrix for each data point. A binned version of the sample point estimator is developed that, for various situations in low to moderate dimensions, exhibits less error (MISE) than the fixed bandwidth estimator and the balloon estimator. A practical implementation of the sample point estimator is shown through simulation and example to do a better job at reconstructing features of the underlying density than fixed bandwidth estimators. Computational details, including parameterization of the smoothing matrix, are discussed throughout.

Finite sample properties of an adaptive density estimator

Finite sample properties for the adaptive density estimator, studied in e.g. [1], are given. Our approach is numerical and also allows the meaning of the sensitivity parameter, <$>\\alpha<$>, to be studied for different densities. The performance of the adaptive density estimator is compared to the kernel density estimator and the estimator by Hjort and Glad given in [2] for the normal mixture truths introduced in [3]. Our results confirm that the adaptive density estimator is a good estimator in many cases and that the choice frequently is a good one.

Numerical results concerning a sharp adaptive density estimator

We present here a simulation study of the behavior of a particular kernel density estimator. It was previously proven that this nonparametric estimator is sharp in the sense of the minimax adaptive theory, which means that it is equally well performing for very smooth or unsmooth densities. The method selects locally both the bandwidth and the kernel function according to the evaluated smoothness of the underlying density. In this paper we describe the method and apply it successfully to i.i.d. simulated data of different probability densities.

Mixing strategies for density estimation

General results on adaptive density estimation are obtained with respect to any countable collection of estimation strategies under Kullback-Leibler and squared $L_2$ losses. It is shown that without knowing which strategy works best for the underlying density, a single strategy can be constructed by mixing the proposed ones to be adaptive in terms of statistical risks. A consequence is that under some mild conditions, an asymptotically minimax-rate adaptive estimator exists for a given countable collection of density classes; that is, a single estimator can be constructed to be simultaneously minimax-rate optimal for all the function classes being considered. A demonstration is given for high-dimensional density estimation on $[0,1]^d$ where the constructed estimator adapts to smoothness and interaction-order over some piecewise Besov classes and is consistent for all the densities with finite entropy.

Neural tree density estimation for novelty detection

In this paper, a neural competitive learning tree is introduced as a computationally attractive scheme for adaptive density estimation and novelty detection. The learning rule yields equiprobable quantization of the input space and provides an adaptive focusing mechanism capable of tracking time-varying distributions. It is shown by simulation that the neural tree performs reasonably well while being much faster than any of the other competitive learning algorithms.

Formula omitted] adaptive density estimation in a β mixing framework

We study the L π − i n t e g r a t e d risk with π ≥ 2 of an adaptive density estimator by wavelets method for absolutely regular observations. By a duality argument, the study of the risk is linked to the control of the supremum of the empirical process over a suitable class of functions. The main argument is a generalization to absolutely regular variables of a result of Talagrand stated for i.i.d. variables. Assuming that the sequence of the β-mixing coefficients ( β l ) l≥0 is arithmetically decreasing, we prove that our estimator is adaptive in a class of Besov spaces with unknown smoothness.