Malliavin calculus for the optimal estimation of the invariant density of discretely observed diffusions in intermediate regime

We aim at estimating in a non-parametric way the density π of the stationary distribution of a d-dimensional stochastic differential equation , for , from the discrete observations of a finite sample ,…, with . We propose a kernel density estimator and we study its convergence rates for the pointwise estimation of the invariant density under anisotropic Hölder smoothness constraints. First of all, we find some conditions on the discretization step that ensures it is possible to recover the same rates as if the continuous trajectory of the process was available. As proven in the recent work [Amorino C, Gloter A. Minimax rate of estimation for invariant densities associated to continuous stochastic differential equations over anisotropic Holder classes; 2021. arXiv preprint arXiv:2110.02774], such rates are optimal and new in the context of density estimator. Then we deal with the case where such a condition on the discretization step is not satisfied, which we refer to as the intermediate regime. In this new regime we identify the convergence rate for the estimation of the invariant density over anisotropic Hölder classes, which is the same convergence rate as for the estimation of a probability density belonging to an anisotropic Hölder class, associated to n iid random variables . After that we focus on the asynchronous case, in which each component can be observed at different time points. Even if the asynchronicity of the observations complexifies the computation of the variance of the estimator, we are able to find conditions ensuring that this variance is comparable to the one of the continuous case. We also exhibit that the non-synchronicity of the data introduces additional bias terms in the study of the estimator.

Learning density estimation is important in probabilistic modeling and reasoning with uncertainty. Since B-spline basis functions are piecewise polynomials with local support, density estimation with B-splines shows its advantages when intensive numerical computations are involved in the subsequent applications. To obtain an optimal local density estimation with B-splines, we need to select the bandwidth (i.e., the distance of two adjacent knots) for uniform B-splines. However, the selection of bandwidth is challenging, and the computation is costly. On the other hand, nonuniform B-splines can improve on the approximation capability of uniform B-splines. Based on this observation, we perform density estimation with nonuniform B-splines. By introducing the error indicator attached to each interval, we propose an adaptive strategy to generate the nonuniform knot vector. The error indicator is an approximation of the information entropy locally, which is closely related to the number of kernels when we construct the nonuniform estimator. The numerical experiments show that, compared with the uniform B-spline, the local density estimation with nonuniform B-splines not only achieves better estimation results but also effectively alleviates the overfitting phenomenon caused by the uniform B-splines. The comparison with the existing estimation procedures, including the state-of-the-art kernel estimators, demonstrates the accuracy of our new method.

Stein's method is a collection of probabilistic techniques that allow one to assess the distance between two probability distributions by means of differential operators. In 2007, the authors discovered that one can combine Stein's method with the powerful Malliavin calculus of variations, in order to deduce quantitative central limit theorems involving functionals of general Gaussian fields. This book provides an ideal introduction both to Stein's method and Malliavin calculus, from the standpoint of normal approximations on a Gaussian space. Many recent developments and applications are studied in detail, for instance: fourth moment theorems on the Wiener chaos, density estimates, Breuer–Major theorems for fractional processes, recursive cumulant computations, optimal rates and universality results for homogeneous sums. Largely self-contained, the book is perfect for self-study. It will appeal to researchers and graduate students in probability and statistics, especially those who wish to understand the connections between Stein's method and Malliavin calculus.

Conditional value-at-risk (CVaR) and value-at-risk, also called the superquantile and quantile, are frequently used to characterize the tails of probability distributions and are popular measures of risk in applications where the distribution represents the magnitude of a potential loss. buffered probability of exceedance (bPOE) is a recently introduced characterization of the tail which is the inverse of CVaR, much like the CDF is the inverse of the quantile. These quantities can prove very useful as the basis for a variety of risk-averse parametric engineering approaches. Their use, however, is often made difficult by the lack of well-known closed-form equations for calculating these quantities for commonly used probability distributions. In this paper, we derive formulas for the superquantile and bPOE for a variety of common univariate probability distributions. Besides providing a useful collection within a single reference, we use these formulas to incorporate the superquantile and bPOE into parametric procedures. In particular, we consider two: portfolio optimization and density estimation. First, when portfolio returns are assumed to follow particular distribution families, we show that finding the optimal portfolio via minimization of bPOE has advantages over superquantile minimization. We show that, given a fixed threshold, a single portfolio is the minimal bPOE portfolio for an entire class of distributions simultaneously. Second, we apply our formulas to parametric density estimation and propose the method of superquantiles (MOS), a simple variation of the method of moments where moments are replaced by superquantiles at different confidence levels. With the freedom to select various combinations of confidence levels, MOS allows the user to focus the fitting procedure on different portions of the distribution, such as the tail when fitting heavy-tailed asymmetric data.

Multivariate density estimation is an important problem that is frequently encountered in statistical learning and signal processing. One of the most popular techniques is Parzen windowing, also referred to as kernel density estimation. Gaussianization is a procedure that allows one to estimate multivariate densities efficiently from the marginal densities of the individual random variables. In this paper, we present an optimal density estimation scheme that combines the desirable properties of Parzen windowing and Gaussianization, using minimum Kullback-Leibler divergence as the optimality criterion for selecting the kernel size in the Parzen windowing step. The performance of the estimate is illustrated in a classifier design example

Multivariate density estimation is an important problem that is frequently encountered in statistical learning and signal processing. One of the most popular techniques is Parzen windowing, also referred to as kernel density estimation. Gaussianization is a procedure that allows one to estimate multivariate densities efficiently from the marginal densities of the individual random variables. In this paper, we present an optimal density estimation scheme that combines the desirable properties of Parzen windowing and Gaussianization, using minimum Kullback---Leibler divergence as the optimality criterion for selecting the kernel size in the Parzen windowing step. The utility of the estimate is illustrated in classifier design, independent components analysis, and Prices' theorem.

A method for measuring the density of data sets that contain an unknown number of clusters of unknown sizes is proposed. This method, called Pareto Density Estimation (PDE), uses hyper spheres to estimate data density. The radius of the hyper spheres is derived from information optimal sets. PDE leads to a tool for the visualization of probability density distributions of variables (PDEplot). For Gaussian mixture data this is an optimal empirical density estimation. A new kind of visualization of the density structure of high dimensional data set, the P-Matrix is defined. The P-Matrix for a 79- dimensional data set from DNA array analysis is shown. The P-Matrix reveals local concentrations of data points representing similar gene expressions. The P-Matrix is also a very effective tool in the detection of clusters and outliers in data sets.

Knowledge of the probability distribution of error in a regression problem plays an important role in verification of an assumed regression model, making inference about predictions, finding optimal regression estimates, suggesting confidence bands and goodness of fit tests as well as in many other issues of the regression analysis. This article is devoted to an optimal estimation of the error probability density in a general heteroscedastic regression model with possibly dependent predictors and regression errors. Neither the design density nor regression function nor scale function is assumed to be known, but they are suppose to be differentiable and an estimated error density is suppose to have a finite support and to be at least twice differentiable. Under this assumption the article proves, for the first time in the literature, that it is possible to estimate the regression error density with the accuracy of an oracle that knows “true” underlying regression errors. Real and simulated examples illustrate importance of the error density estimation as well as the suggested oracle methodology and the method of estimation.

It is well known that estimation of a bivariate cumulative distribution function of a pair of right censored lifetimes presents challenges unparalleled to the univariate case where a product-limit Kaplan–Meyer’s methodology typically yields optimal estimation, and the literature on optimal estimation of the joint probability density is next to none. The paper, for the first time in the survival analysis literature, develops the theory and methodology of sharp minimax and adaptive nonparametric estimation of the joint density under the mean integrated squared error (MISE) criterion. The theory shows how an underlying joint density, together with the bivariate distribution of censoring variables, affect the estimation, and what and how may or may not be estimated in the presence of censoring. Practical example illustrates the problem.

Convergence of Distributions Generated by Stationary Stochastic Processes

We consider nonparametric invariant density and drift estimation for a class of multidimensional degenerate resp. hypoelliptic diffusion processes, so-called stochastic damping Hamiltonian systems or kinetic diffusions, under anisotropic smoothness assumptions on the unknown functions. The analysis is based on continuous observations of the process, and the estimators’ performance is measured in terms of the sup-norm loss. Regarding invariant density estimation, we obtain highly nonclassical results for the rate of convergence, which reflect the inhomogeneous variance structure of the process. Concerning estimation of the drift vector, we suggest both non-adaptive and fully data-driven procedures. All of the aforementioned results strongly rely on tight uniform moment bounds for empirical processes associated to deterministic and stochastic integrals of the investigated process, which are also proven in this paper.

We study the problem of the non-parametric estimation for the density π of the stationary distribution of the multivariate stochastic differential equation with jumps (Xt)0≤t≤T, when the dimension d is such that d≥3. From the continuous observation of the sampling path on [0,T] we show that, under anisotropic Hölder smoothness constraints, kernel based estimators can achieve fast convergence rates. In particular, they are as fast as the ones found by Dalalyan and Reiss [11] for the estimation of the invariant density in the case without jumps under isotropic Hölder smoothness constraints. Moreover, they are faster than the ones found by Strauch [32] for the invariant density estimation of continuous stochastic differential equations, under anisotropic Hölder smoothness constraints. Furthermore, we obtain a minimax lower bound on the L2-risk for pointwise estimation, with the same rate up to a log(T) term. It implies that, on a class of diffusions whose invariant density belongs to the anisotropic Holder class we are considering, it is impossible to find an estimator with a rate of estimation faster than the one we propose.

Ten yeah of stratosphere geopotential height data are analyzed in an attempt to determine whether there are preferred flow regimes in the Northern Hemisphere winter stratosphere. The data are taken from Stratospheric Sounding Units on board NOAA satellites. The probability density estimate of the amplitude of the wavenumber 1 10-mb height is found to be bimodal. The density distribution is composed of a dominant large-amplitude mode and a less frequent low-amplitude mode. When the wavenumber 1 10-mb height data are projected onto the phase plant defined by the 10-mb zonal-mean winds and wavenumber 1 100-mb heights, three preferred regimes are evident. The small-amplitude mode separates into a strong zonal wind-weak wave regime and a weak zonal wind-weak wave regime. The large-amplitude mode is an intermediate zonal wind-strong wave regime. Transitions between the large-amplitude regime and the weak zonal wind-weak wave regime are found to be associated with major stratospheric warnings. The clustering of the stratospheric data into the preferred flow regimes is interpreted in light of the bifurcation properties of the Holton and Mass model. The interannual variability of the Northern Hemisphere winter stratosphere is interpreted in terms of the relative frequency of the observed preferred regimes.

Hyperspectral anomaly target detection (also known as hyperspectral anomaly detection (HAD)] is a technique aiming to identify samples with atypical spectra. Although some density estimation-based methods have been developed, they may suffer from two issues: 1) separated two-stage optimization with inconsistent objective functions makes the representation learning model fail to dig out characterization customized for HAD and 2) incapability of learning a low-dimensional representation that preserves the inherent information from the original high-dimensional spectral space. To address these problems, we propose a novel end-to-end local invariant autoencoding density estimation (E2E-LIADE) model. To satisfy the assumption on the manifold, the E2E-LIADE introduces a local invariant autoencoder (LIA) to capture the intrinsic low-dimensional manifold embedded in the original space. Augmented low-dimensional representation (ALDR) can be generated by concatenating the local invariant constrained by a graph regularizer and the reconstruction error. In particular, an end-to-end (E2E) multidistance measure, including mean-squared error (MSE) and orthogonal projection divergence (OPD), is imposed on the LIA with respect to hyperspectral data. More important, E2E-LIADE simultaneously optimizes the ALDR of the LIA and a density estimation network in an E2E manner to avoid the model being trapped in a local optimum, resulting in an energy map in which each pixel represents a negative log likelihood for the spectrum. Finally, a postprocessing procedure is conducted on the energy map to suppress the background. The experimental results demonstrate that compared to the state of the art, the proposed E2E-LIADE offers more satisfactory performance.

We consider the problem of predictive density estimation for normal models under Kullback-Leibler loss (KL loss) when the parameter space is constrained to a convex set. More particularly, we assume that $X\sim {\cal N}_{p}(\mu,v_{x}I)$ is observed and that we wish to estimate the density of $Y\sim {\cal N}_{p}(\mu,v_{y}I)$ under KL loss when μ is restricted to the convex set C⊂ℝp. We show that the best unrestricted invariant predictive density estimator pU is dominated by the Bayes estimator pπC associated to the uniform prior πC on C. We also study so called plug-in estimators, giving conditions under which domination of one estimator of the mean vector μ over another under the usual quadratic loss, translates into a domination result for certain corresponding plug-in density estimators under KL loss. Risk comparisons and domination results are also made for comparisons of plug-in estimators and Bayes predictive density estimators. Additionally, minimaxity and domination results are given for the cases where: (i) C is a cone, and (ii) C is a ball.