Nonparametric Conditional Density Estimation Research Articles

Correction: Nonparametric conditional density estimation in a deep learning framework for short-term forecasting

Correction: Nonparametric conditional density estimation in a deep learning framework for short-term forecasting

Income Inequality, Poverty and Growth

Today, one of the most important causes of global income inequality and poverty is the unequal sharing of the increase in economic growth between countries and regions. The increase in income inequality creates an important obstacle to economic development as it reduces the share of the poor in economic growth. Therefore, reducing poverty and income inequality is a fundamental policy goal in the development process. Because in the process of poverty alleviation, providing a fair income distribution as well as growth is important in terms of obtaining more effective results. For this reason, the aim of the study is to analyze the direction of global income inequality and its change by using the nonparametric conditional density estimation method. The analysis provides evidence to support that global income inequality did not decrease between 1995 and 2017. This result indicates that the gap is widening between countries with more even distribution and countries with an unfair distribution of income. Besides, the other aim of this study is to investigate the relationship between income inequality, poverty and economic growth. In this context, countries are classified according to income groups and analyzed with cross-section data estimation method. The general findings of the study vary in income groups in accordance with the theoretical literature. According to the results for all income groups, economic growth decreases poverty and income inequality increases poverty. On the other hand, results show that while increasing income inequality in the middle-income group increases growth, in the high-income group the increase in growth decreases inequality.

Minimax optimal conditional density estimation under total variation smoothness

This paper studies the minimax rate of nonparametric conditional density estimation under a weighted absolute value loss function in a multivariate setting. We first demonstrate that conditional density estimation is impossible if one only requires that $p_{X|Z}$ is smooth in $x$ for all values of $z$. This motivates us to consider a sub-class of absolutely continuous distributions, restricting the conditional density $p_{X|Z}(x|z)$ to not only be H\"older smooth in $x$, but also be total variation smooth in $z$. We propose a corresponding kernel-based estimator and prove that it achieves the minimax rate. We give some simple examples of densities satisfying our assumptions which imply that our results are not vacuous. Finally, we propose an estimator which achieves the minimax optimal rate adaptively, i.e., without the need to know the smoothness parameter values in advance. Crucially, both of our estimators (the adaptive and non-adaptive ones) impose no assumptions on the marginal density $p_Z$, and are not obtained as a ratio between two kernel smoothing estimators which may sound like a go to approach in this problem.

Nonparametric conditional density estimation in a deep learning framework for short-term forecasting

Short-term forecasting is an important tool in understanding environmental processes. In this paper, we incorporate machine learning algorithms into a conditional distribution estimator for the purposes of forecasting tropical cyclone intensity. Many machine learning techniques give a single-point prediction of the conditional distribution of the target variable, which does not give a full accounting of the prediction variability. Conditional distribution estimation can provide extra insight on predicted response behavior, which could influence decision-making and policy. We propose a technique that simultaneously estimates the entire conditional distribution and flexibly allows for machine learning techniques to be incorporated. A smooth model is fit over both the target variable and covariates, and a logistic transformation is applied on the model output layer to produce an expression of the conditional density function. We provide two examples of machine learning models that can be used, polynomial regression and deep learning models. To achieve computational efficiency, we propose a case–control sampling approximation to the conditional distribution. A simulation study for four different data distributions highlights the effectiveness of our method compared to other machine learning-based conditional distribution estimation techniques. We then demonstrate the utility of our approach for forecasting purposes using tropical cyclone data from the Atlantic Seaboard. This paper gives a proof of concept for the promise of our method, further computational developments can fully unlock its insights in more complex forecasting and other applications.

Conditional density estimation tools in python and R with applications to photometric redshifts and likelihood-free cosmological inference

It is well known in astronomy that propagating non-Gaussian prediction uncertainty in photometric redshift estimates is key to reducing bias in downstream cosmological analyses. Similarly, likelihood-free inference approaches, which are beginning to emerge as a tool for cosmological analysis, require a characterization of the full uncertainty landscape of the parameters of interest given observed data. However, most machine learning (ML) or training-based methods with open-source software target point prediction or classification, and hence fall short in quantifying uncertainty in complex regression and parameter inference settings. As an alternative to methods that focus on predicting the response (or parameters) $\mathbf{y}$ from features $\mathbf{x}$, we provide nonparametric conditional density estimation (CDE) tools for approximating and validating the entire probability density function (PDF) $\mathrm{p}(\mathbf{y}|\mathbf{x})$ of $\mathbf{y}$ given (i.e., conditional on) $\mathbf{x}$. As there is no one-size-fits-all CDE method, the goal of this work is to provide a comprehensive range of statistical tools and open-source software for nonparametric CDE and method assessment which can accommodate different types of settings and be easily fit to the problem at hand. Specifically, we introduce four CDE software packages in $\texttt{Python}$ and $\texttt{R}$ based on ML prediction methods adapted and optimized for CDE: $\texttt{NNKCDE}$, $\texttt{RFCDE}$, $\texttt{FlexCode}$, and $\texttt{DeepCDE}$. Furthermore, we present the $\texttt{cdetools}$ package, which includes functions for computing a CDE loss function for tuning and assessing the quality of individual PDFs, along with diagnostic functions. We provide sample code in $\texttt{Python}$ and $\texttt{R}$ as well as examples of applications to photometric redshift estimation and likelihood-free cosmological inference via CDE.

ABC–CDE: Toward Approximate Bayesian Computation With Complex High-Dimensional Data and Limited Simulations

ABSTRACTApproximate Bayesian computation (ABC) is typically used when the likelihood is either unavailable or intractable but where data can be simulated under different parameter settings using a forward model. Despite the recent interest in ABC, high-dimensional data and costly simulations still remain a bottleneck in some applications. There is also no consensus as to how to best assess the performance of such methods without knowing the true posterior. We show how a nonparametric conditional density estimation (CDE) framework, which we refer to as ABC–CDE, help address three nontrivial challenges in ABC: (i) how to efficiently estimate the posterior distribution with limited simulations and different types of data, (ii) how to tune and compare the performance of ABC and related methods in estimating the posterior itself, rather than just certain properties of the density, and (iii) how to efficiently choose among a large set of summary statistics based on a CDE surrogate loss. We provide theoretical and empirical evidence that justify ABC–CDE procedures that directly estimate and assess the posterior based on an initial ABC sample, and we describe settings where standard ABC and regression-based approaches are inadequate. Supplemental materials for this article are available online.

Nonparametric conditional density estimation for censored data based on a recursive kernel

Consider a regression model in which the response is subject to random right censoring. The main goal of this paper concerns the kernel estimation of the conditional density function in the case of censored interest variable. We employ a recursive version of the Nadaraya-Watson estimator in this context. The uniform strong consistency of the recursive kernel conditional density estimator is derived. Also, we prove the asymptotic normality of this estimator.

Distribution-free Prediction Bands for Non-parametric Regression

SummaryWe study distribution-free, non-parametric prediction bands with a focus on their finite sample behaviour. First we investigate and develop different notions of finite sample coverage guarantees. Then we give a new prediction band by combining the idea of ‘conformal prediction’ with non-parametric conditional density estimation. The proposed estimator, called COPS (conformal optimized prediction set), always has a finite sample guarantee. Under regularity conditions the estimator converges to an oracle band at a minimax optimal rate. A fast approximation algorithm and a data-driven method for selecting the bandwidth are developed. The method is illustrated in simulated and real data examples.

Nonparametric conditional density estimation of short-term interest rate movements: procedures, results and risk management implications

This article shows how to estimate the conditional density of daily changes in the 3-month T-bill rate, using an extension of the kernel-based estimator proposed by Rosenblatt (1969 Rosenblatt, M. 1969. “Conditional probability density and regression estimators”. In Multivariate Analysis II, Edited by: Krishnaiah, PR. 25–31. New York: Academic Press. [Google Scholar]). The shape of the estimated density is allowed to vary with both the level and the lagged change in rates. Due to the nonparametric character of the estimation procedure, the model produces conditional quantile estimates that are based only on the data and are independent of the modellers’ assumptions. The obtained results do not support the assumption of systematically mean-reverting behaviour underlying some theoretical models of short-term interest rate dynamics. However, they clearly indicate the presence of nonlinear first-order autocorrelation and volatility clustering effects, as well as a positive relationship between yield volatility and level.

Distribution Free Prediction Interval for Uncertainty Quantification in Remaining Useful Life Prediction

Remaining useful life (RUL) prediction is an important component for system health monitoring and prognosis. Ideally, one expects the prediction algorithm to provide the complete distribution of the RUL prediction over time taking various uncertainties into account. However, the dynamic model be- ing used to characterize state estimation and future loading uncertainties is often simplified through various approximations, leading to non-credible predicted distribution. Nevertheless, certain algorithm may only provide a point estimate of the RUL, making it difficult to quantify the uncertainty of the prediction. In this paper, we focus on interval prediction with high probability that guarantees finite sample validity without the knowledge of statistical distribution of the noise. The key idea is to leverage the newly proposed conformal pre- diction framework with non-parametric conditional density estimation. Under certain regularity conditions, the proposed interval estimator converges to an oracle band at a minimax optimal rate. In addition, we apply a data driven method to automatically select the bandwidth in the kernel density estimator. We discuss practical approximations to speed up the computation. The proposed method can be used to predict the RUL interval with physics-based model in a distribution free manner. It can also be applied to assess the validity of other prognostic algorithms from experimental data. We demonstrate the effectiveness of the RUL prediction for Li-Ion batteries using both simulated and experimental data.

Conditional stochastic kernel estimation by nonparametric methods

This article generalizes the conditional stochastic kernel developed by Quah (1997, 1998) for multiple and more general conditioning schemes using nonparametric conditional density estimation. We utilize this methodology to analyze conditional convergence in income for Brazilian municipalities between 1970 and 1991.

Nonparametric Conditional Density Estimation Using Piecewise-Linear Solution Path of Kernel Quantile Regression

The goal of regression analysis is to describe the stochastic relationship between an input vector x and a scalar output y. This can be achieved by estimating the entire conditional density p(y / x). In this letter, we present a new approach for nonparametric conditional density estimation. We develop a piecewise-linear path-following method for kernel-based quantile regression. It enables us to estimate the cumulative distribution function of p(y / x) in piecewise-linear form for all x in the input domain. Theoretical analyses and experimental results are presented to show the effectiveness of the approach.