Conditional Density Estimation Research Articles (Page 1)

ABSTRACTThis paper proposes the Deep Symbolic Learning (DSL) model, a deep learning‐based framework for robust regression, specifically designed when both the response and predictors are histogram‐valued variables. DSL utilizes cumulative distribution functions (CDFs) of covariate histograms within a one‐dimensional convolutional neural network (1D‐CNN) to transform the conditional density estimation problem into a multi‐class classification task, optimized using the joint binary cross‐entropy (JBCE) loss function. Extensive simulations and real‐world applications, including air quality, traffic volume, and climate data, demonstrate that the DSL model outperforms existing methods across three key evaluation metrics: CDF distance, empirical coverage of the 90% prediction interval, and average quantile loss. This work contributes to the field of symbolic data analysis and conditional density estimation.

This article introduces a unified approach to estimating the mutual density ratio, defined as the ratio between the joint density function and the product of the individual marginal density functions of two random vectors. It serves as a fundamental measure for quantifying the relationship between two random vectors. Our method uses the Bregman divergence to construct the objective function and leverages deep neural networks to approximate the logarithm of the mutual density ratio. We establish a non-asymptotic error bound for our estimator, achieving the optimal minimax rate of convergence under a bounded support condition. Additionally, our estimator mitigates the curse of dimensionality when the distribution is supported on a lower-dimensional manifold. We extend our results to overparameterized neural networks and the case with unbounded support. Applications of our method include conditional probability density estimation, mutual information estimation, and independence testing. Simulation studies and real data examples demonstrate the effectiveness of our approach. Supplementary materials for this article are available online, including a standardized description of the materials available for reproducing the work.

CPD is a statistical technique that finds the change points in data sequences where the statistical properties of the data have shifted. This technique has valuable applications in the field of economics as well as finance, and public health. In this study a CPD methodology is developed by proposing an enhanced three-step nonparametric approach based on the existing two-step method. The proposed framework couples Kernel Conditional Density Estimation with Fourier features and machine learning techniques for the precise identification and classification of change points. Data preprocessing for smoothness and noise reduction will be included, followed by KCDE-F for conditional density estimation, and then a machine-learning classifier refines the sensitivity and specificity of the detected change points. This paper identifies critical change points in Nigerian inflation dynamics using data from 2019 to 2023. The result shows that the developed the three-step procedure for change point detection presented here is not only also capable in change point detection but also in the estimation of structural breaks in time-series data. The wide applicability of this methodology is envisioned to extend beyond economics into other domains where the need for change point detection is compelling.

A precise measurement of photometric redshifts (photo-z) is crucial for the success of modern photometric galaxy surveys. Machine learning (ML) methods show great promise in this context, but suffer from covariate shift in training sets due to selection bias where interesting sources, e.g., high redshift objects, are underrepresented, and the corresponding ML models exhibit poor generalisation properties. We present an application of the StratLearn method to the estimation of photo-z (StratLearn-z), validating against simulations where we enforce the presence of covariate shift to different degrees. StratLearn is a statistically principled approach which relies on splitting the combined source and target datasets into strata, based on estimated propensity scores. The latter is the probability for an object in the dataset to be in the source set, given its observed covariates. After stratification, two conditional density estimators are fit separately within each stratum, and then combined via a weighted average. We benchmark our results against the GPz algorithm, quantifying the performance of the two algorithms with a set of metrics. Our results show that the StratLearn-z metrics are only marginally affected by the presence of covariate shift, while GPz shows a significant degradation of performance, specifically concerning the photo-z prediction for fainter objects for which there is little training data. In particular, for the strongest covariate shift scenario considered, StratLearn-z yields a reduced fraction of catastrophic errors, a factor of 2 improvement for the RMSE as well as one order of magnitude improvement on the bias. We also assess the quality of the predicted conditional redshift estimates using the probability integral transform (PIT) and the continuous rank probability score (CRPS). The PIT for StratLearn-z indicates that predictions are well-centered around the true redshift value, if conservative in their variance; the CRPS shows marked improvement at high redshifts when compared with GPz. Our julia implementation of the method, StratLearn-z, is publicly available at .

A rolling horizon heuristic approach for a multi-stage stochastic waste collection problem

In this paper, our goal is to generate synthetic data for heterogeneous (mixed-type) tabular datasets with high machine learning utility (MLu). Since the MLu performance depends on accurately approximating the conditional distributions, we focus on devising a synthetic data generation method based on conditional distribution estimation. We introduce MaCoDE by redefining the consecutive multi-class classification task of Masked Language Modeling (MLM) as histogram-based non-parametric conditional density estimation. Our approach enables the estimation of conditional densities across arbitrary combinations of target and conditional variables. We bridge the theoretical gap between distributional learning and MLM by demonstrating that minimizing the orderless multi-class classification loss leads to minimizing the total variation distance between conditional distributions. To validate our proposed model, we evaluate its performance in synthetic data generation across 10 real-world datasets, demonstrating its ability to adjust data privacy levels easily without re-training. Additionally, since masked input tokens in MLM are analogous to missing data, we further assess its effectiveness in handling training datasets with missing values, including multiple imputations of the missing entries.

In financial econometrics, it is empirically challenging to test the predictability of lagged predictors with varying levels of persistence in predictive quantile regression. A recent double-weighted method developed by Cai, Chen, and Liao (2023) has demonstrated desirable local power properties for both non stationary and stationary predictors. In this article, we propose a strategy to improve the construction of the auxiliary variables in the double-weighted method. This improvement makes it applicable to a broader range of persistent types in empirical analysis. Furthermore, we propose a random weighted bootstrap procedure to address the challenges involved in conditional density estimation. Simulation results demonstrate the effectiveness of the proposed test in correcting size distortion at the lower and upper quantiles. Finally, we apply the proposed test to reassess the predictability of macroeconomic and financial predictors on stock returns across different quantile levels, finding fewer significant predictors at the tails compared to Cai, Chen, and Liao (2023). Our results highlight that this test serves as a more conservative inference tool for practitioners evaluating the predictability of financial returns.

Understanding the efficiency of agricultural eco-product value realization is critical for sustainable development and regional equity. Here, we present a comprehensive analysis of the spatiotemporal patterns and regional disparities in the value realization efficiency of agricultural ecological products across China’s 31 provinces from 2010 to 2022. Utilizing an advanced Super-NSBM model, we quantify three dimensions of efficiency: overall value realization, economic value conversion, and social welfare value realization. Spatial mapping and dynamic evolution analysis are conducted through Dagum Gini coefficient decomposition and conditional kernel density estimation. Our results reveal three key insights: (1) China’s agricultural eco-product value realization efficiency remains suboptimal, with a gradual upward trend. Economic value conversion outperforms social welfare value realization, which exhibits significant regional heterogeneity. A distinct east–west gradient is observed, with Western regions demonstrating notable progress despite initial inefficiencies. (2) Inter-regional disparities are narrowing, particularly between Eastern and Central regions. While polarization in Northeast China has diminished, Western regions show widening efficiency gaps and emerging polarization trends. (3) Regional differences are predominantly driven by inter-group disparities, with Eastern China exhibiting the lowest intra-regional variability. Cross-regional differences follow a U-shaped trajectory, decreasing initially before rebounding in recent years. These findings provide a robust empirical foundation for optimizing regional strategies in ecological product value conversion and offer critical insights for addressing spatial inequities in sustainable agricultural development.

lpcde: Estimation and Inference for Local Polynomial Conditional Density Estimators

Sequential neural posterior estimation (SNPE) techniques have been recently proposed for dealing with simulation-based models with intractable likelihoods. Unlike approximate Bayesian computation, SNPE techniques learn the posterior from sequential simulation using neural network-based conditional density estimators by minimizing a specific loss function. The SNPE method proposed by Lueckmann et al. (NeurIPS 2017) used a calibration kernel to boost the sample weights around the observed data, resulting in a concentrated loss function. However, the use of calibration kernels may increase the variances of both the empirical loss and its gradient, making the training inefficient. To improve the stability of SNPE, this paper proposes to use an adaptive calibration kernel and several variance reduction techniques. The proposed method greatly speeds up the process of training and provides a better approximation of the posterior than the original SNPE method and some existing competitors as confirmed by numerical experiments. We also managed to demonstrate the superiority of the proposed method for a high-dimensional model with a real-world dataset.

Abstract A methodology is proposed for the determination of factor-dependent bandwidths for the kernel-based estimation of the conditional density $\rho (x|z)$ underlying a set of observations. The adaptive determination of the bandwidths is based on a $z$-dependent effective number of samples and variance. The procedure extends to categorical factors, where a non-trivial ‘bandwidth’ can be designed that optimally uses across-class information while capturing class-specific traits. A hierarchy of algorithms is developed, and their effectiveness is demonstrated on synthetic and real-world data.

In the literature, it is well known that in kernel density estimation, plug-in and cross-validation techniques, for the selection of the smoothing parameter, tend to provide under-or over-smoothed estimators when the sample size is small or medium, or the function to be estimated is complex. To overcome this latest problem, recently, the Bayesian approach has been proposed as an alternative to these classical methods. In this paper, we restricted attention to extending the idea of the Bayes rule to estimate the smoothing parameter of the conditional density kernel estimation. We are interested in the impact of the use of the global Bayesian approach for the smoothing parameter selection on the performance (the average of the Integrated Squared Error (ISE) and the calculation time required for their implementation) of the kernel conditional density estimator compared to the classical method trough a Monté Carlo simulation. Moreover, based on simulated samples of different sizes from two different conditional models, we adopted the Bayesian approach compared with the classical smoothing parameter selection procedure (the cross-validation method), using the Gaussian kernel to construct the estimators in question.

A novel spatial prediction method for soil heavy metal based on unbiased conditional kernel density estimation

Regression trees for fast and adaptive prediction intervals

Is the post-processing of global horizontal irradiance (GHI) forecasts necessary for issuing good photovoltaic (PV) power forecasts? Whenever this question is raised, the instinctive supposition always seems to be “yes,” because GHI is the most important weather parameter governing the amount of PV power generated, and surely, the better the GHI forecasts are, the better the PV power forecasts should result. To attend to this question more scientifically and more formally, two classic deterministic-to-deterministic post-processing methods, namely, the model output statistics and kernel conditional density estimation, are applied at various stages of PV power forecasting, resulting in four distinct workflows. These different workflows are trained and tested on three PV plants in Hungary, using data from a four-year (2017–2020) period. Both ground-based GHI and satellite-retrieved GHI are used as the “truth” with which numerical weather prediction (NWP) GHI forecasts are post-processed. A very thorough deterministic forecast verification exercise is conducted following the best practices. It is found that contrary to the common supposition, post-processing GHI only leads to marginal, if that can be quantified at all, benefits, so long as the PV power forecasts are to be post-processed. This ought to be deemed as a very important finding, as it puts into question the “GHI forecasting + post-processing + irradiance-to-power conversion” workflow that has dominated solar forecasting for decades.

A hybrid residual correction method with auxiliary particle filter and conditional kernel density estimation for remaining useful life prediction

This paper introduces FlexCodeTS, a new conditional density estimator for time series. FlexCodeTS is a flexible nonparametric method, which can be based on an arbitrary regression method. It is shown that FlexCodeTS inherits the rate of convergence of the chosen regression method. Hence, FlexCodeTS can adapt its convergence by employing the regression method that best fits the structure of data. From an empirical perspective, FlexCodeTS is compared to NNKCDE and GARCH in both simulated and real data. FlexCodeTS is shown to generally obtain the best performance among the selected methods according to either the CDE loss or the pinball loss.

Quantile regression and conditional density estimation can reveal structure that is missed by mean regression, such as multimodality and skewness. In this article, we introduce a deep learning generative model for joint quantile estimation called Penalized Generative Quantile Regression (PGQR). Our approach simultaneously generates samples from many random quantile levels, allowing us to infer the conditional distribution of a response variable given a set of covariates. Our method employs a novel variability penalty to avoid the problem of vanishing variability, or memorization, in deep generative models. Further, we introduce a new family of partial monotonic neural networks (PMNN) to circumvent the problem of crossing quantile curves. A major benefit of PGQR is that it can be fit using a single optimization, thus, bypassing the need to repeatedly train the model at multiple quantile levels or use computationally expensive cross-validation to tune the penalty parameter. We illustrate the efficacy of PGQR through extensive simulation studies and analysis of real datasets. Code to implement our method is available at https://github.com/shijiew97/PGQR.

The reduction in agricultural carbon emissions (ACEs) in Shandong Province is essential to China’s carbon peak and carbon neutrality objectives. In this regard, we constructed an ACE inventory for Shandong Province at a resolution of 1 km × 1 km, integrating the emission factor method with geographic information system (GIS) technology. Building upon this, we explored the dynamic evolution patterns of ACEs using kernel density estimation and conditional probability density estimation. Additionally, long short-term memory networks were trained to predict ACEs under various scenarios. The results showed that: (1) ACEs in Shandong Province exhibited two stages of change, i.e., “rise and decline”. Notably, 64.39% of emissions originated from the planting industry. The distribution of emissions was closely correlated with regional agricultural production modes. Specifically, CO2 emissions were predominantly distributed in crop cultivation areas, while CH4 and N2O emissions were primarily distributed in livestock breeding areas. The uncertainty of the emission inventory ranged from −12.04% to 10.74%, mainly caused by emission factors. (2) The ACE intensity of various cities in Shandong Province is decreasing, indicating a decoupling between ACEs and agricultural economic growth. Furthermore, the emission disparities among different cities are diminishing, although significant spatial non-equilibrium still persists. (3) From 2022 to 2030, the ACEs in Shandong Province will show a continuous downward trend. By 2030, the projected values under the baseline scenario, low-carbon scenario I, and low-carbon scenario II will be 6301.74 × 104 tons, 5980.67 × 104 tons, and 5850.56 × 104 tons. The low-carbon scenario reveals greater potential for ACE reduction while achieving efficient rural economic development and urbanization simultaneously. This study not only advances the methodology of the ACE inventory but also provides quantitative references and scientific bases for promoting low-carbon, efficient, and sustainable regional agriculture.

ABSTRACT The advent of massive broad-band photometric surveys enabled photometric redshift estimates for unprecedented numbers of galaxies and quasars. These estimates can be improved using better algorithms or by obtaining complementary data such as narrow-band photometry, and broad-band photometry over an extended wavelength range. We investigate the impact of both approaches on photometric redshifts for quasars using data from Southern Photometric Local Universe Survey (S-PLUS) DR4, Galaxy Evolution Explorer (GALEX) DR6/7, and the unWISE catalog for the Wide-field Infrared Survey Explorer (WISE) in three machine learning methods: Random Forest, Flexible Conditional Density Estimation (FlexCoDE), and Bayesian Mixture Density Network (BMDN). Including narrow-band photometry improves the root-mean-square error by 11 per cent in comparison to a model trained with only broad-band photometry. Narrow-band information only provided an improvement of 3.8 per cent when GALEX and WISE colours were included. Thus, narrow bands play a more important role for objects that do not have GALEX or WISE counterparts, which respectively makes 92 per cent and 25 per cent of S-PLUS data considered here. Nevertheless, the inclusion of narrow-band information provided better estimates of the probability density functions obtained with FlexCoDE and BMDN. We publicly release a value-added catalogue of photometrically selected quasars with the photo-z predictions from all methods studied here. The catalogue provided with this work covers the S-PLUS DR4 area (∼3000 square degrees), containing 645 980, 244 912, 144 991 sources with the probability of being a quasar higher than, 80 per cent, 90 per cent, 95 per cent up to r < 21.3 and good photometry quality in the detection image. More quasar candidates can be retrieved from the S-PLUS data base by considering less restrictive selection criteria.