Nonparametric Density Estimation Research Articles

Asymptotic results for recursive multivariate associated-kernel estimators of the probability density mass function of a data stream

. In this article, our central focus is to investigate the non parametric estimator of the probability density or mass function of a data stream by using the general family of kernels, also called multivariate associated kernels. Being able to estimate categorial, count, discrete, and (semi)continuous distributions, these multivariate associated kernel estimators unify several particular cases. Within this framework, we first introduce a recursive estimator for non classical associated kernels. Under reasonable assumptions, we subsequently exhibit their asymptotic results, such as the local uniform L q -consistency, the pointwise asymptotic normality and finally the strong global consistency. Illustrative examples of associated kernels are revisited.

Study on the distribution of average wind speeds at a mountainous bridge site for structural durability design

Conventional wind speed distribution methods (e.g., Rayleigh distribution and Weibull distribution) may not adequately capture complex characteristics of wind fields in mountainous areas. To address this problem, this study proposes a semi-parametric mix method for modeling the distribution of average wind speeds based on the combination of nonparametric Kernel Density Estimation (KDE) and Generalized Pareto Distribution (GPD). In the proposed method, KDE focuses on capturing the distribution in the main part of average wind speeds, while GPD aims at performing the distribution in terms of those in the extreme part. The segment point (i.e., the threshold) between KDE and GPD distributions is determined based on the combination of conditional mean excesses criterion and empirical rule. Meanwhile, the selection of modeling parameters should ensure that the mix distribution model is continuous and differentiable at the identified threshold point. Then, the commonly-used conditional probability model is further introduced to describe the wind direction distribution. Finally, a case study based on the measured 10-min average wind speeds at a mountainous bridge site is employed to demonstrate the effectiveness of the proposed method. The results indicate that: (1) the distribution of omnidirectional average wind speeds in the mountainous bridge site exhibits an obviously single-peak characteristic, while those considering wind directionality present a certain bimodal characteristic; (2) the proposed method can effectively describe wind speed distributions with different statistical characteristics, and the fitting accuracy outperforms the frequently-employed Weibull distribution model.

Big data analytics for quality variation over work shifts in manufacturing systems

ABSTRACT In the manufacturing industry, mass production enables manufacturers to produce parts with high precision and lower costs. Multiple shifts operate production processes to maximize efficiency. Several researches have been conducted on the impact of shift work on labor’s health and habits. However, there have been few studies on the influence of shift work on the consistency of product quality. This paper provides a methodology to analyze the impact of different work shifts in a real electronic board manufacturing industry. The study uses big data and analytics to assess product quality from data. Non-parametric kernel density estimation is used to approximate the distribution of good products in each work slot. Then several metrics are used to measure the dissimilarity between the estimated densities. The approach can be leveraged in various problems related to process performance and quality. The obtained results show that there is no significant difference in terms of product quality between work shifts. These prove the consistency of the manufacturing processes and the homogeneity of performance across work shifts in the studied factory. Compared to the literature, the paper presents the first quantitative analysis to compare production process performance and product quality over shift works.

Distributionally Robust Optimization for integrated energy system accounting for refinement utilization of hydrogen and ladder-type carbon trading mechanism

This study presents a hydrogen-IES with hydrogen refinement utilization under the framework of ladder-type carbon trading mechanism (LCTM). To navigate the uncertainties associated with RE in the proposed hydrogen-IES, we introduce a data-driven Distributionally Robust Optimization (DRO) method based on an innovative ambiguity set construction approach. This method leverages nonparametric Kernel Density Estimation (KDE) to fit the error probability distribution function and determine the predicted interval for wind turbine (WT) and photovoltaic (PV) output under a given confidence level. Subsequently, Latin hypercube sampling (LHS) and a reduction method based on probability distance are employed to generate original and typical scenarios, respectively. Ultimately, the construction of ambiguity sets imposes constraints on typical scenarios and probabilities using 1-norm and ∞-norm. Case study is conducted to validate the effectiveness of the proposed innovations. The results underscore the significant advantages of hydrogen refinement utilization within the proposed framework. Specifically, it demonstrates notable economic benefits, with costs lower by 11.98%, 2.46%, and 1.53% compared to scenarios omitting gas synthesis, hydrogen blending combustion in CHP and GB, and hydrogen storage, respectively. Moreover, hydrogen refinement exhibits superior performance in accommodating WT and PV, with ratios 25.29%, 0.43%, and 1.91% higher than alternative methods, with lower carbon emissions of 42.23%, 11.91%, and 6.13%. The LCTM can reduce more carbon emissions by 7.60% and 0.145% compared to scenarios without carbon trading mechanisms and with fixed-price trading mechanisms. The probability distribution derived from nonparametric KDE aligns closely with the true distribution, fostering a more objective analysis and mitigating the conservatism of the ambiguity set and optimization scheduling. Finally, sensitivity analyses are conducted in detail, including hydrogen blending ratio, basic price and growth ratio of LCTM.

A hybrid nonparametric multivariate density estimator with applications to risk management

. Multivariate density estimation is plagued by the curse of dimensionality in theory and practice. We propose a hybrid density estimator of a multivariate density f that combines the strengths of the kernel estimator and the exponential series estimator. This estimator refines a preliminary kernel estimate f ̂ 0 with a multiplicative correction that estimates the ratio r = f / f ̂ 0 with an exponential series estimator. Thanks to the consistency of the pilot estimate, the coefficients of the series expansion tend to approach zero asymptotically. Accordingly, we design a thresholding method for basis function selection. A major obstacle of multivariate exponential series estimator is the calculation of its normalization factor. We resolve this difficulty with Monte Carlo integration, using the pilot kernel estimate as the trial density for importance sampling. This approach greatly enhances the practicality of the hybrid estimator. Numerical simulations demonstrate the good finite sample performance of the hybrid estimator. We present one empirical application in financial risk management.

Particle filtering supported probability density estimation of mobility patterns

This paper presents a methodology that aims to enhance the accuracy of probability density estimation in mobility pattern analysis by integrating prior knowledge of system dynamics and contextual information into the particle filter algorithm. The quality of the data used for density estimation is often inadequate due to measurement noise, which significantly influences the distribution of the measurement data. Thus, it is crucial to augment the information content of the input data by incorporating additional sources of information beyond the measured position data. These other sources can include the dynamic model of movement and the spatial model of the environment, which influences motion patterns. To effectively combine the information provided by positional measurements with system and environment models, the particle filter algorithm is employed, which generates discrete probability distributions. By subjecting these discrete distributions to exploratory techniques, it becomes possible to extract more certain information compared to using raw measurement data alone. Consequently, this study proposes a methodology in which probability density estimation is not solely based on raw positional data but rather on probability-weighted samples generated through the particle filter. This approach yields more compact and precise modeling distributions. Specifically, the method is applied to process position measurement data using a nonparametric density estimator known as kernel density estimation. The proposed methodology is thoroughly tested and validated using information-theoretic and probability metrics. The applicability of the methodology is demonstrated through a practical example of mobility pattern analysis based on forklift data in a warehouse environment.

A novel wind power deterministic and interval prediction framework based on the critic weight method, improved northern goshawk optimization, and kernel density estimation

Wind power prediction holds significant importance for the stable operation of power systems and the enhancement of energy utilization efficiency. Interval prediction, capable of providing more uncertain information, has garnered widespread attention. In this study we have designed a hybrid prediction framework for both deterministic and interval prediction. Initially, the original wind power data undergo cleaning using the quartile method and the fuzzy C-means (FCM) clustering algorithm. This approach not only eliminates scattered outliers but also identifies concentrated outliers more effectively. Subsequently, five distinct single prediction models were employed for point predictions. The critic weight method was applied to yield point forecasting results with higher prediction accuracy. Nonparametric kernel density estimation (KDE) and normal distribution (ND) were incorporated to calculate the prediction interval (PI) under varying confidence levels. Finally, an improved northern goshawk optimization (INGO) algorithm was proposed, integrating three enhancement methods: levy flight, sinusoidal mapping, and a reverse learning strategy. This optimization aims to fine-tune the weight coefficients of the aforementioned two interval prediction methods. The resulting PI overcomes the limitations of a single-interval prediction model. To effectively demonstrate the forecasting performance of the proposed prediction framework, two datasets from a wind turbine in northwest China were selected for simulation verification. In deterministic prediction, the designed hybrid forecast model demonstrates a reduction in root mean square error (RMSE), mean absolute percentage error (MAPE), and mean absolute error (MAE) by 8.5%, 16.98%, and 11.49%, respectively, compared to other single prediction models. In interval prediction, the prediction interval coverage probability (PICP) is enhanced, while the prediction interval normalized average width (PINAW) is effectively reduced. The designed interval prediction model can reduce the comprehensive evaluation index coverage width-based criterion (CWC) by 10.51%, 9.62%, and 12.95% on average at 95%, 90%, and 80% confidence levels, respectively. Simulation results verified the validity and feasibility of the proposed prediction framework.

Bayesian semi-parametric estimation of compound inhomogeneous Poisson processes for ultra-high frequency financial transaction data

Marked point processes provide a flexible framework for studying ultra-high frequency financial data that records the time and price for each transaction. This paper estimates compound, inhomogeneous Poisson processes (CIPP) where trades arrive according to an inhomogeneous Poisson process, and returns are drawn from a distribution after arrival. The intensity functions have Gaussian process priors to model time-of-day (diurnal) effects, which have bursts of activity at the open and close and a midday lull, and lagged returns from the previous trade, which are idiosyncratic and depend on the stock. The nonparametric density estimator for returns imposes a shape constraint to obtain a unimodal distribution that is asymmetric and leptokurtic. A mixture model separates trades into time sensitive and time insensitive trades with different intensity functions. The latter group tends to have larger orders and higher returns. The proposed model is compared to B-splines and geometric Brownian motion.

Structural EEG signal analysis for sleep apnea classification.

Diagnosing the sleep apnea can be critical in preventing the person having sleep disorder from unhealthy results. The aim of this study is to obtain a sleep apnea scoring approach by comparing parametric and non-parametric power spectral density (PSD) estimation methods from EEGsignals recorded from different brain regions (C4-M1 and O2-M1) for transient signal analysis of sleep apnea patients. Power Spectral Density (PSD) methods (Burg, Yule-Walker, periodogram, Welch and multi-taper) are examined for the detection of apnea transition states including pre-apnea, intra-apnea and post-apnea together with statistical methods. In the experimental studies, EEG recordings available in the database were analyzed with PSD methods. Results showed that there are statistically significant differences between parametric and non-parametric methods applied for PSD analysis of apnea transition states in delta, theta, alpha and beta bands. Moreover, it was also revealed that PSD of EEG signals obtained from C4-M1 and O2-M1 channels were also found statistically different as proved by classification using the K-nearest neighbour (KNN) method. It was concluded that not only applying different PSD methods, but also EEG signals from different brain regions provided different statistical results in terms of apnea transition states as obtained from KNN classification.

Point Cloud Denoising and Feature Preservation: An Adaptive Kernel Approach Based on Local Density and Global Statistics.

Noise removal is a critical stage in the preprocessing of point clouds, exerting a significant impact on subsequent processes such as point cloud classification, segmentation, feature extraction, and 3D reconstruction. The exploration of methods capable of adapting to and effectively handling the noise in point clouds from real-world outdoor scenes remains an open and practically significant issue. Addressing this issue, this study proposes an adaptive kernel approach based on local density and global statistics (AKA-LDGS). This method constructs the overall framework for point cloud denoising using Bayesian estimation theory. It dynamically sets the prior probabilities of real and noise points according to the spatial function relationship, which varies with the distance from the points to the center of the LiDAR. The probability density function (PDF) for real points is constructed using a multivariate Gaussian distribution, while the PDF for noise points is established using a data-driven, non-parametric adaptive kernel density estimation (KDE) approach. Experimental results demonstrate that this method can effectively remove noise from point clouds in real-world outdoor scenes while maintaining the overall structural features of the point cloud.

Robust Bayesian estimation via the [formula omitted]-divergence

In this paper, we introduce a novel framework for robust Bayesian parameter estimation using the K-divergence. The framework incorporates an outlier resilient pseudo-posterior density function, called the K-posterior, which is based on an empirical version of the K-divergence. The latter involves utilizing Parzen’s non-parametric Kernel density estimator to mitigate the influence of outliers. Under the quadratic loss, a new robust analog of the posterior mean estimator (PME), referred here to as the KPME, is obtained. In the paper, we examine the asymptotic behavior of the KPME and investigate its robustness in the presence of outliers. Furthermore, we tackle the task of data-guided selection for the bandwidth parameter of the kernel in order to optimize a performance-oriented objective. Lastly, the KPME is successfully applied to robust Bayesian source localization under intermittent jamming.

Regional disparities, dynamic evolution, and spatial spillover effects of urban-rural carbon emission inequality in China

ObjectiveThis study recalculates the carbon emissions of urban and rural residents in China, analyzing the dynamic evolution trends of urban and rural carbon emissions. It explores the spatial spillover effects centered around the inequality in carbon emissions between urban and rural areas.MethodsThe study calculates the carbon emissions of urban and rural residents in each province based on the IPCC method. Non-parametric kernel density estimation is employed to depict the dynamic evolution characteristics of national, urban, and rural carbon emissions. The Theil Index is used to measure the disparities in urban and rural carbon emissions in major strategic regions, further applying the Theil Index to evaluate the inequality of urban and rural carbon emissions across provinces. This helps identify the driving factors affecting the inequality of urban and rural carbon emissions and their spatio-temporal effects.FindingCarbon emissions from urban and rural residents in China present a divergent development pattern. Urban emissions have increased, with inter-provincial disparities widening; rural emissions tend to stabilize, with slight growth in inter-provincial gaps. The overall inequality of carbon emissions in various regions of China experiences a three-phase journey of rise, decline, and stabilization. Urban inequality first increases then decreases, while rural inequality gradually lessens, showing clear regional and urban-rural differences. Market and government factors significantly impact the inequality of urban and rural carbon emissions. The development of the digital economy aids in reducing inequality and generates significant spatial spillover effects. The relationship between economic development level and carbon emission inequality is U-shaped. Industrial structure optimization can reduce urban-rural inequality, but its spatial spillover effect is not significant. Government intervention has limited effects, while environmental regulations may increase inequality. Opening up to the outside world helps reduce inequality, and the impact of population density is complex.

Nonparametric density estimation over its unknown support for right censored data

The long-standing problem of nonparametric density estimation of the lifetime of interest over its unknown support, when only right-censored observations are available, is considered. Two cases, when the support of the lifetime of interest is a proper subset of the support of censoring variable and when the supports are the same, are explored. An orthogonal series density estimation, over a random interval defined by a specially chosen sequence of order statistics, is proposed and rates of the mean integrated squared error convergence are evaluated.

Not All Probability Density Functions Are Tomograms.

This paper delves into the significance of the tomographic probability density function (pdf) representation of quantum states, shedding light on the special classes of pdfs that can be tomograms. Instead of using wave functions or density operators on Hilbert spaces, tomograms, which are the true pdfs, are used to completely describe the states of quantum systems. Unlike quasi-pdfs, like the Wigner function, tomograms can be analysed using all the tools of classical probability theory for pdf estimation, which can allow a better quality of state reconstruction. This is particularly useful when dealing with non-Gaussian states where the pdfs are multi-mode. The knowledge of the family of distributions plays an important role in the application of both parametric and nonparametric density estimation methods. We show that not all pdfs can play the role of tomograms of quantum states and introduce the conditions that must be fulfilled by pdfs to be "quantum".

Multivariate Density Estimation by Neural Networks.

We propose nonparametric methods to obtain the Probability Density Function (PDF) to assess the properties of the underlying data generating process (DGP) without imposing any assumptions on the DGP, using neural networks (NNs). The proposed NN has advantages compared to well-known parametric and nonparametric density estimators. Our approach builds on literature on cumulative distribution function (CDF) estimation using NN. We extend this literature by providing analytical derivatives of this obtained CDF. Our approach hence removes the numerical approximation error in differentiating the CDF output, leading to more accurate PDF estimates. The proposed solution applies to any NN model, i.e., for any number of hidden layers or hidden neurons in the multilayer perceptron (MLP) structure. The proposed solution applies the PDF estimation by NN to continuous distributions as well as discrete distributions. We also show that the proposed solution to obtain the PDF leads to good approximations when applied to correlated variables in a multivariate setting. We test the performance of our method in a large Monte Carlo simulation using various complex distributions. Subsequently, we apply our method to estimate the density of the number of vehicle counts per minute measured with road sensors for a time window of 24 h.

Analysis of optimization methods for nonparametric estimation of probability density in large samples

Analysis of optimization methods for nonparametric estimation of probability density in large samples

Floating-point histograms for exploratory analysis of large scale real-world data sets

Histograms are among the most popular methods used in exploratory analysis to summarize univariate distributions. In particular, irregular histograms are good non-parametric density estimators that require very few parameters: the number of bins with their lengths and frequencies. Although many approaches have been proposed in the literature to infer these parameters, most existing histogram methods are difficult to exploit for exploratory analysis in the case of real-world data sets, with scalability issues, truncated data, outliers or heavy-tailed distributions. In this paper, we focus on the G-Enum histogram method, which exploits the Minimum Description Length (MDL) principle to build histograms without any user parameter. We then propose to extend this method by exploiting a new modeling space based on floating-point representation, with the objective of building histograms resistant to outliers or heavy-tailed distributions. We also suggest several heuristics and a methodology suitable for the exploratory analysis of large scale real-world data sets, whose underlying patterns are difficult to recover for digitization reasons. Extensive experiments show the benefits of the approach, evaluated with a dual objective: the accuracy of density estimation in the case of outliers or heavy-tailed distributions, and the effectiveness of the approach for exploratory data analysis.

Analysis of power dispatching decisions with energy storage systems using the optimal probability distribution model of renewable energy

Effectively managing the inherent unpredictability and fluctuation attributes of renewable energy sources within scheduling systems has emerged as a critical matter demanding immediate attention. The incorporation of energy storage technology offers notable advantages by mitigating fluctuations in wind power generation and curtailing peak shaving costs in scheduling systems through economical utilization of energy storage. This study employs the adjustment of energy storage’s reserve capacity function to modify the economic gains and losses arising from fixed spinning reserve. This approach tailors the reserve capacity function by formulating the uncertainty inherent in wind power generation. The paper presents an enhanced selection method for calculating optimal bandwidth within the framework of least squares approximation probability density function scenarios. The fusion of least squares approximation and optimal bandwidth computation permits precise refinement of bandwidth in non-parametric kernel density estimation approaches. Consequently, the study puts forth a comprehensive model for coordinated scheduling and regulation of a multi-scale energy storage power system. Finally, the stability and economy of the proposed method in the scheduling operation process were verified through simulation, and the research results can provide a theoretical basis for future uncertain wind power scheduling.

Electric Vehicle and Photovoltaic Power Scenario Generation under Extreme High-Temperature Weather

In recent years, with the intensification of global warming, extreme weather has become more frequent, intensifying the uncertainty of new energy output and load power, and seriously affecting the safe operation of power systems. Scene generation is an effective method to solve the uncertainty problem of stochastic planning of integrated systems of new energy generation. Therefore, this paper proposes a scenario generation and scenario reduction model of photovoltaic (PV) output and electric vehicle (EV) load power under extreme weather based on the copula function. Firstly, the non-parametric kernel density estimation method is used to fit a large number of sample data. The kernel density estimation expressions of PV and EV powers under extreme weather conditions are obtained and the corresponding goodness of fit tests are carried out. Then, a variety of joint distribution models based on the copula function are established to judge the goodness of fit of each model, and the optimal copula function is selected as the joint probability distribution function by combining the Kendall and Spearman correlation coefficients of each model. Finally, the optimal copula joint probability distribution is used to generate PV and EV power scenarios. The data of extremely hot weather in a certain province were selected for an example analysis. The results show that the output scenario obtained conforms to the correlation under this extreme weather, and has higher accuracy in reflecting the actual PV output and load power in this province under this extreme weather, which can provide a reference for reliability analyses of power systems and power grid planning.

Analysis of optimization methods for nonparametric estimation of probability density in large volume samples

A method is proposed for selecting the blurriness coefficient of kernel functions for nonparametric estimation of the probability density of a one-dimensional random variable with large volumes of statistical data, for example, obtained by remote sensing of natural objects. In the proposed method for selecting the blurriness coefficient, a regression estimate of the probability density is used. A method for synthesizing a regression probability density estimate is presented. The synthesis of the estimate is based on compression of the initial sample by decomposition of the range of values of a random variable. To decompose the range of values of a random variable, the Heinhold-Gaede rule and the formula for optimal selection of the number of sampling intervals are applied. Two approaches to the selection of the blurriness coefficient of the regression estimation of probability density using the traditional and proposed by the authors optimization methods of nonparametric estimation of probability density are considered. The traditional method of optimizing nonparametric estimation of probability density is based on minimizing its mean square deviation. In the proposed method, the selection of the blurriness coefficients of the kernel functions is based on the conditions of the minimum error of approximation of the regression estimate of the desired probability density. The approximation properties of the regression estimation of probability density using two methods of its optimization are analyzed. The conditions of their competence in estimating the probability densities of random variables with a lognormal distribution law are established. The results obtained allow for development when optimizing a regression estimate of the probability density of a multidimensional random variable.