Multivariate Kernel Density Estimation Research Articles

Evolving demographics: a dynamic clustering approach to analyze residential segregation in Berlin

This paper examines the phenomenon of residential segregation in Berlin over time using a dynamic clustering analysis approach. Previous research has examined the phenomenon of residential segregation in Berlin at a high spatial and temporal aggregation and statically, i.e. not over time. We propose a methodology to investigate the existence of clusters of residential areas according to migration background, age group, gender, and socio-economic dimension over time. To this end, we have developed a sequential mixed methods approach that includes a multivariate kernel density estimation technique to estimate the density of subpopulations and a dynamic cluster analysis to discover spatial patterns of residential segregation over time (2009-2020). The dynamic analysis shows the emergence of clusters on the dimensions of migration background, age group, gender and socio-economic variables. We also identified a structural change in 2015, resulting in a new cluster in Berlin that reflects the changing distribution of subpopulations with a particular migratory background. Finally, we discuss the findings of this study with previous research and suggest possibilities for policy applications and future research using a dynamic clustering approach for analyzing changes in residential segregation at the city level.

ANDClust: An Adaptive Neighborhood Distance‐Based Clustering Algorithm to Cluster Varying Density and/or Neck‐Typed Datasets

AbstractAlthough density‐based clustering algorithms can successfully define clusters in arbitrary shapes, they encounter issues if the dataset has varying densities or neck‐typed clusters due to the requirement for precise distance parameters, such as eps parameter of DBSCAN. These approches assume that data density is homogenous, but this is rarely the case in practice. In this study, a new clustering algorithm named ANDClust (Adaptive Neighborhood Distance‐based Clustering Algorithm) is proposed to handle datasets with varying density and/or neck‐typed clusters. The algorithm consists of three parts. The first part uses Multivariate Kernel Density Estimation (MulKDE) to find the dataset's peak points, which are the start points for the Minimum Spanning Tree (MST) to construct clusters in the second part. Lastly, an Adaptive Neighborhood Distance (AND) ratio is used to weigh the distance between the data pairs. This method enables this approach to support inter‐cluster and intra‐cluster density varieties by acting as if the distance parameter differs for each data of the dataset. ANDClust is tested on synthetic and real datasets to reveal its efficiency. The algorithm shows superior clustering quality in a good run‐time compared to its competitors. Moreover, ANDClust could effectively define clusters of arbitrary shapes and process high‐dimensional, imbalanced datasets may have outliers.

Cross-entropy based importance sampling for composite systems reliability evaluation with consideration of multivariate dependence

Multivariate kernel density estimation (KDE) is powerful for dependence modeling, but producing a non-sparse probability density estimation model. To address this issue, a sparse density estimation technique termed as generalized cross-entropy (GCE) density estimator, is proposed to estimate the joint probability density function (PDF) of correlated random variables (RVs). Moreover, to effectively accelerate the composite systems reliability evaluation, a unified cross-entropy based importance sampling (CEIS) method accommodating both GCE and KDE density estimators is presented, and a creative CE optimization approach is also developed based on the composition sampling algorithm, through which the analytical updating rules of IS-PDF parameters can be derived even though GCE and KDE estimators do not belong to the exponential distribution family (EF). Compared to the KDE estimator, the GCE estimator encounters much less dimensionality difficulty in CE optimization due to the model sparsity merit, and consequently achieves more accurate IS-PDF parameters and better CEIS efficiency. The superiority of GCE estimator over KDE estimator in correlation modeling and CEIS is verified by several numerical tests.

ImpKmeans: An Improved Version of the K-Means Algorithm, by Determining Optimum Initial Centroids, based on Multivariate Kernel Density Estimation and Kd-Tree

ImpKmeans: An Improved Version of the K-Means Algorithm, by Determining Optimum Initial Centroids, based on Multivariate Kernel Density Estimation and Kd-Tree

A data-driven approach for generating load profiles based on InfoGAN and MKDE

High-quality demand-side management requires an abundance of load profiles to support decision-making processes. However, customer energy consumption data often contains sensitive personal information, and service providers face significant challenges in accessing a substantial amount of energy consumption data. To generate a large volume of customer data without compromising privacy, this study introduces a data-driven approach integrating Information Maximizing Generative Adversarial Networks (InfoGAN) with Multivariate Kernel Density Estimation (MKDE) for the generation of load profiles. InfoGAN is firstly trained based on existing customer load profiles, with the Q network disentangling the load into feature variables and the generator producing realistic profiles. Subsequently, MKDE is utilized to assess the distribution of these features, enabling the generation of new profiles by sampling new feature variables. The proposed method circumvents the need for intricate sampling or modeling processes and generates realistic data that represents the inherent uncertainties and fluctuations characterizing customers’ electricity consumption. The generated data could be used as the substitution for real electricity consumption data, thereby facilitating further applications without compromising privacy concerns.

A novel adaptive multi-scale Rényi transfer entropy based on kernel density estimation

The utilization of Rényi Transfer Entropy (RTE) as a powerful model for analyzing causal relationships among variables has become pervasive in the field of complex systems time series causality detection. However, there are two major challenges in using RTE: the inherent multilevel structure of causal associations in the systems, and the precision in RTE estimation. To address these challenges, this paper proposes an adaptive multi-scale Rényi transfer entropy based on kernel density estimation. The framework of causal detection based on the novel RTE consists of two parts: adaptive discrete wavelet transform (ADWT)-based time series decomposition and multivariate kernel density estimation (MKDE)-based causality network generation. In the ADWT-based time series decomposition, the original series are decomposed into different frequency bands by optimal wavelet coefficients, which is generated adaptively by an auto-encoder. In the MKDE-based causality network generation, the causal network between the variables is represented by an adjacency matrix composed of their decomposition components in each layer, and the values of the matrix are the RTE values between the variables. In order to accurate estimation of RTE values an evaluation criterion for KDE under a uniform measure in both univariate and multivariate cases and the optimal bandwidth selection is provided in this part. To validate the effectiveness of the novel causal measure in this paper, the proposed method is tested on the synthetic and real data, and the results show that it can effectively detect causal relationships among variables at different levels in non-stationary time series of both bidirectional and undirectional complex systems. Compared to the other RTE estimators, the proposed method can detect the causality accurately and avoid the spurious causality.

Numerical performance of penalized comparison to overfitting for multivariate kernel density estimation

Kernel density estimation is a well known method involving a smoothing parameter (the bandwidth) that needs to be tuned by the user. Although this method has been widely used, the bandwidth selection remains a challenging issue in terms of balancing algorithmic performance and statistical relevance. The purpose of this paper is to study a recently developed bandwidth selection method, called Penalized Comparison to Overfitting (PCO). We first provide new theoretical guarantees by proving that PCO performed with non-diagonal bandwidth matrices is optimal in the oracle and minimax approaches. PCO is then compared to other usual bandwidth selection methods (at least those which are implemented in the R-package) for univariate and also multivariate kernel density estimation on the basis of intensive simulation studies. In particular, cross-validation and plug-in criteria are numerically investigated and compared to PCO. The take home message is that PCO can outperform the classical methods without algorithmic additional cost.

Kernel density estimation by stagewise algorithm with a simple dictionary

This study proposes multivariate kernel density estimation by stagewise minimization algorithm based on U-divergence and a simple dictionary. The dictionary consists of an appropriate scalar bandwidth matrix and a part of the original data. The resulting estimator brings us data-adaptive weighting parameters and bandwidth matrices, and provides a sparse representation of kernel density estimation. We develop the non-asymptotic error bound of the estimator that we obtained via the proposed stagewise minimization algorithm. It is confirmed from simulation studies that the proposed estimator performs as well as, or sometimes better than, other well-known density estimators.

Kernel density estimation by genetic algorithm

ABSTRACT This study proposes a data condensation method for multivariate kernel density estimation by genetic algorithm. First, our proposed algorithm generates multiple subsamples of a given size with replacement from the original sample. The subsamples and their constituting data points are regarded as chromosome and gene, respectively, in the terminology of genetic algorithm. Second, each pair of subsamples breeds two new subsamples, where each data point faces either crossover, mutation, or reproduction with a certain probability. The dominant subsamples in terms of fitness values are inherited by the next generation. This process is repeated generation by generation and results in a kernel density estimator with sparse representation in its completion. We confirmed from simulation studies that the resulting estimator can perform better than other well-known density estimators.

Refreg: An R Package for Estimating Conditional Reference Regions

Multivariate reference regions (MVR) represent the extension of the reference interval concept to the multivariate setting. A reference interval is defined by two threshold points between which a high percentage of healthy subjects' results, usually 95%, are contained. Analogously, an MVR characterizes the values of several diagnostic tests most frequently found among non-diseased subjects by defining a convex hull containing 95% of the results. MVRs have great applicability when working with diseases that are diagnosed via more than one continuous test, e.g., diabetes or hypothyroidism. The present work introduces refreg, an R package for estimating conditional MVRs. The reference region is non-parametrically estimated using a multivariate kernel density estimator, and its shape allowed to change under the influence of covariates. The effects of covariates on the multivariate variable means, and on their variance-covariance matrix, are estimated by flexible additive predictors. Continuous covariate non-linear effects can be estimated by penalized spline smoothers. The package allows the user to propose, for instance, an age-specific diagnostic rule based on the joint distribution of two non-Gaussian, continuous test results. The usefulness of the refreg package in clinical practice is illustrated with a real case in diabetes research, with an age-specific reference region proposed for the joint interpretation of two glycemia markers (fasting plasma glucose and glycated hemoglobin). To show that the refreg package can also be used in other, and indeed very different fields, an example is provided for the joint prediction of two atmospheric pollutants (SO$_2$, and NO$_x$). Additionally, the text discusses how, conceptually, this method could be extended to more than two dimensions.

KDSource, a tool for the generation of Monte Carlo particle sources using kernel density estimation

Monte Carlo radiation transport simulations have clearly contributed to improve the design of nuclear systems. When performing in-beam or shielding simulations a complexity arises due to the fact that particles must be tracked to regions far from the original source or behind the shielding, often lacking sufficient statistics. Different possibilities to overcome this problem such as using particle lists or generating synthetic sources have already been reported. In this work we present a new approach by using the adaptive multivariate kernel density estimator (KDE) method. This concept was implemented in KDSource, a general tool for modelling, optimizing and sampling KDE sources, which provides a convenient user interface. The basic properties of the method were studied in an analytical problem with a known density distribution. Furthermore, the tool was used in two Monte Carlo simulations that modelled neutron beams, which showed good agreement with experimental results.

Interval Prediction of Photovoltaic Power Using Improved NARX Network and Density Peak Clustering Based on Kernel Mahalanobis Distance

Photovoltaic (PV) power forecasting can provide strong support for the safe operation of the power system. Existing forecasting methods are ineffective for grid scheduling decisions or risk analysis. The novel multicluster interval prediction method is proposed to consider the volatility and randomness of PV power output. First, this method utilizes the sparse autoencoder (SAE) and Bayesian regularized NARX network (BRNARX) for point forecasting of PV power. Second, density peak clustering improved by kernel Mahalanobis distance (KMDDPC) is applied to classify the dataset into multiple clusters, including forecasting error and meteorological factors. Finally, the joint probability density is established by multivariate kernel density estimation (MKDE) to accomplish the PV power interval prediction. The proposed hybrid method is applied for the interval prediction of PV power at Yulara, Australia. Comparative research of point forecasting is implemented to evaluate the machine learning and deep learning methods, with the proposed SAE-BRNARX under four different periods. Results shows that the average values of nRMSE, MRE, nMAE, and R2 for the four periods are 4.45%, 0.90%, −0.15%, 3.39%, and 95.93%, respectively. Moreover, the results of interval prediction obtained by the other interval prediction approaches are compared with the proposed KMDDPC-MKDE. It shows that the average values of PICP, PINAW, ACE, and nMPICD for four periods are 93.93%, 9.50%, 3.93%, and 7.10% at 90% confidence level, respectively. Outcomes demonstrate that the proposed method can obtain more accuracy, a higher coverage rate, narrower average bandwidth, and a closer distance between the middle of interval and actual value than other methods.

Laser ablation inductively coupled plasma mass spectrometry analysis of potash and m-Na-Al glasses in China- using Kernel methods for trace element analysis

Monochrome drawn beads were widely circulated in South and Southeast Asia as early as the second century BC. This article aims to identify the glass beads unearthed from different sites in China and discuss their possible sources. Twenty-seven mineral soda alumina (m-Na-Al) glass and eighty-seven potash glass beads unearthed in different provinces in China were analysed by Laser Ablation Inductively Coupled Plasma Mass Spectrometry. The trace element analysis through Multivariate Kernel Density Estimation and Maximum Mean Discrepancy indicates the silica sources of m-Na-Al glass and most of the potash glass unearthed from Guangxi are identical. They were presumably produced somewhere in northeastern India or Southeast Asia and exported through the Maritime Silk Road. The silica sources of m-Na-Al glass in Henan and the rest of the potash glasses are geologically close. They were likely produced in southern India or Sri Lanka and exported through the North and Southwest Silk Roads. Future research on isotopic analysis will reveal more information about primary/secondary glass production in China, South and Southeast Asia.

Probabilistic Revenue Analysis of Microgrid Considering Source-Load and Forecast Uncertainties

Due to the randomness of load and renewable energy generation (REG), microgrids face multiple uncertainties. These uncertainties lead to the uncertainty of microgrid operation and bring more challenges to the economic evaluation of microgrids. In this paper, an economic evaluation method for determining microgrid revenue distribution is proposed. Considering the dual uncertainties of source-load and forecast, and temporal autocorrelation of time series, the probabilistic model of uncertainties is established by multivariate kernel density estimation (KDE). Then the random scenarios including forecasting values are generated and used in optimal dispatch calculation for the detailed production simulation. The probabilistic revenue is derived with a method based on Monte Carlo method. Finally, a case study is carried out based on the real data of an industrial park. The results demonstrate the necessity and effectiveness of the probabilistic revenue analysis proposed in this paper. This method can reveal the actual values of each component of a microgrid (e.g., device or algorithm) in specific scenes and provides more insights into investment decisions.

Supervised Multivariate Kernel Density Estimation for Enhanced Plasma Etching Endpoint Detection

The advancement of semiconductor technology nodes requires precise control of their manufacturing process, including plasma etching, which is highly important in terms of the yield, cost, and device performance. Endpoint detection (EPD) is an imperative technique for controlling this process. Here, we propose a novel EPD scheme based on multivariate kernel density estimation (MKDE). The proposed approach is developed by extending the conventional unsupervised learning MKDE method to supervised learning. The performance of the proposed scheme is validated on randomly selected optical emission spectroscopy data collected from an industrial semiconductor manufacturing process. Because the proposed approach uses target values (labeling) of data, it demonstrates enhanced EPD performance compared to the conventional MKDE method, even without threshold presetting.

A local density-based outlier detection method for high dimension data

The researchers faced challenges in the outlier detection process, mainly when deals with the high dimensional dataset; to handle this problem, we use The principal component analysis. Outlier detection or anomaly detection, with local density-based methods, compares the density of observation with the surrounding local density neighbors. We apply the outlier score as a measure of comparison. In this research, we choose different density estimation functions and calculated different distances. Weighted kernel density estimation with adaptive bandwidth for multivariate kernel density estimation(Gaussian) considered the $KNN$ and RNN. $KNN$ is considered too for the Epanenchnikov kernel density estimation. Lastly, we estimate the LOF as a base method in detecting outliers. Extensive experiments on a synthetic dataset have shown that RKDOS and EPA are more efficient than LOF using the precision evaluation criterion.

Laser ablation inductively coupled plasma mass spectrometry analysis of Chinese lead-barium glass: combining multivariate kernel density estimation and maximum mean discrepancy to reinterpret the raw glass used for producing lead-barium glass

Laser ablation inductively coupled plasma mass spectrometry analysis of Chinese lead-barium glass: combining multivariate kernel density estimation and maximum mean discrepancy to reinterpret the raw glass used for producing lead-barium glass

Strong consistency of the mode of multivariate recursive kernel density estimator under strong mixing hypothesis

In this research paper, we define a kernel estimator of the mode based on the recursive kernel density estimator developed by [23]. In addition, we establish its almost sure convergence under strong mixing hypothesis. Finally, we corroborate these theoretical results through numerical simulations.

An Efficient Radio Map Learning Scheme Based on Kernel Density Function

Training an accurate and up-to-date radio map has always been a primary concern for implementing a WiFi fingerprint-based localization system. This paper presents a novel radio map learning scheme for online learning a set of kernel density functions, which function like a traditional radio map in WiFi fingerprint-based localization systems. To be specific, each kernel density function corresponds to one specific access point (AP) and predicts the probability of having a received signal strength (RSS) measurement from this AP at any position. In the offline phase, a set of coarse-grained kernel density functions are initially established by the Multivariate Kernel Density Estimation (MKDE) through a fast site survey. In the online phase, these kernel density functions are recursively refined through Incremental-MKDE (IMKDE) using freshly crowdsourced fingerprints, which essentially fuses the historical features in the previous kernel density functions and up-to-date features in the current fingerprints. A compression technique is applied to improve the computation and storage efficiency. In addition, pedestrian dead reckoning (PDR) is leveraged to calibrate the location labels of crowdsourced fingerprints. Extensive experiments are carried out in a real scenario of nearly 1000 <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$m^2$</tex-math></inline-formula> for five months, and a comparison made between the IMKDE based method and several existing popular solutions confirms the superiority of the proposed IMKDE in terms of both computational complexity and localization accuracy.

On the Use of a Modified Intersection of Confidence Intervals (MICIH) Kernel Density Estimation Approach

Density estimation is an important aspect of statistics. Statistical inference often requires the knowledge of observed data density. A common method of density estimation is the kernel density estimation (KDE). It is a nonparametric estimation approach which requires a kernel function and a window size (smoothing parameter H). It aids density estimation and pattern recognition. So, this work focuses on the use of a modified intersection of confidence intervals (MICIH) approach in estimating density. The Nigerian crime rate data reported to the Police as reported by the National Bureau of Statistics was used to demonstrate this new approach. This approach in the multivariate kernel density estimation is based on the data. The main way to improve density estimation is to obtain a reduced mean squared error (MSE), the errors for this approach was evaluated. Some improvements were seen. The aim is to achieve adaptive kernel density estimation. This was achieved under a sufficiently smoothing technique. This adaptive approach was based on the bandwidths selection. The quality of the estimates obtained of the MICIH approach when applied, showed some improvements over the existing methods. The MICIH approach has reduced mean squared error and relative faster rate of convergence compared to some other approaches. The approach of MICIH has reduced points of discontinuities in the graphical densities the datasets. This will help to correct points of discontinuities and display adaptive density. Keywords: approach, bandwidth, estimate, error, kernel density