Multivariate Kernel Density Estimation Research Articles

Multivariate mixed kernel density estimators and their application in machine learning for classification of biological objects based on spectral measurements

A problem of non-parametric multivariate density estimation for machine learning and data augmentation is considered. A new mixed density estimation method based on calculating the convolution of independently obtained kernel density estimates for unknown distributions of informative features and a known (or independently estimated) density for non-informative interference occurring during measurements is proposed. Properties of the mixed density estimates obtained using this method are analyzed. The method is compared with a conventional Parzen-Rosenblatt window method applied directly to the training data. The equivalence of the mixed kernel density estimator and the data augmentation procedure based on the known (or estimated) statistical model of interference is theoretically and experimentally proven. The applicability of the mixed density estimators for training of machine learning algorithms for the classification of biological objects (elements of grain mixtures) based on spectral measurements in the visible and near-infrared regions is evaluated.

Incremental novelty detection and fault identification scheme applied to a kinematic chain under non-stationary operation

Classical methods for monitoring electromechanical systems lack two critical functions for effective industrial application: management of unexpected events and the incorporation of new patterns into the knowledge database. This study presents a novel, high-performance condition-monitoring method based on a four-stage incremental learning approach. First, non-stationary operation is characterised using normalised time-frequency maps. Second, operating novelties are detected using multivariate kernel density estimators. Third, the operating novelties are characterised and labelled to increase the knowledge available for subsequent diagnosis. Fourth, operating faults are diagnosed and classified using neural networks. The proposed method is validated experimentally with an industrial camshaft-based machine under a variety of operating conditions.

Adaptive Nonparametric Kernel Density Estimation Approach for Joint Probability Density Function Modeling of Multiple Wind Farms

The uncertainty of wind power brings many challenges to the operation and control of power systems, especially for the joint operation of multiple wind farms. Therefore, the study of the joint probability density function (JPDF) of multiple wind farms plays a significant role in the operation and control of power systems with multiple wind farms. This research was innovative in two ways. One, an adaptive bandwidth improvement strategy was proposed. It replaced the traditional fixed bandwidth of multivariate nonparametric kernel density estimation (MNKDE) with an adaptive bandwidth. Two, based on the above strategy, an adaptive multi-variable non-parametric kernel density estimation (AMNKDE) approach was proposed and applied to the JPDF modeling for multiple wind farms. The specific steps of AMNKDE were as follows: First, the model of AMNKDE was constructed using the optimal bandwidth. Second, an optimal model of bandwidth based on Euclidean distance and maximum distance was constructed, and the comprehensive minimum of these distances was used as a measure of optimal bandwidth. Finally, the ordinal optimization (OO) algorithm was used to solve this model. The scenario results indicated that the overall fitness error of the AMNKDE method was 8.81% and 11.6% lower than that of the traditional MNKDE method and the Copula-based parameter estimation method, respectively. After replacing the modeling object the overall fitness error of the comprehensive Copula method increased by as much as 1.94 times that of AMNKDE. In summary, the proposed approach not only possesses higher accuracy and better applicability but also solved the local adaptability problem of the traditional MNKDE.

Fast and Stable Multivariate Kernel Density Estimation by Fast Sum Updating

Kernel density estimation and kernel regression are powerful but computationally expensive techniques: a direct evaluation of kernel density estimates at M evaluation points given N input sample points requires a quadratic operations, which is prohibitive for large scale problems. For this reason, approximate methods such as binning with fast Fourier transform or the fast Gauss transform have been proposed to speed up kernel density estimation. Among these fast methods, the fast sum updating approach is an attractive alternative, as it is an exact method and its speed is independent of the input sample and the bandwidth. Unfortunately, this method, based on data sorting, has for the most part been limited to the univariate case. In this article, we revisit the fast sum updating approach and extend it in several ways. Our main contribution is to extend it to the general multivariate case for general input data and rectilinear evaluation grid. Other contributions include its extension to a wider class of kernels, including the triangular, cosine, and Silverman kernels, its combination with parsimonious additive multivariate kernels, and its combination with a fast approximate k-nearest-neighbors bandwidth for multivariate datasets. Our numerical tests confirm the speed, accuracy, and stability of the method.

Time resolved measurements of droplet preferential concentration in homogeneous isotropic turbulence without mean flow

An experimental investigation of the time-dependent spatial distribution of droplet concentration in a “box” of stationary homogeneous and isotropic turbulence without mean flow was performed for polydispersed droplet clouds with a wide range of mean droplet diameters and droplet size distributions, characterized by a representative Stokes number, based on the droplet arithmetic diameter and the Kolmogorov time scale of the flow, varying between 0.2 and 6, and for turbulent Reynolds numbers of 98, 127, 147, and 235. A novel morphological tracking scheme was used to provide temporal tracking of droplet clusters according to cluster morphological features. The tracking scheme was based on Voronoï space tessellation, multivariate kernel density estimation, and mean shift feature space tracking. The developed tracking scheme was used to establish the relationship between the morphological features of droplet clusters, i.e., number of clusters per unit area and cluster length scales, and the velocity of droplet clusters. The time scale of the droplet clusters increased with both the turbulent Reynolds number and the Stokes number, but its dependence was stronger on the turbulence level. In addition, the effect of filling factors of turbulent flows by turbulent structures on droplet clustering was discussed, which led to suggestions of potential flow control methods to mitigate droplet preferential concentration.

Data Analytics for Performance Evaluation Under Uncertainties Applied to an Industrial Refrigeration Plant

Artificial intelligence has bounced into industrial applications contributing several advantages to the field and have led to the possibility to open new ways to solve many actual problems. In this paper, a data-driven performance evaluation methodology is presented and applied to an industrial refrigeration system. The strategy takes advantage of the Multivariate Kernel Density Estimation technique and Self-Organizing Maps to develop a robust method, which is able to determine a near-optimal performance map, taking into account the system uncertainties and the multiple signals involved in the process. A normality model is used to detect and filter non-representative operating samples to subsequently develop a reliable performance map. The performance map allows comparing the plant assessment under the same operating conditions and permits to identify the potential system improvement capabilities. To ensure that the resulting evaluation is trustworthy, a robustness strategy is developed to identify either possible new operation conditions or abnormal situations in order to avoid uncertain assessments. Furthermore, the proposed approach is tested with real industrial plant data to validate the suitability of the method.

Unified Probabilistic Modeling of Wind Reserves for Demand Response and Frequency Regulation in Islanded Microgrids

Islanded microgrids provide unique challenges due to the lack of transmission support, requiring local energy supply and grid regulation architecture. This paper presents a unified probabilistic assessment of wind reserves for demand response (DR) and frequency regulation in islanded microgrids. A multivariate nonparametric kernel density estimation algorithm is used to generate the probabilistic models of the wind resource, electrical demand, and predicted performance of wind generation. These models are numerically combined to evaluate the capability of wind generation to act as a dynamic reserve and predict its performance for DR, secondary generation, and frequency regulation in an islanded system. The probabilistic model captures multivariate cross-correlation, nonstationary environmental and load behavior, as well as multimodality in their underlying probability distributions. A case study is conducted to validate the proposed model, which predicts wind generation effectiveness for varying load profiles, wind profiles, and generation capacities. PLEXIM simulation software is used to implement a model microgrid to demonstrate the integration of wind generation and its regulatory capabilities. The proposed algorithm has applications in power system planning and operation, and it provides a method using probabilistic data set for long term energy management and optimization of islanded microgrids.

A Privacy-Aware Architecture at the Edge for Autonomous Real-Time Identity Reidentification in Crowds

The capability to perform identity reidentification in a crowd (e.g., from video feeds from a network of cameras, and social media platforms, such as Facebook and Instagram) efficiently and effectively is increasingly important, as evident in recent real-world events (e.g., terrorist attacks on places of mass gatherings in different countries). However, real-time reidentification in a network of cameras, such as those deployed in a smart city, and from other sources, such as social media platforms, remains a challenging task. In this paper, a new embedding algorithm pipeline is presented to extract and administrate the crowd-sourced facial image features (e.g., social media platforms and multicameras in a dense crowd, such as a stadium or airport). The proposed facial embedding is a privacy-aware parameterized function, which maps facial images to high-dimensional vectors in order to facilitate the identification and tracking of individuals. In other words, we are able to uniquely identify person(s) of interest, without the need to determine their true identity. To extract the facial embedding information in crowds, concurrent residual neural network (ResNet) embedding pipeline for each camera is proposed. Specifically, facial embedding feature vectors are generated in real-time by each camera using the proposed enhanced ResNet architecture, which is trained with vectorized-l2-loss function for face recognition. The multivariate kernel density estimation matching algorithm is then applied to facial embedding pipelines generated by cameras at the fog cloud for identity reidentification and security verification. This allows us to ensure the privacy of individuals captured by the camera without compromising on the capability for identity reidentification. Evaluations using mixed datasets in real-time demonstrate that our proposed approach achieves a 2.6% accuracy over other state-of-the-art approaches.

Multivariate Density Estimation Using a Multivariate Weighted Log-Normal Kernel

This paper suggests a multivariate asymmetric kernel density estimation using a multivariate weighted log-normal (LN) kernel for non-negative multivariate data. Asymptotic properties of the multivariate weighted LN kernel density estimator are studied. Simulation studies are also conducted in the bivariate situation.

Multivariate generalized Birnbaum—Saunders kernel density estimators

ABSTRACTIn this article, we first propose the classical multivariate generalized Birnbaum–Saunders kernel estimator for probability density function estimation in the context of multivariate non negative data. Then, we apply two multiplicative bias correction (MBC) techniques for multivariate kernel density estimator. Some properties (bias, variance, and mean integrated squared error) of the corresponding estimators are also investigated. Finally, the performances of the classical and MBC estimators based on family of generalized Birnbaum–Saunders kernels are illustrated by a simulation study.

Fault and performance management in multi-cloud based NFV using shallow and deep predictive structures

Deployment of network function virtualization (NFV) over multiple clouds accentuates its advantages such as flexibility of virtualization, proximity to customers and lower total cost of operation. However, NFV over multiple clouds has not yet attained the level of performance to be a viable replacement for traditional networks. One of the reasons is the absence of a standard based fault, configuration, accounting, performance and security (FCAPS) framework for the virtual network services. In NFV, faults and performance issues can have complex geneses within virtual resources as well as virtual networks and cannot be effectively handled by traditional rule-based systems. To tackle the above problem, we propose a fault detection and localization model based on a combination of shallow and deep learning structures. Relatively simpler detection has been effectively shown to be handled by shallow machine learning structures such as support vector machine (SVM). Deeper structure, i.e., the stacked autoencoder has been found to be useful for a more complex localization function where a large amount of information needs to be worked through to get to the root cause of the problem. We provide evaluation results using a dataset adapted from fault datasets available on Kaggle and another based on multivariate kernel density estimation and Markov sampling.

3D particle shape modelling and optimization through proper orthogonal decomposition

Based on proper orthogonal decomposition (POD), a new method is presented in order to statistically characterize arbitrary particle shapes using an optimal choice of shape functions identified on a set of 1000 digitized railway ballast particles obtained through 3D Scan. The coefficients of the POD expansion enable a description of ballast grains with varying levels of accuracy. On exploiting the knowledge of their statistical distribution we are able, implementing an appropriate multivariate kernel density estimation method, to generate irregular particles with similar morphological features. The description and generation methods are validated by comparing statistical distributions of basic characteristics: surface area, volume, average radius, elongation, flatness, and aspect ratio. Using suitable geometric descriptors defining local curvatures, we identify which surface points might be regarded as forming faces. This shows that the proposed particle generation method is well suited for irregularly shaped granular materials, as a first geometric definition step, before numerical simulations of their collective mechanical properties are carried out by a Discrete Element code dealing with polyhedral shapes. We illustrate this process with the simple case of the assembling of a granular pack from a loose configuration, by one-dimensional compression, using different levels of accuracy in the representation of grain shape.

Bandwidth Selection for Multivariate Kernel Density Estimation Using Mcmc

Paper not available. Full text of working paper suppressed by author. We provide Markov chain Monte Carlo (MCMC) algorithms for computing the bandwidth matrix for multivariate kernel density estimation. Our approach is based on treating the elements of the bandwidth matrix as parameters to be estimated, which we do by optimizing the likelihood cross-validation criterion. Numerical results show that the resulting bandwidths are superior to all existing methods; for dimensions greater than two, our algorithm is the first practical method for estimating the optimal bandwidth matrix. Moreover, the MCMC algorithm for bandwidth selection for multivariate data has no increased difficulty as the dimension of data increases.

FFT-Based Fast Computation of Multivariate Kernel Density Estimators With Unconstrained Bandwidth Matrices

ABSTRACTThe problem of fast computation of multivariate kernel density estimation (KDE) is still an open research problem. In our view, the existing solutions do not resolve this matter in a satisfactory way. One of the most elegant and efficient approach uses the fast Fourier transform. Unfortunately, the existing FFT-based solution suffers from a serious limitation, as it can accurately operate only with the constrained (i.e., diagonal) multivariate bandwidth matrices. In this article, we describe the problem and give a satisfactory solution. The proposed solution may be successfully used also in other research problems, for example, for the fast computation of the optimal bandwidth for KDE. Supplementary materials for this article are available online.

FFT-based fast bandwidth selector for multivariate kernel density estimation

The performance of multivariate kernel density estimation (KDE) depends strongly on the choice of bandwidth matrix. The high computational cost required for its estimation provides a big motivation to develop fast and accurate methods. One of such methods is based on the Fast Fourier Transform. However, the currently available implementation works very well only for the univariate KDE and its multivariate extension suffers from a very serious limitation as it can accurately operate only with diagonal bandwidth matrices. A more general solution is presented where the above mentioned limitation is relaxed. Moreover, the presented solution can be easily adopted also for the task of efficient computation of integrated density derivative functionals involving an arbitrary derivative order. Consequently, bandwidth selection for kernel density derivative estimation is also supported. The practical usability of the new solution is demonstrated by comprehensive numerical simulations.

A cautionary note on the use of hypervolume kernel density estimators in ecological niche modelling

AbstractBlonder et al. (, Global Ecology and Biogeography, 23, 595–609) introduced a new multivariate kernel density estimation (KDE) method to infer Hutchinsonian hypervolumes in the modelling of ecological niches. The authors argued that their KDE method matches or outperforms several methods for estimating hypervolume geometries and for conducting species distribution modelling. Further clarification, however, is appropriate with respect to the assumptions and limitations of KDE as a method for species distribution modelling. Using virtual species and controlled environmental scenarios, we show that KDE both under‐ and overestimates niche volumes depending on the dimensionality of the dataset and the number of occurrence records considered. We suggest that KDE may be a viable approach when dealing with large sample sizes, limited sampling bias and only a few environmental dimensions.

A Kernel Density Estimator-Based Maximum A Posteriori Image Reconstruction Method for Dynamic Emission Tomography Imaging.

A novel maximum a posteriori (MAP) method for dynamic single-photon emission computed tomography image reconstruction is proposed. The prior probability is modeled as a multivariate kernel density estimator (KDE), effectively modeling the prior probability non-parametrically, with the aim of reducing the effects of artifacts arising from inconsistencies in projection measurements in low-count regimes where projections are dominated by noise. The proposed prior spatially and temporally limits the variation of time-activity functions (TAFs) and attracts similar TAFs together. The similarity between TAFs is determined by the spatial and range scaling parameters of the KDE-like prior. The resulting iterative image reconstruction method is evaluated using two simulated phantoms, namely the extended cardiac-torso (XCAT) heart phantom and a simulated Mini-Deluxe Phantom. The phantoms were chosen to observe the effects of the proposed prior on the TAFs based on the vicinity and abutments of regions with different activities. Our results show the effectiveness of the proposed iterative reconstruction method, especially in low-count regimes, which provides better uniformity within each region of activity, significant reduction of spatiotemporal variations caused by noise, and sharper separation between different regions of activity than expectation maximization and an MAP method employing a more traditional Gibbs prior.

Pointwise and uniform convergence of multivariate kernel density estimators using random bandwidths

ABSTRACTWe obtain the rates of pointwise and uniform convergence of multivariate kernel density estimators using a random bandwidth vector obtained by some data-based algorithm. We are able to obtain faster rate for pointwise convergence. The uniform convergence rate is obtained under some moment condition on the marginal distribution. The rates are obtained under i.i.d. and strongly mixing type dependence assumptions.

Estimating the Density of Ethnic Minorities and Aged People in Berlin: Multivariate Kernel Density Estimation Applied to Sensitive Georeferenced Administrative Data Protected Via Measurement Error

SummaryModern systems of official statistics require the timely estimation of area-specific densities of subpopulations. Ideally estimates should be based on precise geocoded information, which is not available because of confidentiality constraints. One approach for ensuring confidentiality is by rounding the geoco-ordinates. We propose multivariate non-parametric kernel density estimation that reverses the rounding process by using a measurement error model. The methodology is applied to the Berlin register of residents for deriving density estimates of ethnic minorities and aged people. Estimates are used for identifying areas with a need for new advisory centres for migrants and infrastructure for older people.

Co-evolution of hydrological components under climate change scenarios in the Mediterranean area

The Mediterranean area is historically characterized by high human pressure on water resources. Today, while climate is projected to be modified in the future, through precipitation decrease and temperature increase, that jointly and non-linearly may affect runoff, concerns about water availability are increasing. For these reasons, quantitative assessment of future modifications in the mean annual water availability are important; likewise, the description of the future interannual variability of some hydrological components such as runoff and evapotranspiration are highly wished for water management and ecosystems dynamics analyses. This study investigates at basin spatial scale future runoff and evapotranspiration, exploring their probability density functions and their interdependence as functions of climatic changes. In order to do that, a parsimonious conceptual lumped model is here used. The model is forced by different future climate scenarios, generated through a weather generator based on a stochastic downscaling of an ensemble of General Circulation Models (GCMs) realizations. The use of the adopted hydrological model, under reliable stochastic future climate scenarios, allows to project future values of evapotranspiration and runoff in a probabilistic framework and, at the same time, the evaluation of their bivariate frequency distributions for changes through the Multivariate Kernel Density Estimation method. As a case study, a benchmark Mediterranean watershed has been proposed (Imera Meridionale, Italy). Results suggest a radical shift and shape modification of the annual runoff and evapotranspiration probability density functions. Possible implications and impacts on water resources management are here addressed and discussed.