Adaptive Kernel Density Estimation Research Articles

Robust Parameter Optimisation of Noise-Tolerant Clustering for DENCLUE Using Differential Evolution

Clustering samples based on similarity remains a significant challenge, especially when the goal is to accurately capture the underlying data clusters of complex arbitrary shapes. Existing density-based clustering techniques are known to be best suited for capturing arbitrarily shaped clusters. However, a key limitation of these methods is the difficulty in automatically finding the optimal set of parameters adapted to dataset characteristics, which becomes even more challenging when the data contain inherent noise. In our recent work, we proposed a Differential Evolution-based DENsity CLUstEring (DE-DENCLUE) to optimise DENCLUE parameters. This study evaluates DE-DENCLUE for its robustness in finding accurate clusters in the presence of noise in the data. DE-DENCLUE performance is compared against three other density-based clustering algorithms—DPC based on weighted local density sequence and nearest neighbour assignment (DPCSA), Density-Based Spatial Clustering of Applications with Noise (DBSCAN), and Variable Kernel Density Estimation–based DENCLUE (VDENCLUE)—across several datasets (i.e., synthetic and real). The study has consistently shown superior results for DE-DENCLUE compared to other models for most datasets with different noise levels. Clustering quality metrics such as the Silhouette Index (SI), Davies–Bouldin Index (DBI), Adjusted Rand Index (ARI), and Adjusted Mutual Information (AMI) consistently show superior SI, ARI, and AMI values across most datasets at different noise levels. However, in some cases regarding DBI, the DPCSA performed better. In conclusion, the proposed method offers a reliable and noise-resilient clustering solution for complex datasets.

Distribution reconstruction and reliability assessment of complex LSFs via an adaptive Non-parametric Density Estimation Method

Complex limit state functions (LSFs), often characterized by strong nonlinearity, non-smoothness, or discontinuity, pose challenges for structural reliability analysis in engineering practices. Conventional methods for uncertainty propagation and reliability assessment may struggle to handle these issues effectively. This paper introduces a novel approach to adaptively reconstruct the unknown distributions of complex LSFs. The Non-parametric Density Estimation Method based on Harmonic Transform (NDEM-HT) is employed as the tool for this purpose. An adaptive strategy is then proposed to determine the number of harmonic moments required in NDEM-HT for achieving high accuracy. Specifically, the Adaptive Kernel Density Estimation (AKDE) method is also adopted to provide an initial estimation of the rough distribution. Subsequently, the optimal number of harmonic moments is determined by minimizing the relative entropy between the distributions obtained by AKDE and NDEM-HT. The efficacy of the proposed method is demonstrated through five numerical examples, considering various types of complex LSFs. Comparative results are also provided employing MCS along with both conventional and state-of-the-art methods.

Multi-step ozone concentration prediction model based on improved secondary decomposition and adaptive kernel density estimation

Accurately predicting ozone concentration is crucial for human pollution prevention and health protection. In recent years, decomposition ensemble frameworks have been widely applied to ozone concentration forecasting. However, due to the inefficiencies of secondary decomposition and insufficient emphasis on interval predictions, developing a stable and efficient model for ozone concentration prediction remains a challenging task. Therefore, this study proposes a hybrid model based on an improved secondary decomposition algorithm (ISD) and adaptive kernel density estimation (AKDE). The original sequences are initially decomposed and reconstructed using Complex Empirical Mode Decomposition (CEEMD) and K-means clustering based on sample entropy. Subsequently, Optimal Variational Mode Decomposition (OVMD) is employed to address the strong fluctuations in high-frequency components, followed by establishing individual Long Short-Term Memory (LSTM) prediction models for each component. The error correction model is then constructed by combining ISD and Gated circulation unit (GRU), culminating in uncertainty quantification using AKDE. Empirical analysis of the Beijing and Shanghai, China datasets shows that the average R2 of their multi-step forecasts are 0.9966 and 0.9938, respectively, which are significantly better than the baseline model.

Probabilistic load flow based voltage stability assessment of solar power integration into power grids

The interconnection of renewable energies increases the complexity of modern power systems. Hence, stability assessments should be made to ensure the system’s stability after penetration. Solar power is a type of renewable energy that has become a widespread energy source among renewable energy sources. About 1 MWp of the solar power plant has been prepared to be interconnected to the IEEE 8 bus of Manokwari grid, and this paper investigates the voltage profile, power losses, and stability of the solar power plant penetration using an adaptive kernel density estimator (AKDE) and compares it to a Monte Carlo simulation (MCS)-based probabilistic load flow (PLF). About 5000 samples have been used to test the grid after the connection. Results of simulations show that the solar penetration can reduce power losses from 0.4084 MW to 0.3080 MW and 0.3045 MW by the proposed method and MCS method, respectively, and further increase the bus voltage profile. The power network has the stability to be connected to solar power, as indicated by the small stability index values of each bus. The proposed method using the AKDE method has a more accurate result in stability indices indicated by small fast voltage stability index (FVSI), line stability index (Lmn), and line stability factor (LQP) indices.

A Time-Series Feature-Extraction Methodology Based on Multiscale Overlapping Windows, Adaptive KDE, and Continuous Entropic and Information Functionals

We propose a new methodology to transform a time series into an ordered sequence of any entropic and information functionals, providing a novel tool for data analysis. To achieve this, a new algorithm has been designed to optimize the Probability Density Function (PDF) associated with a time signal in the context of non-parametric Kernel Density Estimation (KDE). We illustrate the applicability of this method for anomaly detection in time signals. Specifically, our approach combines a non-parametric kernel density estimator with overlapping windows of various scales. Regarding the parameters involved in the KDE, it is well-known that bandwidth tuning is crucial for the kernel density estimator. To optimize it for time-series data, we introduce an adaptive solution based on Jensen–Shannon divergence, which adjusts the bandwidth for each window length to balance overfitting and underfitting. This solution selects unique bandwidth parameters for each window scale. Furthermore, it is implemented offline, eliminating the need for online optimization for each time-series window. To validate our methodology, we designed a synthetic experiment using a non-stationary signal generated by the composition of two stationary signals and a modulation function that controls the transitions between a normal and an abnormal state, allowing for the arbitrary design of various anomaly transitions. Additionally, we tested the methodology on real scalp-EEG data to detect epileptic crises. The results show our approach effectively detects and characterizes anomaly transitions. The use of overlapping windows at various scales significantly enhances detection ability, allowing for the simultaneous analysis of phenomena at different scales.

GalaxyFlow: upsampling hydrodynamical simulations for realistic mock stellar catalogues

ABSTRACT Cosmological N-body simulations of galaxies operate at the level of ‘star particles’ with a mass resolution on the scale of thousands of solar masses. Turning these simulations into stellar mock catalogues requires ‘upsampling’ the star particles into individual stars following the same phase-space density. In this paper, we introduce two new upsampling methods. First, we describe GalaxyFlow, a sophisticated upsampling method that utilizes normalizing flows to both estimate the stellar phase-space density and sample from it. Secondly, we improve on existing upsamplers based on adaptive kernel density estimation (KDE), using maximum likelihood estimation to fine-tune the bandwidth for such algorithms in a way that improves both the density estimation accuracy and upsampling results. We demonstrate our upsampling techniques on a neighbourhood of the Solar location in two simulated galaxies: Auriga 6 and h277. Both yield smooth stellar distributions that closely resemble the stellar densities seen in the Gaia DR3 catalogue. Furthermore, we introduce a novel multimodel classifier test to compare the accuracy of different upsampling methods quantitatively. This test confirms that GalaxyFlow more accurately estimates the density of the underlying star particles than methods based on KDE, at the cost of being more computationally intensive.

Stable training of probabilistic models using the leave-one-out maximum log-likelihood objective

Probabilistic modelling of power systems operation and planning processes depends on data-driven methods, which require sufficiently large datasets. When historical data lacks this, it is desired to model the underlying data generation mechanism as a probability distribution to assess the data quality and generate more data, if needed. Kernel density estimation (KDE) based models are popular choices for this task, but they fail to adapt to data regions with varying densities. In this paper, an adaptive KDE model is employed to circumvent this, where each kernel in the model has an individual bandwidth. The leave-one-out maximum log-likelihood (LOO-MLL) criterion is proposed to prevent the singular solutions that the regular MLL criterion gives rise to, and it is proven that LOO-MLL prevents these. Relying on this guaranteed robustness, the model is extended by adjustable weights for the kernels. In addition, a modified expectation–maximization algorithm is employed to accelerate the optimization speed reliably. The performance of the proposed method and models are exhibited on two power systems datasets using different statistical tests and by comparison with Gaussian mixture models. Results show that the proposed models have promising performance, in addition to their singularity prevention guarantees.

Uncovering the progressive failure process of primary coal-rock mass specimens: Insights from energy evolution, acoustic emission crack patterns, and visual characterization

The failure and instability of deep coal-rock mass structures, which are influenced by mining disturbances, are major contributors to disasters like rock bursts. This study involved conducting indoor experiments using uniaxial loading test and acoustic emission (AE) experiments on indigenous primary coal-rock mass (PCRM). We utilize chaotic bifurcation based on strain energy to characterize the progressive damage stages of PCRM, demonstrating the chaotic characteristics of energy evolution during the gradual destruction process. The degree of damage resulting from PCRM failure was evaluated by jointly applying P-wave velocity tomography and AE event localization. Additionally, the classification of crack patterns in PCRM was conducted using the RA-AF correlation analysis method. The segmentation boundaries of crack classification patterns were then redefined using the Gaussian mixture model (GMM) algorithm. The research findings indicate that the chaotic bifurcation model of strain energy classifies stress into four stages: the stable region (Ⅰ), the metastable region (Ⅱ), the bifurcation region (Ⅲ), and the chaotic region (Ⅳ). The presence of a low-velocity zone in P-waves may indicate crack accumulation and damage within the system. Additionally, areas with abnormal P-wave velocities exhibit numerous AE events. The crack classification patterns of the four stages undergo an evolution from predominantly tensile cracks to a mixture of tensile and shear cracks, and the GMM algorithm successfully identifies the optimal separation path for crack classification boundaries. We propose an adaptive kernel density estimation (AKDE) algorithm to quantify the spatial distribution of AE events, thus offering a visual representation of the damage patterns at various stages.

A novel framework for high resolution air quality index prediction with interpretable artificial intelligence and uncertainties estimation

Accurate air quality index (AQI) prediction is essential in environmental monitoring and management. Given that previous studies neglect the importance of uncertainty estimation and the necessity of constraining the output during prediction, we proposed a new hybrid model, namely TMSSICX, to forecast the AQI of multiple cities. Firstly, time-varying filtered based empirical mode decomposition (TVFEMD) was adopted to decompose the AQI sequence into multiple internal mode functions (IMF) components. Secondly, multi-scale fuzzy entropy (MFE) was applied to evaluate the complexity of each IMF component and clustered them into high and low-frequency portions. In addition, the high-frequency portion was secondarily decomposed by successive variational mode decomposition (SVMD) to reduce volatility. Then, six air pollutant concentrations, namely CO, SO2, PM2.5, PM10, O3, and NO2, were used as inputs. The secondary decomposition and preliminary portion were employed as the outputs for the bidirectional long short-term memory network optimized by the snake optimization algorithm (SOABiLSTM) and improved Catboost (ICatboost), respectively. Furthermore, extreme gradient boosting (XGBoost) was applied to ensemble each predicted sub-model to acquire the consequence. Ultimately, we introduced adaptive kernel density estimation (AKDE) for interval estimation. The empirical outcome indicated the TMSSICX model achieved the best performance among the other 23 models across all datasets. Moreover, implementing the XGBoost to ensemble each predicted sub-model led to an 8.73%, 8.94%, and 0.19% reduction in RMSE, compared to SVM. Additionally, by utilizing SHapley Additive exPlanations (SHAP) to assess the impact of the six pollutant concentrations on AQI, the results reveal that PM2.5 and PM10 had the most notable positive effects on the long-term trend of AQI. We hope this model can provide guidance for air quality management.

Point Cloud Denoising and Feature Preservation: An Adaptive Kernel Approach Based on Local Density and Global Statistics.

Noise removal is a critical stage in the preprocessing of point clouds, exerting a significant impact on subsequent processes such as point cloud classification, segmentation, feature extraction, and 3D reconstruction. The exploration of methods capable of adapting to and effectively handling the noise in point clouds from real-world outdoor scenes remains an open and practically significant issue. Addressing this issue, this study proposes an adaptive kernel approach based on local density and global statistics (AKA-LDGS). This method constructs the overall framework for point cloud denoising using Bayesian estimation theory. It dynamically sets the prior probabilities of real and noise points according to the spatial function relationship, which varies with the distance from the points to the center of the LiDAR. The probability density function (PDF) for real points is constructed using a multivariate Gaussian distribution, while the PDF for noise points is established using a data-driven, non-parametric adaptive kernel density estimation (KDE) approach. Experimental results demonstrate that this method can effectively remove noise from point clouds in real-world outdoor scenes while maintaining the overall structural features of the point cloud.

A new method for analysis of marine sustainable energy systems

This paper proposes to utilise a new adaptive kernel density estimation (KDE) methodology based on linear diffusion processes for predicting the probability distribution tails of sea-state parameters. A key conclusion was reached that the proposed new methodology can lead to more accurate prediction results than traditional methods based on fittings to a measured significant wave height data set at National Data Buoy Center station 46014. This proposed methodology was subsequently utilised for deriving an accurate 50-year environmental contour line that was used in the dynamic analysis of a two-body point absorber wave energy converter. After systematically analysing the calculation results, another key conclusion was drawn that it is advantageous to use a more reliable contour line derived using the proposed new methodology for long-term dynamic analysis of wave energy converters. In summary, the proposed new adaptive KDE methodology is recommended to be utilised and to be refined continuously in future research work in the field of long-term reliability analysis of marine sustainable energy systems.

Method for analysing of wind power wave nature based on kernel density estimation

It is beneficial for improving the accuracy of wind power output prediction by analysing and mastering the inherent laws of the fluctuation characteristics of wind power output, guiding the power grid dispatching department to reasonably arrange power generation plans, and improving the economic efficiency of system operation. In order to characterize the probability density distribution of wind power output fluctuation, two adaptive bandwidth kernel density estimation models are established by correcting the fixed bandwidth obtained from the empirical method and unbiased cross-validation method respectively, and then the two models are combined and optimized, and ultimately, the probability density distribution model of wind power output fluctuation based on Hybrid Adaptive Kernel Density Estimation (HAKDE) is established. A variety of probability density distribution models are used to fit the wind power output fluctuations at different spatial and temporal scales in a province in North China, and the example results show that the hybrid adaptive kernel density estimation model has the best fitting effect, thus verifying the effectiveness of the hybrid adaptive kernel density estimation model.

A new method for long-term safety analysis of marine structures

This paper proposes to utilise a new adaptive kernel density estimation (KDE) method based on a linear diffusion process for fitting the significant wave height data in order to calculate the sea state parameter distribution tails more accurately, particularly the higher, more extreme values. The accuracy of the proposed method has been demonstrated based on fittings to two measured ocean wave datasets. This proposed method has subsequently been utilised for deriving accurate 50-year and 500-year environmental contour lines. The advantages of using a more reliable contour line derived from using the proposed new method for long-term safety analysis of marine structures have been substantiated.

Research on a Deep Ensemble Learning Model for the Ultra-Short-Term Probabilistic Prediction of Wind Power

An accurate method for predicting wind power is crucial in effectively mitigating wind energy fluctuations and ensuring a stable power supply. Nevertheless, the inadequacy of the stability of wind energy severely hampers the consistent functioning of the power grid and the reliable provision of electricity. To enhance the accuracy of wind power forecasting, this paper proposes an ensemble model named the complete ensemble empirical mode decomposition with adaptive noise (CEEMDAN) and convolutional bidirectional long short-term memory (CNN-BiLSTM), which incorporates a data preprocessing technique, feature selection method, deep ensemble model, and adaptive control. Initially, CEEMDAN is utilized to decompose wind speed and power sequences and hence obtain decomposed subsequences for further analysis. Subsequently, the CNN is used to extract features from each subsequence, whereas each subsequence is processed by BiLSTM to obtain an ultra-short-term deterministic prediction model. Additionally, the adaptive kernel density estimation (AKDE) method is employed to estimate the probabilistic distribution of prediction error, enabling ultra-short-term probabilistic wind power prediction. Finally, based on real datasets, the reliability of the model in probabilistic prediction is verified through the evaluation metrics of multi-step prediction intervals (PIs).

Probabilistic Load Flow Analysis Using Nonparametric Distribution

In the pursuit of sustainable energy solutions, this research addresses the critical need for accurate probabilistic load flow (PLF) analysis in power systems. PLF analysis is an essential tool for estimating the statistical behavior of power systems under uncertainty. It plays a vital part in power system planning, operation, and dependability studies. To perform accurate PLF analysis, this article proposes a Kernel density estimation with adaptive bandwidth for probability density function (PDF) estimation of power injections from sustainable energy sources like solar and wind, reducing errors in PDF estimation. To reduce the computational burden, a Latin hypercube sampling approach was incorporated. Input random variables are modeled using kernel density estimation (KDE) in conjunction with Latin hypercube sampling (LHS) for probabilistic load flow (PLF) analysis. To test the proposed techniques, IEEE 14 and IEEE 118 bus systems are used. Two benchmark techniques, the Monte Carlo Simulation (MCS) method and Hamiltonian Monte Carlo (HMC), were set side by side for validation of results. The results illustrate that an adaptive bandwidth kernel density estimation with the Latin hypercube sampling (AKDE-LHS) method provides better performance in terms of precision and computational efficiency. The results also show that the suggested technique is more feasible in reducing errors, uncertainties, and computational time while depicting arbitrary distributions of photovoltaic and wind farms for probabilistic load flow analysis. It can be a potential solution to tackle challenges posed by sustainable energy sources in power systems.

Scenario-based ultra-short-term rolling optimal operation of a photovoltaic-energy storage system under forecast uncertainty

The rapid development of renewable energy sources (RESs) facilitates the coordinated operation of different energy sources to hedge against the uncertain and non-dispatchable nature of RESs. In this paper, we propose an effective approach for ultra-short-term optimal operation of a photovoltaic-energy storage hybrid generation system (PV-ES HGS) under forecast uncertainty. First, a generic approach for modelling forecast uncertainty is designed to capture PV output characteristics in the form of scenarios. Then, stochastic model predictive control (SMPC) is utilized in the coordinated operation of PV-ES HGS for load shifting under forecast uncertainty. Finally, a PV-ES power station located in Northeast China is selected as a case study to validate the feasibility and effectiveness of the proposed method in detail. The results show that: (1) The forecast uncertainty modelling method, which incorporates adaptive kernel density estimation (KDE) and two-layer nested copula, could capture the characteristics of PV output effectively. (2) The proposed sampling procedure ensures the representativeness and temporal dependence of scenarios extracted from the joint predictive density, leading to stability and efficiency in solving the stochastic optimization problems. (3) The SMPC-based coordinated operation strategy exhibits significant superiority in load shifting performance and resistance to risks. The standard deviation of residual load (SDORL) is reduced by 17.5% on average compared to the conventional scheme. Thus, this work provides operators with guidance in the ultra-short-term optimal operation of a PV-ES HGS under forecast uncertainty.

Risk Assessment Based on KDE of Ship Collision Candidates for Ship Routing Waterway

To study the geographical distribution characteristics of maritime traffic risks, statistical representations of potential accident scenarios and macro collision risk models were established, and the waters with higher maritime traffic risks were generated. To better evaluate the risk of ship collision candidates during routing waterways, an improved adaptive bandwidth kernel density estimation (KDE) is proposed. This proposed algorithm is used for evaluating risk reduction of the ship routing system, which schedules and adjusts maritime traffic in congested harbor waterways. Larger bandwidth can make the hot spot region more obvious on a global scale. Moreover, the bandwidth is positively correlated with the dispersion of points. Concerning the data with sparse point distribution, a larger bandwidth should be used whereas, for data with dense points of interest, a smaller bandwidth should be considered. The results show that the KDE, with optimized bandwidth, can fit the ship encountering distribution and obtain the frequent spots for ship encountering. The comparison between KDE results before and after the ship routing system shows that the hot spots of ship collision candidates are reduced after the ship routing waterway is established.

Adaptive kernel density estimation weighted twin support vector machine and its sample screening method

SummaryIn twin support vector machines (TSVM), noise blurs the boundary between positive and negative samples, increasing the probability of classification errors. In this article, we propose an adaptive kernel density estimation weighted twin support vector machine(AKWTSVM). AKWTSVM uses KDE based on K‐nearest neighbor estimation to calculate the probability density of samples. It automatically selects the optimal bandwidth based on the local density of the samples to improve the robustness of the algorithm. However, TSVM has high time complexity, to reduce the time costs, a sample screening method is proposed for AKWTSVM, named AKWTSVM‐SSM, which is based on the overall distance and local density, and reduces the time costs of the algorithm by reducing the sample size while ensuring the accuracy of the algorithm. The experiment with differently scaled noise environments of 0%, 5%, 10%, 15%, and 20% on 12 UCI datasets validate the accuracy and running time of AKWTSVM and AKWTSVM‐SSM. Experimental results prove the effectiveness and robustness of AKWTSVM, the robustness of AKWTSVM‐SSM, and its applicability to large‐scale datasets.

TAKDE: Temporal Adaptive Kernel Density Estimator for Real-Time Dynamic Density Estimation.

Real-time density estimation is ubiquitous in many applications, including computer vision and signal processing. Kernel density estimation is arguably one of the most commonly used density estimation techniques, and the use of "sliding window" mechanism adapts kernel density estimators to dynamic processes. In this article, we derive the asymptotic mean integrated squared error (AMISE) upper bound for the "sliding window" kernel density estimator. This upper bound provides a principled guide to devise a novel estimator, which we name the temporal adaptive kernel density estimator (TAKDE). Compared to heuristic approaches for "sliding window" kernel density estimator, TAKDE is theoretically optimal in terms of the worst-case AMISE. We provide numerical experiments using synthetic and real-world datasets, showing that TAKDE outperforms other state-of-the-art dynamic density estimators (including those outside of kernel family). In particular, TAKDE achieves a superior test log-likelihood with a smaller run-time.

Incipient Fault Detection in a Hydraulic System Using Canonical Variable Analysis Combined with Adaptive Kernel Density Estimation.

Incipient fault detection in a hydraulic system is a challenge in the condition monitoring community. Existing research mainly monitors abnormal working conditions in hydraulic systems by separately detecting the key working parameter, which often causes a high miss warning rate for incipient faults due to the oversight of parameter dependence. A principal component analysis provides an effective method for incipient fault detection by taking the correlation of multiple parameters into consideration, but this technique assumes the systems are Gaussian-distributed, making it invalid for a dynamic non-Gaussian system. In this paper, we combine a canonical variable analysis (CVA) and adaptive kernel density estimation (AKDE) for the early fault detection of nonlinear dynamic hydraulic systems. The collected hydraulic system data set was used to construct the typical variable space, and the state space and residual space are divided to represent the characteristics of different correlations between the two variables, which are quantitatively described using Hotelling's T2 and Q. In order to investigate the proper upper control limits, AKDE was utilised to estimate the underlying probability density functions of T2 and Q by taking the nonlinearity of the hydraulic system variables into consideration. The advantages of the proposed approach for incipient fault detection are illustrated via a marine power plant lubrication system.