Parzen Window Research Articles

Enhancing Hopfield network performance for pattern retrieval using sparse recovery algorithm and Parzen estimator

An improved pattern recovery approach that integrates the Hopfield neural network (HNN) with the iterative signal reconstruction and Parzen window-based classification is proposed. The HNN is observed as a form of associative memory network, used for various pattern recognition and optimization tasks. However, when the input pattern is highly damaged with a very limited set of available samples, the Hopfield network fails to perform the retrieval. The convex optimization-based gradient descent algorithm is considered for pattern recovery of damaged inputs in order to provide an improved pattern approximation for further processing within the HNN, enabling successful network performance. Additionally, in the case of grayscale images, the Parzen window approach is used to classify the probability density functions (pdfs) of the training set and to choose those being comparable to the pdf of the input pattern, therefore refining the selection of patterns and providing better convergence to the exact retrieval. The theoretical considerations are verified experimentally, showing the high performance of the proposed approach when only 10 % of the pixels are available for binary patterns and 40 % of pixels for grayscale patterns.

A macroscopic safety indicator for road segment: application of entropy theory

Road traffic safety is confronted with unique challenges worldwide due to the varied road network, high population density, and the rapid increase in vehicle numbers. This necessitates an effective evaluation of traffic safety at the road segment level to improve traffic safety management. Existing safety indicators, such as crash frequency (which lacks real-time availability) and Surrogate Safety Measures (focused on micro-level interactions), fall short in quantifying the overall safety status of road segments. To overcome these limitations, we introduce the concept of entropy, typically used to measure disorder in complex systems, into road traffic safety assessment. Our research presents a novel macroscopic safety indicator named ‘velocity entropy,’ along with its calculation method, which involves discretization and the Parzen window technique. The efficacy of velocity entropy is validated using the UTE dataset, a high-fidelity trajectory dataset from China, showing a more significant correlation with traffic conflicts than other macroscopic traffic properties. Notably, velocity entropy can track the temporal and spatial evolution of traffic safety, providing insights into the dynamic development of traffic risks. Overall, velocity entropy offers an innovative, practical, and theoretically meaningful tool for assessing and predicting road safety.

A Hierarchical Heuristic Architecture for Unmanned Aerial Vehicle Coverage Search with Optical Camera in Curve-Shape Area

This paper focuses on the problem of dynamic target search in a curve-shaped area by an unmanned aerial vehicle (UAV) with an optical camera. Our objective is to generate an optimal path for UAVs to obtain the maximum detection reward by a camera in the shortest possible time, while satisfying the constraints of maneuverability and obstacle avoidance. First, based on prior qualitative information, the original target probability map for the curve-shaped area is modeled by Parzen windows with 1-dimensional Gaussian kernels, and then several high-value curve segments are extracted by density-based spatial clustering of applications with noise (DBSCAN). Then, given an example that a target floats down river at a speed conforming to beta distribution, the downstream boundary of each curve segment in the future time is expanded and predicted by the mean speed. The rolling self-organizing map (RSOM) neural network is utilized to determine the coverage sequence of curve segments dynamically. On this basis, the whole path of UAVs is a successive combination of the coverage paths and the transferring paths, which are planned by the Dubins method with modified guidance vector field (MGVF) for obstacle avoidance and communication connectivity. Finally, the good performance of our method is verified on a real river map through simulation. Compared with the full sweeping method, our method can improve the efficiency by approximately 31.5%. The feasibility is also verified through a real experiment, where our method can improve the efficiency by approximately 16.3%.

Synthesis of stochastic algorithms for image registration by the criterion of maximum mutual information

We discuss a synthesis of stochastic algorithms, obtaining expressions for gradients of Shannon, Renyi and Tsallis mutual information on the basis of the mathematical apparatus of stochastic gradient adaptation of algorithms for estimating image registration parameters. To obtain the expressions, derivatives of the image entropy with respect to the estimated parameters are used. The entropies are calculated using a Parzen window method. A comparative study of the synthesized algorithms in terms of stability and accuracy of the registration parameter estimates, including in conditions of additive noise, is carried out.

Exploring Binary Classification Models for Parkinson’s Disease Detection

Parkinson’s disease (PD) is a degenerative condition characterised by a combination of motor symptoms, such as tremors and rigidity, and non-motor symptoms, including cognitive impairment and anxiety. Although there is currently no known cure, there are available treatments that can effectively mitigate symptoms and enhance life expectancy. Our research primarily centred around vocal disorders, given that an estimated 90% of individuals diagnosed with Parkinson’s disease experience the onset of such disorders in the early stages of the disease. In this study, a dataset is collected from UCI repository consisting of 252 cases was utilised, with 188 cases classified as Parkinson’s disease and 64 cases classified as non-Parkinson’s disease. The results of our study indicate that the K-nearest neighbours (KNN) algorithm demonstrated a peak accuracy of 98.52% and an average accuracy of 97.33%, surpassing the performance of previously established models. Additionally, we introduced the implementation of Parzen Windows Estimation (a KDE technique) and computed classifier accuracies based on mean test set scores after multiple rounds of data reshuffling and model training. Our results enhance credibility compared to previous works in this field. Given the high misdiagnosis rate (26%) and the expense of imaging tests, deploying a precise and cost-effective algorithm for early PD detection is critical.

A More Meticulous Method for Modeling Renewable Energy Storage Capacity

Parzen window estimation technique promises greater accuracy, and profit, for wind and solar energy production

Optimization of wind and solar energy storage system capacity configuration based on the Parzen window estimation method

Compressed air energy storage (CAES) effectively reduces wind and solar power curtailment due to randomness. However, inaccurate daily data and improper storage capacity configuration impact CAES development. This study uses the Parzen window estimation method to extract features from historical data, obtaining distributions of typical weekly wind power, solar power, and load. These distributions are compared to Weibull and Beta distributions. The wind–solar energy storage system's capacity configuration is optimized using a genetic algorithm to maximize profit. Different methods are compared in island/grid-connected modes using evaluation metrics to verify the accuracy of the Parzen window estimation method. The results show that it surpasses parameter estimation for real-time series-based configuration. Under grid-connected mode, rated power configurations are 1107 MW for wind, 346 MW for solar, and 290 MW for CAES. The CAES system has a rated capacity of 2320 MW·h, meeting average hourly power demand of 699.26 MW. It saves $6.55 million per week in electricity costs, with a maximum weekly profit of $0.61 million. Payback period for system investment is 5.6 years, excluding penalty costs.

Neural network for ordinal classification of imbalanced data by minimizing a Bayesian cost

Ordinal classification of imbalanced data is a challenging problem that appears in many real world applications. The challenge is to simultaneously consider the order of the classes and the class imbalance, which can notably improve the performance metrics. The Bayesian formulation allows to deal with these two characteristics jointly: It takes into account the prior probability of each class and the decision costs, which can be used to include the imbalance and the ordinal information, respectively. We propose to use the Bayesian formulation to train neural networks, which have shown excellent results in many classification tasks. A loss function is proposed to train networks with a single neuron in the output layer and a threshold based decision rule. The loss is an estimate of the Bayesian classification cost, based on the Parzen windows estimator, which is fitted for a thresholded decision. Experiments with several real datasets show that the proposed method provides competitive results in different scenarios, due to its high flexibility to specify the relative importance of the errors in the classification of patterns of different classes, considering the order and independently of the probability of each class.

Parzen Window Approximation on Riemannian Manifold

In graph motivated learning, label propagation largely depends on data affinity represented as edges between connected data points. The affinity assignment implicitly assumes even distribution of data on the manifold. This assumption may not hold and may lead to inaccurate metric assignment due to drift towards high-density regions. The drift affected heat kernel based affinity with a globally fixed Parzen window either discards genuine neighbors or forces distant data points to become a member of the neighborhood. This yields a biased affinity matrix. In this paper, the bias due to uneven data sampling on the Riemannian manifold is catered to by a variable Parzen window determined as a function of neighborhood size, ambient dimension, flatness range, etc. Additionally, affinity adjustment is used which offsets the effect of uneven sampling responsible for the bias. An affinity metric which takes into consideration the irregular sampling effect to yield accurate label propagation is proposed. Extensive experiments on synthetic and real-world data sets confirm that the proposed method increases the classification accuracy significantly and outperforms existing Parzen window estimators in graph Laplacian manifold regularization methods.

Online Monitoring of Automotive Engine Lubricating Oil Based on Internet of Things Technology

In order to realize the online monitoring of automotive engine lubricating oil, a method based on Internet of Things technology is proposed. This method uses the self-organizing neural network of the Internet of Things to fuse the original multidimensional feature data to obtain the fusion value. The Parzen window method is used to formulate the limit value of fusion value, and the samples are divided into three states: normal, warning, and abnormal. Weka software is used to extract rules from oil data. This method can identify different wear state information from oil spectral data, extract knowledge rules, and use them to build the knowledge base of automobile engine wear diagnosis system, so as to realize the automation and intelligence of automobile engine fault diagnosis based on lubricating oil spectral wear data. The measurement method of the lubricating oil sensor is mainly to comprehensively reflect the relationship between oil quality and electrical signal, so as to effectively provide users with reliable information. After many studies, it is concluded that the conductivity of lubricating oil has a good linear relationship with its acid value, metal particles, moisture content, and the change of additive content. Measuring the change of conductivity is an effective means to detect the change of lubricating oil quality. The experimental results show that using the extracted knowledge rules to verify the state of samples, the recognition rate is 97.47%. In order to more fully explain the difference between important element fusion and all feature fusion, all features are extracted. At the same time, the fault diagnosis and recognition rate of all features is not high, only 62.39%. It is proved that the Internet of Things technology can effectively realize online monitoring of the automotive engine lubricating oil.

Manifold Alignment Aware Ants: A Markovian Process for Manifold Extraction.

The presence of manifolds is a common assumption in many applications, including astronomy and computer vision. For instance, in astronomy, low-dimensional stellar structures, such as streams, shells, and globular clusters, can be found in the neighborhood of big galaxies such as the Milky Way. Since these structures are often buried in very large data sets, an algorithm, which can not only recover the manifold but also remove the background noise (or outliers), is highly desirable. While other works try to recover manifolds either by pushing all points toward manifolds or by downsampling from dense regions, aiming to solve one of the problems, they generally fail to suppress the noise on manifolds and remove background noise simultaneously. Inspired by the collective behavior of biological ants in food-seeking process, we propose a new algorithm that employs several random walkers equipped with a local alignment measure to detect and denoise manifolds. During the walking process, the agents release pheromone on data points, which reinforces future movements. Over time the pheromone concentrates on the manifolds, while it fades in the background noise due to an evaporation procedure. We use the Markov chain (MC) framework to provide a theoretical analysis of the convergence of the algorithm and its performance. Moreover, an empirical analysis, based on synthetic and real-world data sets, is provided to demonstrate its applicability in different areas, such as improving the performance of t-distributed stochastic neighbor embedding (t-SNE) and spectral clustering using the underlying MC formulas, recovering astronomical low-dimensional structures, and improving the performance of the fast Parzen window density estimator.

Adaptive manifold density estimation

When data concentrate on lower dimensional manifolds embedded in a higher dimensional representation space and these manifolds have different intrinsic geometry and dimensions, density estimators with locally adaptive smoothing parameters show substantial gains over those with fixed bandwidths. However, it is difficult to decide how the local bandwidth parameters should be parameterized. In this paper, we propose a sample-point density estimator. For each observation, we associate it with a unique bandwidth parameter which is achieved by computing the local covariance matrix of its k nearest neighbours. Compared to the fixed choice of the neighbourhood size for all the samples in [Vincent P, Bengio Y. Manifold parzen windows. In: Becker S, Thrun S, and Obermayer K, editors, Advances in neural information processing systems Vol. 15. MIT Press; 2003. p. 849–856], our adaptive manifold density estimator allows samples to select their own neighbourhood sizes.

Security risk assessment of wind integrated power system using Parzen window density estimation

Security risk assessment of wind integrated power system using Parzen window density estimation

A novel data cluster algorithm based on linear regression and residual analysis for human resource management

Human resource management has become an important part of enterprise management. How to select high-quality talent and how to allocate corresponding talent to appropriate work has become an increasingly acute problem. Traditional data cluster methods cannot effectively solve the above problem due to the high-dimensional data. Therefore, we propose a novel data cluster algorithm based on linear regression and residual analysis for human resource management. Improved hybrid entropy weight attribute similarity is adopted for measuring the similarity between objects. The proposed local density calculation method based on k-nearest neighbour (KNN) and Parzen window is used to calculate the density of each object. Then, we utilise the linear regression and residual analysis to select the clustering centre points quickly and automatically, which can eliminate the subjectivity of artificial selection. A new clustering centre objective optimisation model is proposed to determine the real clustering centre. Through theoretical analysis and comparative experiments on artificial data sets and real data sets, it shows that the proposed cluster algorithm can overcome the defects of the original algorithms, and achieve better clustering effect and lower computation time than state-of-the-art methods.

Margin-based approach for outlier detection of industrial design data using a modified general regression neural network

AbstractThe choice of components in industrial design involves setting design parameters that typically must reside inside permissible ranges called “design margins”. This paper proposes a novel automated method called the Margin-Based General Regression Neural Network (MB-GRNN) that classifies design errors for design parameters that are outside of permissible ranges as outliers, directly from industrial design data, using an unsupervised machine learning approach. The method is based on a modified GRNN that estimates extremal margin boundaries of design parameters by self-learning the features from datasets. These extremal permissible margin boundaries are determined by “stretching out” the upper and lower GRNN surfaces using an iterative application of stretch factors (a second kernel weighting factor). The method creates a variable insensitive band surrounding the data cloud, interlinked with the normal regression function, providing upper and lower margin boundaries. These boundaries can then be used to determine outliers and to predict a range of permissible values of design parameters during design. Pushing out extremal margin boundaries reduce the false identification of outliers. This classification technique could be used by industrial engineers to detect likely outliers and to predict a range of permissible output limits for chosen design parameters. The efficacy of this method has been validated against the widespread Parzen window method by comparing experimental results from three multivariate datasets. It was found that the two methods have different but complementary capabilities. The MB-GRNN also uses a modified algorithm for estimating the smoothing parameter using a combination of clustering, k-nearest neighbor, and localized covariance matrix.

A Novel Image Thresholding Method Combining Entropy With Parzen Window Estimation

Abstract Image thresholding is an important and efficient image segmentation technique, which is crucial and essential for image analysis and computer vision. In this paper, we proposed a new image thresholding method based on entropy and parzen window (PW) estimation. First, the probability of each gray-level distribution is approximate by using the PW estimation. Second, by combining the obtained probability information with entropic information of the foreground and background, a new objective function is created. At last, the ideal threshold value is obtained by optimizing the objective function. By comparing with some classical thresholding methods, such as inter class variance method (OTSU), minimum error thresholding method (MET), Kapur’s entropy based method (KAPUR) and the recent methods that take spatial information into consideration (2D-D histogram method, GLLV histogram method and Gabor histogram method), the proposed method, experiment on 10 images (one synthetic image, four nondestructive testing images and five real-world images), presents a better performance on the accuracy, robustness and visual effect of segmentation.

Investigation of the effectiveness of metric classification methods in identifying attacks in VANET

This paper discusses the problem of improving the efficiency of metric machine learning methods of identification attacks in vehicular adhoc networks (VANETs). The main idea of this research is to select the type of nonlinear functions for calculating the distances between the objects of the sample, describing the traffic of VANET using metric methods, such as the method of k-nearest neighbour with linearly decreasing weights and the Parzen window method. The analysis of the effectiveness of the methods considered was carried out on a synthetically generated sample with three different types of attacks on the network. Computational experiments have shown that the k-nearest neighbour method with decreasing weights based on an exponential function with base a < 1 is more efficient than the Parzen window method by about 0.3% and has an accuracy of 84.15%.

Robust Methods for Wideband Compressive Spectrum Sensing Under Non-Gaussian Noise

In a cognitive radio network, non-reconstruction-based wideband compressive spectrum sensing poses challenges under the non-Gaussian noise environment. The maximum correntropy criterion (MCC) is robust to impulsive noise whereas, the Parzen window Renyi entropy is a good choice for spectrum sensing in the presence of Gaussian noise at a low signal-to-noise ratio (SNR). However, the detection performance of these algorithms depends on the accuracy of measured noise variance, which is sensitive to impulse noise. In this letter, we improve the sensing performance of both the aforementioned algorithms in a non-Gaussian noise environment by modifying the kernel and threshold using robust statistics. The robust technique minimizes the influence of impulsive noise in the received signal. Finally, we carry out the simulation results to illustrate the superior performance of robust sensing algorithms under both Bernoulli’s distribution and symmetric <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\alpha $ </tex-math></inline-formula> stable ( <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\text{S}\alpha \text{S}$ </tex-math></inline-formula> ) distribution channel noise. The performance is compared with the non-robust counterpart for sensing multi-carrier Universal-Filtered Multi-Carrier (UFMC) signal.

Identification of Wiener systems based on the variable forgetting factor multierror stochastic gradient and the key term separation

SummaryThe identification of Wiener systems is very difficult because of the output nonlinearity and the parameter product term. To identify the Wiener system, a novel stochastic gradient algorithm based on the multierror and the key term separation is proposed. Firstly, the Wiener system is parameterized as a pseudo‐linear model to avoid the products of the parameters. Secondly, a parzen window is used to estimate the probability density function of the error. Thirdly, a stochastic information gradient algorithm with the multierror is adopted to estimate the parameters. The multierror takes the place of the scalar error by the stacked error, which accelerates the algorithm greatly. Fourthly, a variable forgetting factor considering errors is integrated to further speed the algorithm up. Finally, the proposed algorithm is validated by a numerical example and an industrial case. The estimation results indicate that the proposed algorithm can obtain accurate estimates with fast convergence speed.

An Improved Variable Kernel Density Estimator Based on L2 Regularization

The nature of the kernel density estimator (KDE) is to find the underlying probability density function (p.d.f) for a given dataset. The key to training the KDE is to determine the optimal bandwidth or Parzen window. All the data points share a fixed bandwidth (scalar for univariate KDE and vector for multivariate KDE) in the fixed KDE (FKDE). In this paper, we propose an improved variable KDE (IVKDE) which determines the optimal bandwidth for each data point in the given dataset based on the integrated squared error (ISE) criterion with the L2 regularization term. An effective optimization algorithm is developed to solve the improved objective function. We compare the estimation performance of IVKDE with FKDE and VKDE based on ISE criterion without L2 regularization on four univariate and four multivariate probability distributions. The experimental results show that IVKDE obtains lower estimation errors and thus demonstrate the effectiveness of IVKDE.