Probabilistic Distance Measure Research Articles

Cluster Validity Index for Uncertain Data Based on a Probabilistic Distance Measure in Feature Space

Cluster validity indices (CVIs) for evaluating the result of the optimal number of clusters are critical measures in clustering problems. Most CVIs are designed for typical data-type objects called certain data objects. Certain data objects only have a singular value and include no uncertainty, so they are assumed to be information-abundant in the real world. In this study, new CVIs for uncertain data, based on kernel probabilistic distance measures to calculate the distance between two distributions in feature space, are proposed for uncertain clusters with arbitrary shapes, sub-clusters, and noise in objects. By transforming original uncertain data into kernel spaces, the proposed CVI accurately measures the compactness and separability of a cluster for arbitrary cluster shapes and is robust to noise and outliers in a cluster. The proposed CVI was evaluated for diverse types of simulated and real-life uncertain objects, confirming that the proposed validity indexes in feature space outperform the pre-existing ones in the original space.

A new fuzzy MLE-clustering approach based on object-to-group probabilistic distance measure: from anomaly detection to multi-fault classification in datacenter computational nodes

A new fuzzy MLE-clustering approach based on object-to-group probabilistic distance measure: from anomaly detection to multi-fault classification in datacenter computational nodes

Monitoring variations in multimode surface topography

The recent development of optical measuring instruments has increased the use of surface topographic data for monitoring the quality of the engineered surface. Due to the complexities of modern industrial processes, however, surfaces of final products under the normal manufacturing process may have multiple modes, such that the surface consists of different topographic features from one in-control mode to another. In this case, existing monitoring approaches based on the single mode surface cannot characterise normal surfaces with multiple modes, and result in poor detection performance. In this article, a new approach for monitoring variations in multimode surface topography is presented. We propose a multimode surface prediction model, which characterises the generic behaviour of normal surfaces with multiple in-control modes. Moreover, we present a mode-specific surface monitoring approach that identifies topographic variations on the surfaces based on the similarity between probability density function (PDF) of residuals from observed and normal surfaces obtained through the prediction model. A novel probabilistic distance measure is introduced to effectively measure the similarity between a single residual PDF and a set of residual PDFs under the same mode. The effectiveness of the proposed approach is demonstrated through numerical simulation and real-life application of paper surface monitoring.

Structural anomaly detection based on probabilistic distance measures of transmissibility function and statistical threshold selection scheme

As a mathematical representation of the output-to-output relationship, transmissibility function (TF) has been extensively applied in structural damage detection due to its robustness to influences of the input variations. As in most engineering fields, dealing with the problem of uncertainty in TF-based feature detection is an issue of fundamental importance. In this study, a new statistical, data-driven damage detection algorithm is proposed by rigorously modelling the variability of TF without postprocessing with circularly-symmetric complex Gaussian ratio distribution. The probabilistic distance of Symmetric Kullback-Leibler (SKL) divergence between TFs under baseline condition and potential damage scenarios which can measure the dissimilarity of probability distributions for the TFs under different states are computed as a damage index (DI) to detect structural anomaly. Compared against Mahalanobis distance which has the implicit assumption that the normal condition set is governed by Gaussian statistics, the probabilistic distance measure proposed in this study can deal with the deviations in TFs not following Gaussian distribution. A statistically rigorous threshold selection scheme integrating Bayesian inference strategy and Monte Carlo discordancy test is proposed to detect the the presence of damage by accommodating the uncertainties of measurements and the probabilistic model of TF. Numerical, experimental, and field test studies are conducted to validate the potential of probabilistic distance measure of TFs in anomaly detection under ambient vibration instead of forced vibration testing. Results demonstrate satisfactory performance of the proposed approach for detecting the existence and quantify the relative damage severity from a global perspective.

Application of probabilistic distance measures for inrush and internal fault currents discrimination in power transformer differential protection

Differential protection failure in discrimination of inrush and internal fault currents will impose significant challenges and damages both on the power transformers and power systems. To cope with this issue, this paper investigates the application of Probabilistic Distance Measure (PDM) in discrimination of the inrush and internal fault currents in power transformers. The PDMs are essentially employed to determine the dissimilarities between two sets of distributions. This paper utilizes Symmetric PDM, to determine the discrepancy between the differential current signal and the reconstructed differential current waveform. To such aim, at the first step, the reconstructed differential current waveform obtained from CTs is reproduced. The reconstructed current signal contains only sinusoidal parts of the fault current and thus are reproduced using the modified least squares algorithm. Eventually, the reconstructed current and the differential current signals are fed to PDM to determine the distribution discrepancy between the signals. The performance of the proposed PDM criterion is evaluated utilizing experimentally recorded data and is further compared with the state-of-the-art. The obtained results reveal that the proposed index can be successfully applied for the discrimination of inrush and fault currents even in the case of current transformer (CT) saturation.

A novel probabilistic distance measure for picture fuzzy sets with its application in classification problems

In the present communication, we propose the probabilistic distance measure for picture fuzzy sets where the probability of occurrence/non-occurrence of the picture fuzzy event have been incorporated. This framework has been clearly addressed through outline of a formulated problem and its probable solution structure along with its proof of validity. Further, the proposed probabilistic distance measure has been utilized to present an algorithm for solving some classification decision making problems in a more generalized way. Some important illustrative examples related to the problem of classification - building material classification, mineral classification and a decision making problem of financial investment risk have been worked out to exhibit the implementation of the proposed methodology. The obtained results have also been compared with the existing approaches of solving the classification problems. The uncertainty feature of the problem has been handled in a more broader sense reflecting the advantage of the introduced approach.

Granulized Z-VIKOR Model for Failure Mode and Effect Analysis

In this article, we have developed an improved failure mode and effect analysis (FMEA) model by leveraging the concepts of Z-number, rough number (RN), and probabilistic distance measure. Two new concepts, namely, double upper approximated rough number (DUARN) and granulized Z-number (gZN), a new scheme for measuring distance between two gZNs using weighted similarity and average linkage method, a new risk prioritization model, named, granulized Z-VIKOR, and a scheme for uncertainty assessment using box-plot are proposed. DUARN embodies the notion of double sided upper approximation of an ordinal decision class. gZN is developed using Z-number and DUARN. The distance between two gZNs is computed using the maximum entropy principle that captures the relationship between the A (opinion) and B (reliability of A) parts of gZN. The granulized Z-VIKOR involves synergistic integration of gZN and VIKOR. Both objective and subjective risk measures are computed and a combined risk measure is defined, which considers the interactions among the risk criteria using λ-Shapley index. Two case studies are conducted. Sensitivity and comparative analysis is carried out to demonstrate the applicability, effectiveness, and robustness of the proposed model, as well as its superiority to existing models.

Enhancer prediction in the human genome by probabilistic modelling of the chromatin feature patterns

BackgroundThe binding sites of transcription factors (TFs) and the localisation of histone modifications in the human genome can be quantified by the chromatin immunoprecipitation assay coupled with next-generation sequencing (ChIP-seq). The resulting chromatin feature data has been successfully adopted for genome-wide enhancer identification by several unsupervised and supervised machine learning methods. However, the current methods predict different numbers and different sets of enhancers for the same cell type and do not utilise the pattern of the ChIP-seq coverage profiles efficiently.ResultsIn this work, we propose a PRobabilistic Enhancer PRedictIoN Tool (PREPRINT) that assumes characteristic coverage patterns of chromatin features at enhancers and employs a statistical model to account for their variability. PREPRINT defines probabilistic distance measures to quantify the similarity of the genomic query regions and the characteristic coverage patterns. The probabilistic scores of the enhancer and non-enhancer samples are utilised to train a kernel-based classifier. The performance of the method is demonstrated on ENCODE data for two cell lines. The predicted enhancers are computationally validated based on the transcriptional regulatory protein binding sites and compared to the predictions obtained by state-of-the-art methods.ConclusionPREPRINT performs favorably to the state-of-the-art methods, especially when requiring the methods to predict a larger set of enhancers. PREPRINT generalises successfully to data from cell type not utilised for training, and often the PREPRINT performs better than the previous methods. The PREPRINT enhancers are less sensitive to the choice of prediction threshold. PREPRINT identifies biologically validated enhancers not predicted by the competing methods. The enhancers predicted by PREPRINT can aid the genome interpretation in functional genomics and clinical studies.

Novel Adaptive Clustering Algorithms Based on a Probabilistic Similarity Measure Over Atanassov Intuitionistic Fuzzy Set

This paper presents a novel probabilistic similarity measure (PSM) for Atanassov intuitionistic fuzzy sets. It then exploits PSM to propose an adaptive probabilistic similarity degree and develops the novel probabilistic $\lambda$ -cutting algorithm for clustering. Further, the probabilistic distance measure (obtained from the PSM) is used to develop a new clustering technique, which we have named “probabilistic intuitionistic fuzzy c-mean (PIFCM) algorithm”. Simulation experiments have been conducted over a variety of datasets including UCI machine learning datasets and real-world car dataset. The results obtained have been thoroughly compared with other well-known clustering techniques such as fuzzy c-mean (FCM), intuitionistic fuzzy c-mean, association coefficient method, and $\lambda$ -cutting method. Based upon the experimental results, it can be concluded that our probabilistic $\lambda$ -cutting algorithm and PIFCM algorithm outperform their existing counterparts.

Validity indices for clusters of uncertain data objects

Clustering validity indices are the main tools for evaluating the quality of formed clusters and determining the correct number of clusters. They can be applied on the results of clustering algorithms to validate the performance of those algorithms. In this paper, two clustering validity indices named uncertain Silhouette and Order Statistic, are developed for uncertain data. To the best of our knowledge, there is not any clustering validity index in the literature that is designed for uncertain objects and can be used for validating the performance of uncertain clustering algorithms. Our proposed validity indices use probabilistic distance measures to capture the distance between uncertain objects. They outperform existing validity indices for certain data in validating clusters of uncertain data objects and are robust to outliers. The Order Statistic index in particular, a general form of uncertain Dunn validity index (also developed here), is well capable of handling instances where there is a single cluster that is relatively scattered (not compact) compared to other clusters, or there are two clusters that are close (not well-separated) compared to other clusters. The aforementioned instances can potentially result in the failure of existing clustering validity indices in detecting the correct number of clusters.

Object-to-group probabilistic distance measure for uncertain data classification

Uncertain objects, where each feature is represented by multiple observations or a given or fitted probability density function, arise in applications such as sensor networks, moving object databases and medical and biological databases. We propose a methodology to classify uncertain objects based on a new probabilistic distance measure between an uncertain object and a group of uncertain objects. This object-to-group probabilistic distance measure is unique in that it accounts separately for the correlations among the features within each class and within each object. We compare the proposed object-to-group classifier to two existing classifiers, namely, the K-Nearest Neighbor classifier on object means (certain-KNN) and the uncertain-naïve Bayes classifier. In addition, we compare the object-to-group classifier to an uncertain K-Nearest Neighbor classifier (uncertain-KNN), also proposed here, that uses existing probabilistic distance measures for object-to-object distances. We illustrate the advantages of the proposed classifiers with both simulated and real data.

알파 다이버전스를 이용한 무게중심 모델 기반 음악 유사도

음악 유사도 계산은 음악 검색 및 분류 등의 정보 처리 시스템 구현에 있어서 가장 중요한 부분이다. 본 논문은 최근 제안된 무게중심 모델을 이용한 음악 검색 방법에 대해서 살펴보고, 무게중심 모델의 확률 분포 유사도를 이용하여 음악 검색을 수행하고 성능을 평가하였다. 확률 분포간의 거리는 주어진 두 개의 확률 분포가 특정 기준에서 얼마나 가까운 지를 계산하는 것으로 다이버전스라고 불리기도 한다. 본 논문에서는 무게중심 모델에서 확률 분포 간의 거리 비교 시에 알파 다이버전스를 활용하였다. 알파 다이버전스는 알파 값에 따라 다양한 형태를 가지며, 널리 사용되고 있는 KLD(Kullback-Leibler)와 BD(Bhattacharyya Distance)를 포함한다. 음악 장르와 가수 데이터셋에서 검색 실험을 수행했고, 확률 분포 거리 기반 유사도와 벡터 거리 기반 유사도의 음악 검색 성능을 비교하였다. 알파 다이버전스를 통해서 무게중심 모델 기반 음악 검색 성능을 개선시킬 수 있음을 보였다. Music-similarity computation is crucial in developing music information retrieval systems for browsing and classification. This paper overviews the recently-proposed centroid-model based music retrieval method and applies the distributional similarity measures to the model for retrieval-performance evaluation. Probabilistic distance measures (also called divergence) compute the distance between two probability distributions in a certain sense. In this paper, we consider the alpha divergence in computing distance between two centroid models for music retrieval. The alpha divergence includes the widely-used Kullback-Leibler divergence and Bhattacharyya distance depending on the values of alpha. Experiments were conducted on both genre and singer datasets. We compare the music-retrieval performance of the distributional similarity with that of the vector distances. The experimental results show that the alpha divergence improves the performance of the centroid-model based music retrieval.

Evaluation of Modified Categorical Data Fuzzy Clustering Algorithm on the Wisconsin Breast Cancer Dataset

The early diagnosis of breast cancer is an important step in a fight against the disease. Machine learning techniques have shown promise in improving our understanding of the disease. As medical datasets consist of data points which cannot be precisely assigned to a class, fuzzy methods have been useful for studying of these datasets. Sometimes breast cancer datasets are described by categorical features. Many fuzzy clustering algorithms have been developed for categorical datasets. However, in most of these methods Hamming distance is used to define the distance between the two categorical feature values. In this paper, we use a probabilistic distance measure for the distance computation among a pair of categorical feature values. Experiments demonstrate that the distance measure performs better than Hamming distance for Wisconsin breast cancer data.

Atanassov's intuitionistic fuzzy decision making with probabilistic information and distance measure

In this paper, we develop a new method for Atanassov's intuitionistic fuzzy decision making problems. We present the Atanassov's intuitionistic fuzzy probabilistic ordered weighted distance (IFPOWD) operator. Its main advantage is that it uses distance measures in a unified framework between the probability and the OWA operator that considers the degree of importance of each concept in the aggregation. Moreover, it is able to deal with uncertain environments represented in the form of Atanassov's intuitionistic fuzzy information. We analyze the applicability of this new approach in a group decision making problem concerning selection of investments.

Distribution-Based Dimensionality Reduction Applied to Articulated Motion Recognition

Some articulated motion representations rely on frame-wise abstractions of the statistical distribution of low-level features such as orientation, color, or relational distributions. As configuration among parts changes with articulated motion, the distribution changes, tracing a trajectory in the latent space of distributions, which we call the configuration space. These trajectories can then be used for recognition using standard techniques such as dynamic time warping. The core theory in this paper concerns embedding the frame-wise distributions, which can be looked upon as probability functions, into a low-dimensional space so that we can estimate various meaningful probabilistic distances such as the Chernoff, Bhattacharya, Matusita, Kullback-Leibler (KL) or symmetric-KL distances based on dot products between points in this space. Apart from computational advantages, this representation also affords speed-normalized matching of motion signatures. Speed normalized representations can be formed by interpolating the configuration trajectories along their arc lengths, without using any knowledge of the temporal scale variations between the sequences. We experiment with five different probabilistic distance measures and show the usefulness of the representation in three different contexts-sign recognition (with large number of possible classes), gesture recognition (with person variations), and classification of human-human interaction sequences (with segmentation problems). We find the importance of using the right distance measure for each situation. The low-dimensional embedding makes matching two to three times faster, while achieving recognition accuracies that are close to those obtained without using a low-dimensional embedding. We also empirically establish the robustness of the representation with respect to low-level parameters, embedding parameters, and temporal-scale parameters.

Probabilistic distance measures of the Dirichlet and Beta distributions

We give the analytical definitions of the Chernoff, Bhattacharyya and Jeffreys–Matusita probabilistic distances between two Dirichlet distributions and two Beta distributions as its special case. For all other known probabilistic distances we show their inappropriateness in the analytical case. We discuss the parameter learning of the Dirichlet distribution from a finite sample set and present an application for split-and-merge image segmentation.

From sample similarity to ensemble similarity: probabilistic distance measures in reproducing kernel Hilbert space

This paper addresses the problem of characterizing ensemble similarity from sample similarity in a principled manner. Using reproducing kernel as a characterization of sample similarity, we suggest a probabilistic distance measure in the reproducing kernel Hilbert space (RKHS) as the ensemble similarity. Assuming normality in the RKHS, we derive analytic expressions for probabilistic distance measures that are commonly used in many applications, such as Chernoff distance (or the Bhattacharyya distance as its special case), Kullback-Leibler divergence, etc. Since the reproducing kernel implicitly embeds a nonlinear mapping, our approach presents a new way to study these distances whose feasibility and efficiency is demonstrated using experiments with synthetic and real examples. Further, we extend the ensemble similarity to the reproducing kernel for ensemble and study the ensemble similarity for more general data representations.

Natural Image Statistics for Natural Image Segmentation

We integrate a model for filter response statistics of natural images into a variational framework for image segmentation. Incorporated in a sound probabilistic distance measure, the model drives level sets toward meaningful segmentations of complex textures and natural scenes. Despite its enhanced descriptive power, our approach preserves the efficiency of level set based segmentation since each region comprises two model parameters only. Analyzing thousands of natural images we select suitable filter banks, validate the statistical basis of our model, and demonstrate that it outperforms variational segmentation methods using second-order statistics.

Multiresolution estimates of classification complexity

In this paper, we study two measures of classification complexity based on feature space partitioning: purity and neighborhood separability. The new measures of complexity are compared with probabilistic distance measures and a number of other nonparametric estimates of classification complexity on a total of 10 databases from the University of California, Irvine, (UCI) repository.

Feature Selection for Financial Credit-Risk Evaluation Decisions

Credit-risk evaluation decisions are important for the financial institutions involved due to the high level of risk associated with wrong decisions. Various machine learning methods have been shown to perform reasonably well for this complex and unstructured problem. However, selecting a “good” set of features to be used in any learning system is a hard problem. We survey recent developments in feature selection and propose a new methodology based on the “Blurring” measure. The proposed feature selection method (FSB) is used to preprocess input data for induced decision trees. Three financial credit-risk evaluation data sets are used to illustrate the performance of the proposed method. In addition to FSB, results from randomly selected features, features selected using the Patrick-Fisher probabilistic distance measure, as well as no feature selection are provided for comparison purposes. Given the characteristics of the financial credit-risk evaluation domain, any improvement in classification performance is deemed beneficial. Preliminary results indicate that for comparable tree size, classification performance of FSB on classifying heretofore unseen examples is good compared to the other three methods evaluated in this study.