Finite Probability Distributions Research Articles

A Rényi-type quasimetric with random interference detection

AbstractThis paper introduces a new dissimilarity measure between two discrete and finite probability distributions. The followed approach is grounded jointly on mixtures of probability distributions and an optimization procedure. We discuss the clear interpretation of the constitutive elements of the measure under an information-theoretical perspective by also highlighting its connections with the Rényi divergence of infinite order. Moreover, we show how the measure describes the inefficiency in assuming that a given probability distribution coincides with a benchmark one by giving formal writing of the random interference between the considered probability distributions. We explore the properties of the considered tool, which are in line with those defining the concept of quasimetric—i.e. a divergence for which the triangular inequality is satisfied. As a possible usage of the introduced device, an application to rare events is illustrated. This application shows that our measure may be suitable in cases where the accuracy of the small probabilities is a relevant matter.

Dissimilarity functions co-symmetry property: a focus on probability distributions with involutive negation

In today’s data-rich era, there is a growing need for developing effective similarity and dissimilarity measures to compare vast datasets. It is desirable that these measures reflect the intrinsic structure of the domain of these measures. Recently, it was shown that the space of finite probability distributions has a symmetric structure generated by involutive negation mapping probability distributions into their “opposite” probability distributions and back, such that the correlation between opposite distributions equals –1. An important property of similarity and dissimilarity functions reflecting such symmetry of probability distribution space is the co-symmetry of these functions when the similarity between probability distributions is equal to the similarity between their opposite distributions. This article delves into the analysis of five well-known dissimilarity functions, used for creating new co-symmetric dissimilarity functions. To conduct this study, a random dataset of one thousand probability distributions is employed. From these distributions, dissimilarity matrices are generated that are used to determine correlations similarity between different dissimilarity functions. The hierarchical clustering is applied to better understand the relationships between the studied dissimilarity functions. This methodology aims to identify and assess the dissimilarity functions that best match the characteristics of the studied probability distribution space, enhancing our understanding of data relationships and patterns. The study of these new measures offers a valuable perspective for analyzing and interpreting complex data, with the potential to make a significant impact in various fields and applications.

ANFIS learning using expectation maximization based Gaussian mixture model and multilayer perceptron learning

The Adaptive Neuro-Fuzzy Inference System (ANFIS) is a hybrid learning algorithm that combines the learning ability of neural networks with fuzzy inference systems. While ANFIS has been successfully applied to several real-world problems, effective parameter optimization remains a challenge. This paper presents a novel approach for optimizing parameters of ANFIS in supervised settings. The proposed approach utilizes a combination of probabilistic mixture models and perceptron-based learning to parameterize ANFIS. To learn ANFIS membership functions, it generates a mixture of the finite probability distributions for each input feature. Then the consequent parameters are learned by transforming them into weights of a multi-layer perceptron instance. The effectiveness of the proposed method is evaluated on classification problems of varying complexity, ranging from binary-class to multi-class with variable dimensions. The results show that the proposed algorithm improves ANFIS performance in terms of accuracy rate and speed (train-time) analysis. The effectiveness and efficiency of the proposed approach have been further confirmed using a 5 × 2 cross-validation paired significance t-test. In particular, for binary-class problems, our model outperformed standard methods by up to 10% accuracy improvement. Similarly, for multi-class problems, our model achieved an average increase of 8% in accuracy while reaching up to 14% improvement. On average, the proposed model showed a reduction of about 75% in training time compared to the other models. Overall, the proposed approach offers competitive computational performance and acceptable efficacy for parameter optimization of ANFIS.

Optimizing Erlangization-based approximations for finite discrete distributions and discrete phase-type distributions

In He et al.[8], continuous phase-type (PH) distributions are constructed to approximate finite discrete probability distributions and discrete PH-distributions. The approximations are based on Erlangization with a fixed number of phases. In this article, we first introduce continuous PH approximations with Erlang distributions of different orders. Then we develop an algorithm to find the continuous PH approximation with the minimum variance, among all such PH approximations with the same total number of phases. Thus, the proposed continuous PH approximations lead to a smaller gap between the variances of the Erlangization-based approximations and the original discrete random variables, which is achieved without adding more phases. The new approximations are useful to mitigate the burden in computation caused by the large number of phases needed in Erlangization approximation. Stochastic dominance is shown between the original (discrete) distributions and the approximations, which leads to bounds on the quantities of original distributions and/or stochastic models (e.g., reliability models). The approximation method is applied to analyze reliability models and a COVID-19 isolation program.

Generalized Functional Discriminating Measure For Finite Probability Distributions

"Analysts like Renyi (1961), Csiszar (1963, 1966), Bregman (1967), Burba-Rao (1982), Jain and Saraswat (2012), etc., have introduced and analyzed the different functional discriminating measures for comparing two discrete probability distributions. But in this article, a new functional discriminating measure has been proposed that will compare finite (more than two) discrete probability distributions simultaneously. Further, some intra-relations among the measures at different values of the parameters have been discussed. Also, an interesting connection with Csiszar’s generalized functional measure has been created. Some new inequalities compared to variational discrimination and Chi-square discrimination have been discussed as well. "

Negations of Probability Distributions: A Survey

In recent years many papers have been devoted to the analysis and applications of negations of finite probability distributions (PD), first considered by Ronald Yager. This paper gives a brief overview of some formal results on the definition and properties of negations of PD. Negations of PD are generated by negators of probability values transforming element-by-element PD into a negation of PD. Negators are non-increasing functions of probability values. There are two types of negators: PD-independent and PD-dependent negators. Yager's negator is fundamental in the characterization of linear PD-independent negators as a convex combination of Yager's negator and uniform negator. Involutivity of negations is important in logic, and such involutive negator is considered in the paper. We propose a new simple definition of the class of linear negators generalizing Yager's negator. Different examples illustrate properties of negations of PD. Finally, we consider some open problems in the analysis of negations of probability distributions.

Gradients and subgradients of buffered failure probability

Gradients and subgradients are central to optimization and sensitivity analysis of buffered failure probabilities. We furnish a characterization of subgradients based on subdifferential calculus in the case of finite probability distributions and, under additional assumptions, also a gradient expression for general distributions. Several examples illustrate the application of the results, especially in the context of optimality conditions.

Entropy modulo a prime

Building on work of Kontsevich, we introduce a definition of the entropy of a finite probability distribution in which the probabilities are integers modulo a prime p. The entropy, too, is an integer mod p. Entropy mod p is shown to be uniquely characterized by a functional equation identical to the one that characterizes ordinary Shannon entropy. We also establish a sense in which certain real entropies have residues mod p, connecting the concepts of entropy over R and over Z/pZ. Finally, entropy mod p is expressed as a polynomial which is shown to satisfy several identities, linking into work of Cathelineau, Elbaz-Vincent and Gangl on polylogarithms.

Prior selection method using likelihood confidence region and Dirichlet process Gaussian mixture model for Bayesian inference of building energy models

It is widely acknowledged that Bayesian inference is only beneficial when prior information is properly defined. However, there is no clear rule for prior selection, and it is apparently a matter of subjective selection by the domain expert(s). In other words, because the posterior inference results can vary depending on how the prior is set, a proper definition of the prior is important in terms of objectivity and accuracy for Bayesian inference of building energy models. Hence, the authors suggest a new prior selection method using Dirichlet process Gaussian mixture model (DPGMM) and the likelihood confidence region (hereafter referred to as likelihood CR). The DPGMM is a Bayesian nonparametric clustering technique that optimizes both the cluster shape and the number of clusters. Using the DPGMM, the finite probability distributions that make up the likelihood CR can be estimated, where the distribution with the highest maximum likelihood is applied as the informative prior. In this study, a reference office building of the United States Department of Energy was selected, and a surrogate model was generated using an artificial neural network. Based on a comparison between the authors’ suggestion and traditional informative (and/or non-informative) priors by domain experts, the proposed method requires only minimum information about the parameters (min and max) and performs better than the traditional approach

Asymptotic Majorization of Finite Probability Distributions

This paper studies majorization of high tensor powers of finitely supported probability distributions. Viewing probability distributions as a resource with majorization as a means of transformation corresponds to the resource theory of pure bipartite quantum states under LOCC transformations vis-\`a-vis Nielsen's Theorem. In [T. Fritz (2017)] a formula for the asymptotic exchange rate between any two finitely supported probability distributions was conjectured. The main result of the present paper is Theorem 3.11, which resolves this conjecture.

Identifying the asymmetry of finite support probability distributions on the basis of the first two moments

The asymmetry of a probability distribution of measurement result may be recognised, among other methods, on the basis of the versatile numerical characteristics of distribution. A skewness parameter, although suitable for this task, is often hardly convergent when the estimation is based on a sample. In this paper, the outer assessment of the uncertainty interval for skewness is determined on the basis of knowing the limit points of the distribution support together with its initial two moments. For some of their values the uncertainty interval of the skewness contains solely either significantly negative or significantly positive values. In all such cases the distribution must be asymmetrical. The proposed method for determining the asymmetry of probability distribution is suitable for cases where no sample data is available, and only prior knowledge of the initial two moments is given. Examples are provided of the method’s practical applications to asymmetry detection of the measurement result distributions.

Distributionally robust SDDP

We study a version of stochastic dual dynamic programming (SDDP) with a distributionally robust objective. The classical SDDP algorithm uses a finite (nominal) probability distribution for the random outcomes at each stage. We modify this by defining a distributional uncertainty set in each stage to be a Euclidean neighbourhood of the nominal probability distribution. We derive a formula for the worst-case expectation of future costs over this set that can be applied in the backward pass of SDDP. We verify the correctness of this algorithm, show its almost sure convergence under standard assumptions, and illustrate it by applying it to a model of the New Zealand hydrothermal electricity system.

Extensions on Stochastic Dominance Efficiency Tests

We consider second, third, fourth and fifth order stochastic dominance (SSD, TSD, FOSD and FISD, respectively) as well as decreasing absolute risk aversion (DARA) stochastic dominance (DSD). For comparison with DSD we also consider stochastic dominance (ESD) based on CARA utility functions. Their relevance in practice arises from empirical evidence on individual preferences fitting to preference models underlying such stochastic dominance relations. Assuming a known, discrete and finite probability distribution we derive necessary and sufficient efficiency tests under the six types of stochastic dominance. Simple arguments yield well-known SSD and TSD efficiency tests which are subsequently used to develop new FOSD, FISD, DSD and ESD efficiency tests. We provide numerical demonstration using stock market data of the US.

Efficient Sampling Methods for Discrete Distributions

We study the fundamental problem of the exact and efficient generation of random values from a finite and discrete probability distribution. Suppose that we are given n distinct events with associated probabilities p_1, dots , p_n. First, we consider the problem of sampling from the distribution where the i-th event has probability proportional to p_i. Second, we study the problem of sampling a subset which includes the i-th event independently with probability p_i. For both problems we present on two different classes of inputs—sorted and general probabilities—efficient data structures consisting of a preprocessing and a query algorithm. Varying the allotted preprocessing time yields a trade-off between preprocessing and query time, which we prove to be asymptotically optimal everywhere.

A Generalization of Majorization that Characterizes Shannon Entropy

We introduce a binary relation on the finite discrete probability distributions which generalizes notions of majorization that have been studied in quantum information theory. Motivated by questions in thermodynamics, our relation describes the transitions induced by bistochastic maps in the presence of additional auxiliary systems which may become correlated in the process. We show that this relation is completely characterized by Shannon entropy H, which yields an interpretation of H in resource-theoretic terms, and admits a particularly simple proof of a known characterization of H in terms of natural information-theoretic properties.

On the OWA Aggregation with Probabilistic Inputs

We discuss the use of the ordered weighted averaging (OWA) operator in multi-criteria decision problems as a means of aggregating the individual criteria satisfactions. We emphasize the need for ordering the arguments, the criteria satisfactions, when using the OWA operator. We consider the situation where the criteria satisfactions have some uncertainty, are finite probability distributions and note the requirement of having to order probability distributions. We introduce the idea of using pairwise stochastic dominance to provide the necessary ordering relationship over the probability distributions. We note that while this approach is appropriate, it is often not possible, since the presence of a stochastic dominance relationship between all pairs of probability distributions is not always the case, the relationship is not complete. To circumvent this problem we introduce an approach called the probabilistic exceedance method (PEM), which allows us to provide a surrogate for the OWA aggregation of probability distributions that doesn't require a linear ordering over the probability distributions. We look at this in both the cases in which the criteria have equal and unequal importances.

ALTERNATIVE ANALYSIS OF FINITE-TIME PROBABILITY DISTRIBUTIONS OF RENEWAL THEORY

We introduce a level-crossing analysis of the finite time-t probability distributions of the excess life, age, total life, and related quantities of renewal processes. The technique embeds the renewal process as one cycle of a regenerative process with a barrier at level t, whose limiting probability density function leads directly to the time-t quantities. The new method connects the analysis of renewal processes with the analysis of a large class of stochastic models of Operations Research. Examples are given.

Worst-case asymmetric distributed function computation

We consider a distributed function computation problem where an information sink communicates with N correlated information sources to compute a given deterministic function of source data. A data-vector is drawn from a discrete and finite probability distribution and component x i is revealed to ith, , source. We address this problem in asymmetric communication scenarios where only the sink knows the distribution P. We are interested in computing the minimum number of source bits required, in the worst-case, to solve this problem. We propose the notion of functional ambiguity to carry out the worst-case information-theoretic analysis of distributed function computation problems. We establish its various characteristics and prove that it leads to a valid information measure. Then, we provide a constructive solution for the distributed function computation problem in terms of an interactive communication protocol and prove its optimality. Finally, we establish two equivalence classes of compressible and incompressible functions to classify the set of all computable multivariate functions based on the minimum number of source bits needed in the worst-case to compute a function in the distributed function computation set-up.

Approximability of the two-stage stochastic knapsack problem with discretely distributed weights

In this paper the two-stage knapsack problem with random weights is studied under the aspect of approximability. We assume finite probability distributions for the weights and show that, unless P=NP, the so obtained problem cannot be approximated in polynomial time within a better ratio than K−1/2 (where K is the number of second-stage scenarios). We further study the special cases where in the second stage items can only be added or only be removed, but not both. Positive approximation results are given for three particular cases, namely linearly dependent first- and second-stage rewards, the polynomial scenario model and the case where the number of scenarios is assumed to be a constant. To the best of our knowledge, this is the first study of a two-stage knapsack problem under the aspect of approximability and the first time a non-approximability result has been proven for a stochastic knapsack problem of any kind.

Domestic Load Characterization Through Smart Meter Advance Stratification

The heterogeneity of domestic loads presents distribution network operators with operational uncertainties which may become problematic as generation capacity shrinks and network infrastructure ages. High resolution meter advances recorded by increasingly ubiquitous Smart Meters can be seen as representing base loads along with aggregations of multiple domestic appliances-in this letter, a Bayesian formulation of the finite mixture probability distribution is employed to enumerate and capture generalizations of these, from which compact representations of domestic load profiles can be formed.