Discrete Probability Distributions Research Articles

Finding the modes of some multivariate discrete probability distributions: Application of the resource allocation problem

Determination of the modes of discrete multivariate distributions are of significant interest in many application contexts. Thus far, most papers in the literature have focused on developing specialized algorithms to find the modes of some specific distributions, e.g., the multinomial distribution. In contrast, this paper proposes a unified and efficient solution approach for the mode-finding problem for many well-known multivariate discrete distributions. In particular, we reformulate and solve the mode-finding problem as a discrete resource allocation problem using a greedy algorithm. Numerical examples show the effectiveness of our new solution approach.

Spectral Embedded Deep Clustering.

We propose a new clustering method based on a deep neural network. Given an unlabeled dataset and the number of clusters, our method directly groups the dataset into the given number of clusters in the original space. We use a conditional discrete probability distribution defined by a deep neural network as a statistical model. Our strategy is first to estimate the cluster labels of unlabeled data points selected from a high-density region, and then to conduct semi-supervised learning to train the model by using the estimated cluster labels and the remaining unlabeled data points. Lastly, by using the trained model, we obtain the estimated cluster labels of all given unlabeled data points. The advantage of our method is that it does not require key conditions. Existing clustering methods with deep neural networks assume that the cluster balance of a given dataset is uniform. Moreover, it also can be applied to various data domains as long as the data is expressed by a feature vector. In addition, it is observed that our method is robust against outliers. Therefore, the proposed method is expected to perform, on average, better than previous methods. We conducted numerical experiments on five commonly used datasets to confirm the effectiveness of the proposed method.

Classification of RCS sequences based on KL divergence

Radar cross section (RCS) is an important characteristic of radar targets. The mean, variance, skewness, kurtosis, varying patterns of RCS sequences provide rich features for radar target classification. In this study, an RCS classification method based on Kullback–Leibler divergence (KL divergence) is proposed. KL divergence is a measurement of the distance between two probabilistic distributions p (x) and q (x). In this method, the discrete probability distributions of the training RCS sequences and their varying patterns are firstly calculated and saved. For a target with unknown label, the corresponding probability distributions of its RCS sequence are computed. Then, the KL divergence between this distribution and the pre-trained distributions is also computed. The label of this target is determined by these KL divergences and an adaptive threshold. The method collects statistical features and dynamic features of RCS sequences, and provides a unified solution for classifying RCS sequences, which is of great importance for radar target recognition. The efficiency of this method is verified by experiments on simulated data.

Robust reformulations of ambiguous chance constraints with discrete probability distributions

This paper proposes robust reformulations of ambiguous chance constraints when the underlying family of distributions is discrete and supported in a so-called ``p-box'' or ``p-ellipsoidal'' uncertainty set. Using the robust optimization paradigm, the deterministic counterparts of the ambiguous chance constraints are reformulated as mixed-integer programming problems which can be tackled by commercial solvers for moderate sized instances. For larger sized instances, we propose a safe approximation algorithm that is computationally efficient and yields high quality solutions. The associated approach and the algorithm can be easily extended to joint chance constraints, nonlinear inequalities, and dependent data without introducing additional mathematical optimization complexity to that of the original robust reformulation. In numerical experiments, we first present our approach over a toy-sized chance constrained knapsack problem. Then, we compare optimality and computational performances of the safe approximation algorithm with those of the exact and the randomized approaches for larger sized instances via Monte Carlo simulation.

Exact and Approximate Weighted Model Integration with Probability Density Functions Using Knowledge Compilation

Weighted model counting has recently been extended to weighted model integration, which can be used to solve hybrid probabilistic reasoning problems. Such problems involve both discrete and continuous probability distributions. We show how standard knowledge compilation techniques (to SDDs and d-DNNFs) apply to weighted model integration, and use it in two novel solvers, one exact and one approximate solver. Furthermore, we extend the class of employable weight functions to actual probability density functions instead of mere polynomial weight functions.

Starlike and convex type probability distribution

The purpose of the present paper is to investigate Starlike and Convex type discrete probability distribution and obtain some results regarding moments, factorial moments, mean, variance and moment generating function for these distributions.

A Practical Overview on Comparison Discrete and Continuous Probability Distributions

To accelerate the growth of scientific learning through research gathered from all over the world. We want to be the catalysts for new discoveries in medicine, neurosurgery and other fields of science that will help cure diseases and find new ways of sustainable living.

Model-based testing of stochastically timed systems

Many systems are inherently stochastic: they interact with unpredictable environments or use randomised algorithms. Classical model-based testing is insufficient for such systems: it only covers functional correctness. In this paper, we present two model-based testing frameworks that additionally cover the stochastic aspects in hard and soft real-time systems. Using the theory of Markov automata and stochastic automata for specifications, test cases, and a formal notion of conformance, they provide clean mechanisms to represent underspecification, randomisation, and stochastic timing. Markov automata provide a simple memoryless model of time, while stochastic automata support arbitrary continuous and discrete probability distributions. We cleanly define the theoretical foundations, outline practical algorithms for statistical conformance checking, and evaluate both frameworks’ capabilities by testing timing aspects of the Bluetooth device discovery protocol. We highlight the trade-off of simple and efficient statistical evaluation for Markov automata versus precise and realistic modelling with stochastic automata.

A Maximum Entropy Procedure to Solve Likelihood Equations.

In this article, we provide initial findings regarding the problem of solving likelihood equations by means of a maximum entropy (ME) approach. Unlike standard procedures that require equating the score function of the maximum likelihood problem at zero, we propose an alternative strategy where the score is instead used as an external informative constraint to the maximization of the convex Shannon’s entropy function. The problem involves the reparameterization of the score parameters as expected values of discrete probability distributions where probabilities need to be estimated. This leads to a simpler situation where parameters are searched in smaller (hyper) simplex space. We assessed our proposal by means of empirical case studies and a simulation study, the latter involving the most critical case of logistic regression under data separation. The results suggested that the maximum entropy reformulation of the score problem solves the likelihood equation problem. Similarly, when maximum likelihood estimation is difficult, as is the case of logistic regression under separation, the maximum entropy proposal achieved results (numerically) comparable to those obtained by the Firth’s bias-corrected approach. Overall, these first findings reveal that a maximum entropy solution can be considered as an alternative technique to solve the likelihood equation.

Robust spatiotemporal fractionation schemes in the presence of patient setup uncertainty.

Spatiotemporal fractionation schemes for photon radiotherapy have recently arisen as a promising technique for healthy tissue sparing. Because spatiotemporally fractionated treatments have a characteristic pattern of delivering high doses to different parts of the tumor in each fraction, uncertainty in patient positioning is an even more pressing concern than in conventional uniform fractionation. Until now, such concerns in patient setup uncertainty have not been addressed in the context of spatiotemporal fractionation. A stochastic optimization model is used to incorporate patient setup uncertainty to optimize spatiotemporally fractionated plans using expected penalties for deviations from prescription values. First, a robust uniform reference plan is optimized with a stochastic optimization model. Then, a spatiotemporal plan is optimized with a constrained stochastic optimization model that minimizes a primary clinical objective and constrains the spatiotemporal plan to be at least as good as the uniform reference plan with respect to all other objectives. A discrete probability distribution is defined to characterize the random setup error occurring in each fraction. For the optimization of uniform plans, the expected penalties are computed exactly by exploiting the symmetry of the fractions, and for the spatiotemporal plans, quasi-Monte Carlo sampling is used to approximate the expectation. Using five clinical liver cases, it is demonstrated that spatiotemporally fractionated treatment plans maintain the same robust tumor coverage as a stochastic uniform reference plan and exhibit a reduction in the expected mean BED of the uninvolved liver. This is observed for a spectrum of probability distributions of random setup errors with shifts in the patient position of up to 5mm from the nominal position. For probability distributions with small variance in the patient position, the spatiotemporal plans exhibit an 8-30% reduction in expected mean BED in the healthy liver tissue for shifts up to 2.5mm and reductions of 5-25% for shifts up to 5mm. In the presence of patient setup uncertainty, spatiotemporally fractionated treatment plans exhibit the same robust tumor coverage as their uniformly fractionated counterparts and still retain the benefit in sparing healthy tissues.

Scheduling independent stochastic tasks under deadline and budget constraints

This article discusses scheduling strategies for the problem of maximizing the expected number of tasks that can be executed on a cloud platform within a given budget and under a deadline constraint. The execution times of tasks follow independent and identically distributed probability laws. The main questions are how many processors to enroll and whether and when to interrupt tasks that have been executing for some time. We provide complexity results and an asymptotically optimal strategy for the problem instance with discrete probability distributions and without deadline. We extend the latter strategy for the general case with continuous distributions and a deadline and we design an efficient heuristic which is shown to outperform standard approaches when running simulations for a variety of useful distribution laws.

Generalization of the discrete and continuous Shannon entropy by positive definite functions related to the constant of motion of the non-linear Lotka-Volterra system

In cybernetics, the Shannon formula is a positive function that measures the entropy of discrete probability distributions. The conversion of this formula to continuous probability distributions gives the infinity and negativity catastrophes, similar to the problem of renormalisation in physics. This paper suggests a solution to these two open problems, in starting from the derivation of positive definite functions from the Shannon entropy. These definite positive functions are related to the constant of motion of the non-linear Lotka-Volterra system. Then these positive definite functions are generalized for exchange of information with both discrete and continuous probability distributions. These functions generalized the Kullback-Leibler and Wiener information gain. A practical example is presented which shows a remarkable result with a scale invariance of the information gain. Then in this paper, we recall the properties of the differential equations of the non-linear Volterra predator-prey system and of the autocatalytic system of Lotka. As this paper deals with the conversion of discrete information to continuous information, we have developed the presentation of the discrete Lotka-Volterra equations. The numerical simulation of the hyperincursive discrete Lotka-Volterra equations shows an orbital stability. Moreover it is demonstrated that the hyperincursive discrete Lotka-Volterra equations are separable into two alternating discrete incursive Lotka-Volterra equations, similar to the hyperincursive discrete harmonic oscillator.

Probabilistic Species Tree Distances: Implementing the Multispecies Coalescent to Compare Species Trees Within the Same Model-Based Framework Used to Estimate Them.

Despite the ubiquitous use of statistical models for phylogenomic and population genomic inferences, this model-based rigor is rarely applied to post hoc comparison of trees. In a recent study, Garba et al. derived new methods for measuring the distance between two gene trees computed as the difference in their site pattern probability distributions. Unlike traditional metrics that compare trees solely in terms of geometry, these measures consider gene trees and associated parameters as probabilistic models that can be compared using standard information theoretic approaches. Consequently, probabilistic measures of phylogenetic tree distance can be far more informative than simply comparisons of topology and/or branch lengths alone. However, in their current form, these distance measures are not suitable for the comparison of species tree models in the presence of gene tree heterogeneity. Here, we demonstrate an approach for how the theory of Garba et al. (2018), which is based on gene tree distances, can be extended naturally to the comparison of species tree models. Multispecies coalescent (MSC) models parameterize the discrete probability distribution of gene trees conditioned upon a species tree with a particular topology and set of divergence times (in coalescent units), and thus provide a framework for measuring distances between species tree models in terms of their corresponding gene tree topology probabilities. We describe the computation of probabilistic species tree distances in the context of standard MSC models, which assume complete genetic isolation postspeciation, as well as recent theoretical extensions to the MSC in the form of network-based MSC models that relax this assumption and permit hybridization among taxa. We demonstrate these metrics using simulations and empirical species tree estimates and discuss both the benefits and limitations of these approaches. We make our species tree distance approach available as an R package called pSTDistanceR, for open use by the community.

Finding the Needle in a Haystack: Detrending Photometric Timeseries Data of Strictly Periodic Astrophysical Objects

Light curves of astrophysical objects frequently contain strictly periodic signals. In such cases, we can use that property to aid the detrending algorithm to fully disentangle an unknown periodic signal and an unknown baseline signal with no power at that period. The periodic signal is modeled as a discrete probability distribution function (pdf), while the baseline signal is modeled as a residual timeseries. Those two components are disentangled by minimizing the total variation (length) of the residual timeseries with regard to the per-bin pdf fluxes. We demonstrate the use of the algorithm on a synthetic case, on the eclipsing binary KIC 3953981 and on the eccentric ellipsoidal variable KIC 3547874. We further discuss the parameters and the limitations of the algorithm and speculate on the two most common use cases: detrending the periodic signal of interest and measuring the dependence of instrumental response on controlled instrumental variables. A more sophisticated version of the algorithm is released as open source on github and available via pip.

K-adaptability in stochastic combinatorial optimization under objective uncertainty

Abstract We address combinatorial optimization problems with uncertain objective functions, given by discrete probability distributions. Within this setting, we investigate the so-called K-adaptability approach: the aim is to calculate a set of K feasible solutions such that the objective value of the best of these solutions, calculated in each scenario independently, is optimal in expectation. Interpreted as a stochastic optimization problem, we only consider second-stage variables, however, the corresponding candidate solutions are selected in the first stage, i.e., before the scenario is known. We show that this problem is NP-hard even if the underlying certain problem is trivial, and present further complexity results concerning approximability and fixed-parameter tractability with respect to K. Moreover, we present exact solution methods as well as a heuristic for this problem and compare them in an extensive experimental evaluation, where the underlying problem is the Unconstrained Binary Optimization Problem, the Shortest Path Problem or the Spanning Tree Problem. It turns out that the performance and the ranking of these approaches strongly depends on the parameter K and on the number of scenarios.

Diagnosis of Brain Diseases via Multi-Scale Time-Series Model.

The functional magnetic resonance imaging (fMRI) data and brain network analysis have been widely applied to automated diagnosis of neural diseases or brain diseases. The fMRI time series data not only contains specific numerical information, but also involves rich dynamic temporal information, those previous graph theory approaches focus on local topology structure and lose contextual information and global fluctuation information. Here, we propose a novel multi-scale functional connectivity for identifying the brain disease via fMRI data. We calculate the discrete probability distribution of co-activity between different brain regions with various intervals. Also, we consider nonsynchronous information under different time dimensions, for analyzing the contextual information in the fMRI data. Therefore, our proposed method can be applied to more disease diagnosis and other fMRI data, particularly automated diagnosis of neural diseases or brain diseases. Finally, we adopt Support Vector Machine (SVM) on our proposed time-series features, which can be applied to do the brain disease classification and even deal with all time-series data. Experimental results verify the effectiveness of our proposed method compared with other outstanding approaches on Alzheimer's Disease Neuroimaging Initiative (ADNI) dataset and Major Depressive Disorder (MDD) dataset. Therefore, we provide an efficient system via a novel perspective to study brain networks.

Exploring the Accuracy of Joint-Distribution Approximations Given Partial Information

We test the accuracy of various methods for approximating underspecified joint probability distributions. In particular, we examine the maximum entropy and the analytic center approximations, and we introduce three methods for approximating a discrete joint probability distribution given partial probabilistic information. Our results suggest that recently proposed approximations and our new approximations more accurately represent the possible uncertainty models than do previous models such as maximum entropy.

Visibility testing and counting for uncertain segments

We study two well-known planar visibility problems, namely visibility testing and visibility counting, in a model where there is uncertainty about the input data. The standard versions of these problems are defined as follows: we are given a set S of n segments in R2, and we would like to preprocess S so that we can quickly answer queries of the form: is the given query segment s∈S visible from the given query point q∈R2 (for visibility testing) and how many segments in S are visible from the given query point q∈R2 (for visibility counting).In our model of uncertainty, each segment may or may not exist, and if it does, it is located in one of finitely many possible locations, given by a discrete probability distribution. In this setting, the probabilistic visibility testing problem (PVTP, for short) is to compute the probability that a given segment s∈S is visible from a given query point q and the probabilistic visibility counting problem (PVCP, for short) is to compute the expected number of segments in S that are visible from a query point q. We first show that PVTP is #P-complete. In the special case where uncertainty is only about whether segments exist and not about their location, we show that PVTP is solvable in O(nlog⁡n) time. Our algorithm for PVTP combined with linearity of expectation gives an O(n2log⁡n) time algorithm for PVCP.Using the algorithm for PVTP, together with a few old tricks, we can show that one can preprocess S in O(n5log⁡n) time into a data structure of size O(n4), so that each PVTP query for a fixed segment s can be answered in O(log⁡n) time.We also give a faster 2-approximation algorithm for this problem. At the end, we improve the approximation factor of the algorithm.

An Adaptive Estimation of Distribution Algorithm for Multipolicy Insurance Investment Planning

Insurance has been increasingly realized as an important way of investment and risk aversion. Fruitful insurance products are launched by insurers, but there is little research on how to make a proper insurance investment plan for a specific policyholder given different kinds of policies. In this paper, we aim to propose a practical approach to multipolicy insurance investment planning with a data-driven model and an estimation of distribution algorithm (EDA). First, by making use of the insurance data accumulated in the modern financial market, an optimization model about how to choose endowment and hospitalization policies is built to maximize the yearly profit of insurance investment. With the model parameters set according to the real data from insurance market, the resulting plan is practical and individualized. Second, as the optimal solution cannot be achieved by mathematical deduction under this data-driven model, an EDA is introduced. To adapt the EDA for the considered problem, the proposed EDA is mixed with both the continuous and discrete probability distribution models to handle different kinds of variables. In addition, an adaptive scheme for choosing suitable distribution models and an efficient constraint handling strategy are proposed. Experiments under different conditions confirm the effectiveness and efficiency of the proposed model and method.

Remote Sampling with Applications to General Entanglement Simulation.

We show how to sample exactly discrete probability distributions whose defining parameters are distributed among remote parties. For this purpose, von Neumann’s rejection algorithm is turned into a distributed sampling communication protocol. We study the expected number of bits communicated among the parties and also exhibit a trade-off between the number of rounds of the rejection algorithm and the number of bits transmitted in the initial phase. Finally, we apply remote sampling to the simulation of quantum entanglement in its essentially most general form possible, when an arbitrary finite number m of parties share systems of arbitrary finite dimensions on which they apply arbitrary measurements (not restricted to being projective measurements, but restricted to finitely many possible outcomes). In case the dimension of the systems and the number of possible outcomes per party are bounded by a constant, it suffices to communicate an expected bits in order to simulate exactly the outcomes that these measurements would have produced on those systems.