Multivariate Hypergeometric Distribution Research Articles

Enrichment on steps, not genes, improves inference of differentially expressed pathways.

Enrichment analysis is frequently used in combination with differential expression data to investigate potential commonalities amongst lists of genes and generate hypotheses for further experiments. However, current enrichment analysis approaches on pathways ignore the functional relationships between genes in a pathway, particularly OR logic that occurs when a set of proteins can each individually perform the same step in a pathway. As a result, these approaches miss pathways with large or multiple sets because of an inflation of pathway size (when measured as the total gene count) relative to the number of steps. We address this problem by enriching on step-enabling entities in pathways. We treat sets of protein-coding genes as single entities, and we also weight sets to account for the number of genes in them using the multivariate Fisher's noncentral hypergeometric distribution. We then show three examples of pathways that are recovered with this method and find that the results have significant proportions of pathways not found in gene list enrichment analysis.

Covering Hierarchical Dirichlet Mixture Models on binary data to enhance genomic stratifications in onco-hematology.

Onco-hematological studies are increasingly adopting statistical mixture models to support the advancement of the genomically-driven classification systems for blood cancer. Targeting enhanced patients stratification based on the sole role of molecular biology attracted much interest and contributes to bring personalized medicine closer to reality. In onco-hematology, Hierarchical Dirichlet Mixture Models (HDMM) have become one of the preferred method to cluster the genomics data, that include the presence or absence of gene mutations and cytogenetics anomalies, into components. This work unfolds the standard workflow used in onco-hematology to improve patient stratification and proposes alternative approaches to characterize the components and to assign patient to them, as they are crucial tasks usually supported by a priori clinical knowledge. We propose (a) to compute the parameters of the multinomial components of the HDMM or (b) to estimate the parameters of the HDMM components as if they were Multivariate Fisher's Non-Central Hypergeometric (MFNCH) distributions. Then, our approach to perform patients assignments to the HDMM components is designed to essentially determine for each patient its most likely component. We show on simulated data that the patients assignment using the MFNCH-based approach can be superior, if not comparable, to using the multinomial-based approach. Lastly, we illustrate on real Acute Myeloid Leukemia data how the utilization of MFNCH-based approach emerges as a good trade-off between the rigorous multinomial-based characterization of the HDMM components and the common refinement of them based on a priori clinical knowledge.

Entropy of the Canonical Occupancy (Macro) State in the Quantum Measurement Theory.

The paper analyzes the probability distribution of the occupancy numbers and the entropy of a system at the equilibrium composed by an arbitrary number of non-interacting bosons. The probability distribution is obtained through two approaches: one involves tracing out the environment from a bosonic eigenstate of the combined environment and system of interest (the empirical approach), while the other involves tracing out the environment from the mixed state of the combined environment and system of interest (the Bayesian approach). In the thermodynamic limit, the two coincide and are equal to the multinomial distribution. Furthermore, the paper proposes to identify the physical entropy of the bosonic system with the Shannon entropy of the occupancy numbers, fixing certain contradictions that arise in the classical analysis of thermodynamic entropy. Finally, by leveraging an information-theoretic inequality between the entropy of the multinomial distribution and the entropy of the multivariate hypergeometric distribution, Bayesianism of information theory and empiricism of statistical mechanics are integrated into a common "infomechanical" framework.

Deficiency bounds for the multivariate inverse hypergeometric distribution

Deficiency bounds for the multivariate inverse hypergeometric distribution

A multivariate statistical test for differential expression analysis

Statistical tests of differential expression usually suffer from two problems. Firstly, their statistical power is often limited when applied to small and skewed data sets. Secondly, gene expression data are usually discretized by applying arbitrary criteria to limit the number of false positives. In this work, a new statistical test obtained from a convolution of multivariate hypergeometric distributions, the Hy-test, is proposed to address these issues. Hy-test has been carried out on transcriptomic data from breast and kidney cancer tissues, and it has been compared with other differential expression analysis methods. Hy-test allows implicit discretization of the expression profiles and is more selective in retrieving both differential expressed genes and terms of Gene Ontology. Hy-test can be adopted together with other tests to retrieve information that would remain hidden otherwise, e.g., terms of (1) cell cycle deregulation for breast cancer and (2) “programmed cell death” for kidney cancer.

Hahn polynomials for hypergeometric distribution

Orthogonal polynomials for the multivariate hypergeometric distribution are defined on lattices in polyhedral domains in Rd. Their structures are studied through a detailed analysis of classical Hahn polynomials with negative integer parameters. Factorization of the Hahn polynomials is explored and used to explain the relationship between the index set of orthogonal polynomials and the lattice set in the polyhedral domain. In the multivariate case, these constructions lead to nontrivial families of hypergeometric polynomials vanishing on lattice polyhedra. The generating functions and bispectral properties of the orthogonal polynomials are also discussed.

Analysing the Protein-DNA Binding Sites in Arabidopsis thaliana from ChIP-seq Experiments

Computational genomics aim at supporting the discovery of how the functionality of the genome of the organism under study is affected both by its own sequence and structure, and by the network of interaction between this genome and different biological or physical factors. In this work, we focus on the analysis of ChIP-seq data, for which many methods have been proposed in the recent years. However, to the best of our knowledge, those methods lack an appropriate mathematical formalism. We have developed a method based on multivariate models for the analysis of the set of peaks obtained from a ChIP-seq experiment. This method can be used to characterize an individual experiment and to compare different experiments regardless of where and when they were conducted. The method is based on a multivariate hypergeometric distribution, which fits the complexity of the biological data and is better suited to deal with the uncertainty generated in this type of experiments than the dichotomous models used by the state of the art methods. We have validated this method with Arabidopsis thaliana datasets obtained from the Remap2020 database, obtaining results in accordance with the original study of these samples. Our work shows a novel way for analyzing ChIP-seq data.

Sampling a two dimensional matrix

A new sampling design for populations whose units can be arranged as an N×M matrix is proposed. The sample must satisfy some constraints: row and column sample sizes are set in advance. The proposed sampling method gives the same selection probability to all the sample matrices that satisfy the constraints. Three algorithms to select a sample uniformly in the feasible set are presented: an exact algorithm based on the multivariate hypergeometric distribution, an MCMC algorithm, and the cube method. Their performances are evaluated using Monte Carlo simulations. The designs for sampling elements in a given row or a given column are investigated and the single inclusion and joint selection probabilities under the proposed design are evaluated. Several variance estimators are proposed for the Horvitz–Thompson estimator of the population mean of the survey variable y and their performances are compared in a Monte Carlo study. A numerical example dealing with a creel survey of fishermen found at 9 sites over 36 days is presented.

A novel association test for rare variants based on algebraic statistics

With the rapid growth of next-generation sequencing technology, more and more rare variants are available in the human genome. In recent years, the point of study has already changed direction to rare variants in genome-wide association studies (GWAS). Although a variety of approaches have been proposed to test associations between rare variants and phenotypes of interest, it is far from the end of this problem, and it is worth exploring new statistical methods based on special features of rare variants. As we all know, the most direct way is to evaluate the association in a two-way contingency table if the phenotype is a discrete variable. The numbers of observations are very close or equal to 0s for most of cells in the contingency table due to the extremely low mutation rates of rare variants. In this paper, we propose a novel association test for rare variants based on a generalization of Fisher’s exact test, and the p-value of this exact test can be computed under the multivariate hypergeometric distribution in the framework of algebraic statistics. Simulation results show that our proposed method outperforms the existing methods, despite there is heterogeneity among causal variants. We also successfully apply our method into the genetic association study of coronary artery disease and hypertension from the Wellcome Trust Case Control Consortium.

MetaSPARSim: a 16S rRNA gene sequencing count data simulator

BackgroundIn the last few years, 16S rRNA gene sequencing (16S rDNA-seq) has seen a surprisingly rapid increase in election rate as a methodology to perform microbial community studies. Despite the considerable popularity of this technique, an exiguous number of specific tools are currently available for proper 16S rDNA-seq count data preprocessing and simulation. Indeed, the great majority of tools have been developed adapting methodologies previously used for bulk RNA-seq data, with poor assessment of their applicability in the metagenomics field. For such tools and the few ones specifically developed for 16S rDNA-seq data, performance assessment is challenging, mainly due to the complex nature of the data and the lack of realistic simulation models. In fact, to the best of our knowledge, no software thought for data simulation are available to directly obtain synthetic 16S rDNA-seq count tables that properly model heavy sparsity and compositionality typical of these data.ResultsIn this paper we present metaSPARSim, a sparse count matrix simulator intended for usage in development of 16S rDNA-seq metagenomic data processing pipelines. metaSPARSim implements a new generative process that models the sequencing process with a Multivariate Hypergeometric distribution in order to realistically simulate 16S rDNA-seq count table, resembling real experimental data compositionality and sparsity. It provides ready-to-use count matrices and comes with the possibility to reproduce different pre-coded scenarios and to estimate simulation parameters from real experimental data. The tool is made available at http://sysbiobig.dei.unipd.it/?q=Software#metaSPARSimand https://gitlab.com/sysbiobig/metasparsim.ConclusionmetaSPARSim is able to generate count matrices resembling real 16S rDNA-seq data. The availability of count data simulators is extremely valuable both for methods developers, for which a ground truth for tools validation is needed, and for users who want to assess state of the art analysis tools for choosing the most accurate one. Thus, we believe that metaSPARSim is a valuable tool for researchers involved in developing, testing and using robust and reliable data analysis methods in the context of 16S rRNA gene sequencing.

Accurate Estimate of the Advantage of Impossible Differential Attacks

Impossible differential attacks, which are taking advantage of differentials that cannot occur, are powerful attacks for block cipher primitives. The power of such attacks is often measured in terms of the advantage — number of key-bits found during the key sieving phase — which determines the time complexity of the exhaustive key search phase. The statistical model used to compute this advantage has been introduced in the seminal work about the resistance of the DEAL cipher to impossible differential attacks. This model, which has not been modified since the end of the 1990s, is implicitly based on the Poisson approximation of the binomial distribution. In this paper, we investigate this commonly used model and experimentally illustrate that random permutations do not follow it. Based on this observation, we propose more accurate estimates of the advantage of an impossible differential attack. The experiments illustrate the accuracy of the estimate derived from the multivariate hypergeometric distribution. The maximal advantage –using the full codebook– of an impossible differential attack is also derived.

On the convergence rate of grid search for polynomial optimization over the simplex

We consider the approximate minimization of a given polynomial on the s tandard simplex, obtained by taking the minimum value over all rational grid points with given denominator r∈N. It was shown in [De Klerk, E., Laurent, M., Sun, Z.: An error analysis for polynomial optimization over the simplex based on the multivariate hypergeometric distribution. SIAM J. Optim. 25(3) 1498–1514 (2015)] that the relative accuracy of this approximation depends on r as O(1/r2) if there exists a rational global minimizer. In this note we show that th e rational minimizer condition is not necessary to obtain the O(1/r2) bound.

The numerics of Premium Bonds

Premium Bonds sold by the UK National Savings and Investments (NS&I) agency are the possibly most popular example of lottery bonds. Premium Bonds holders renounce interest payments but instead participate in a lottery which distributes the equivalent of aggregate interest payments among them. While the random mechanism used in the lottery is well-documented the details of how to determine the probability distribution of a single bond holder's lottery prizes seem to be less well-known. We observe that the lottery prizes distribution is a multivariate hypergeometric distribution and discuss how to exactly calculate its probability masses as well as how to approximate the distribution by means of the Panjer recursion and Fourier transforms. We find that there are good reasons to prefer the approximations based on Panjer recursionor Fourier transforms to exact calculation of the lottery prize value distribution.

GAG-ID: Heparan Sulfate (HS) and Heparin Glycosaminoglycan High-Throughput Identification Software*[S

Heparin and heparan sulfate are very large linear polysaccharides that undergo a complex variety of modifications and are known to play important roles in human development, cell-cell communication and disease. Sequencing of highly sulfated glycosaminoglycan oligosaccharides like heparin and heparan sulfate by liquid chromatography-tandem mass spectrometry (LC-MS/MS) remains challenging because of the presence of multiple isomeric sequences in a complex mixture of oligosaccharides, the difficulties in separation of these isomers, and the facile loss of sulfates in MS/MS. We have previously introduced a method for structural sequencing of heparin/heparan sulfate oligosaccharides involving chemical derivatizations that replace labile sulfates with stable acetyl groups. This chemical derivatization scheme allows the use of reversed phase LC for high-resolution separation and MS/MS for sequencing of isomeric heparan sulfate oligosaccharides. However, because of the large number of analytes present in complex mixtures of heparin/HS oligosaccharides, the resulting LC-MS/MS data sets are large and cannot be annotated with existing glycomics software because of the specifically designed chemical derivatization strategy. We have developed a tool, called GAG-ID, to automate the interpretation of derivatized heparin/heparan sulfate LC-MS/MS data based on a modified multivariate hypergeometric distribution to weight the annotation of more intense peaks. The software is tested on a LC-MS/MS data set collected from a mixture of 21 synthesized heparan sulfate tetrasaccharides. By testing the discrimination of scoring with this system, we show that stratifying peaks into different intensity classes benefits the discrimination of scoring, and GAG-ID is able to properly assign all 21 synthetic tetrasaccharides in a defined mixture from a single LC-MS/MS run.

The Numerics of Premium Bonds

Premium Bonds sold by the UK National Savings and Investments (NS&I) agency are the possibly most popular example of lottery bonds. Premium Bonds holders renounce interest payments but instead participate in a lottery which distributes the equivalent of aggregate interest payments among them. While the random mechanism used in the lottery is well-documented the details of how to determine the probability distribution of a single bond holder's lottery prizes seem to be less well-known. We observe that the lottery prizes distribution is a multivariate hypergeometric distribution and discuss how to exactly calculate its probability masses as well as how to approximate the distribution by means of the Panjer recursion and Fourier transforms. We find that there are good reasons to prefer the approximations based on Panjer recursion or Fourier transforms to exact calculation of the lottery prize value distribution.

An Error Analysis for Polynomial Optimization over the Simplex Based on the Multivariate Hypergeometric Distribution

We study the minimization of fixed-degree polynomials over the simplex. This problem is well-known to be NP-hard, as it contains the maximum stable set problem in graph theory as a special case. In this paper, we consider a rational approximation by taking the minimum over the regular grid, which consists of rational points with denominator $r$ (for given $r$). We show that the associated convergence rate is $O(1/r^2)$ for quadratic polynomials. For general polynomials, if there exists a rational global minimizer over the simplex, we show that the convergence rate is also of the order $O(1/r^2)$. Our results answer a question posed by De Klerk, Laurent, and Sun [Math. Program., 151 (2015), pp. 433--457]. and improves on previously known $O(1/r)$ bounds in the quadratic case.

Problems with Double Blind in Medicine

Double blind tests of drugs and procedures depend on obtaining two equal and randomly assigned groups to be compared. With a diagnosis based only on symptoms, but with different etiologies, this is not very likely. We here show the probability of obtaining two equal groups with one diagnosis but three etiologies. The mathematical name for such a problem is multivariate hypergeometric distribution. We find that increasing the group size decreases the probability.

Population and genetic structure of two dioecious timber species Virola surinamensis and Virola koschnyi (Myristicaceae) in southwestern Costa Rica

Selective logging regulations generally fail to account for sex ratios, sex size distribution, spatial patterns and genetic structure in dioecious timber species. Furthermore, sympatric congeneric dioecious tropical species are harvested under the same vernacular name, failing to account for potential variation in species population traits. This practice is expected to have deleterious consequences in the population density and reproduction of the least abundant species. Here we document density, sex ratios, sex size distribution, spatial patterns and genetic structure in two dioecious timber tree species, Virola surinamensis and V. koschnyi in southwestern Costa Rica. In addition, we assessed the probability that harvesting these two species under the same vernacular name will cause a significant decline in either sex density of the least abundant species, which is expected to unbalance sex ratios, therefore, reducing the reproductive potential of the species. In a 62ha plot we tagged, geo-referenced and sampled for cambium tissue all adults of the two species (dbh>30cm) for genetic analyses. Microsatellites loci were used to describe genetic diversity parameters and spatial genetic structure. In a nuclear subplot (42ha) we measured dbh and monitored sex expression during two reproductive events to describe population density, sex ratios, sex size distribution and spatial patterns. Adult density was twofold higher for V. surinamensis than V. koschnyi. The proportion of flowering males and females and diametric size distribution did not differ within species. Adults of both Virola species were spatially aggregated, but sexes were distributed randomly. We found a significant but weak spatial genetic structure for V. surinamensis, but not for V. koschnyi. Finally, there is a high probability (Multivariate hypergeometric distribution, p=0.47) that harvesting these two species under the same vernacular name will cause a drastic decline in the density of male or female trees of V. koschnyi. Overall our results suggest that dioecy does not influence tree size or spatial distribution of these two timber species. The weak spatial genetic structure in V. surinamensis is likely due to clumped seed dispersal and absence of thinning during the recruitment of genetically related seeds to the adult stage. Harvesting these two species under the same vernacular name will have important consequences in the reproduction of V. koschnyi. We suggest that selective logging regulations for dioecious species should encourage appropriated species identification, ascertain the sex of reproductive individuals, harvest these species in proportion to their sex ratios and reduce the proportion of harvested individuals in the population.

Multivariate Hypergeometric Similarity Measure

We propose a similarity measure based on the multivariate hypergeometric distribution for the pairwise comparison of images and data vectors. The formulation and performance of the proposed measure are compared with other similarity measures using synthetic data. A method of piecewise approximation is also implemented to facilitate application of the proposed measure to large samples. Example applications of the proposed similarity measure are presented using mass spectrometry imaging data and gene expression microarray data. Results from synthetic and biological data indicate that the proposed measure is capable of providing meaningful discrimination between samples, and that it can be a useful tool for identifying potentially related samples in large-scale biological data sets.

Exact and asymptotically robust permutation tests

Given independent samples from P and Q, two-sample permutation tests allow one to construct exact level tests when the null hypothesis is P=Q. On the other hand, when comparing or testing particular parameters $\theta$ of P and Q, such as their means or medians, permutation tests need not be level $\alpha$, or even approximately level $\alpha$ in large samples. Under very weak assumptions for comparing estimators, we provide a general test procedure whereby the asymptotic validity of the permutation test holds while retaining the exact rejection probability $\alpha$ in finite samples when the underlying distributions are identical. The ideas are broadly applicable and special attention is given to the k-sample problem of comparing general parameters, whereby a permutation test is constructed which is exact level $\alpha$ under the hypothesis of identical distributions, but has asymptotic rejection probability $\alpha$ under the more general null hypothesis of equality of parameters. A Monte Carlo simulation study is performed as well. A quite general theory is possible based on a coupling construction, as well as a key contiguity argument for the multinomial and multivariate hypergeometric distributions.