Noncentral Hypergeometric Distribution Research Articles

Enrichment on steps, not genes, improves inference of differentially expressed pathways.

Enrichment analysis is frequently used in combination with differential expression data to investigate potential commonalities amongst lists of genes and generate hypotheses for further experiments. However, current enrichment analysis approaches on pathways ignore the functional relationships between genes in a pathway, particularly OR logic that occurs when a set of proteins can each individually perform the same step in a pathway. As a result, these approaches miss pathways with large or multiple sets because of an inflation of pathway size (when measured as the total gene count) relative to the number of steps. We address this problem by enriching on step-enabling entities in pathways. We treat sets of protein-coding genes as single entities, and we also weight sets to account for the number of genes in them using the multivariate Fisher's noncentral hypergeometric distribution. We then show three examples of pathways that are recovered with this method and find that the results have significant proportions of pathways not found in gene list enrichment analysis.

Network meta-analysis of rare events using penalized likelihood regression.

Network meta-analysis (NMA) of rare events has attracted little attention in the literature. Until recently, networks of interventions with rare events were analyzed using the inverse-variance NMA approach. However, when events are rare the normal approximations made by this model can be poor and effect estimates are potentially biased. Other methods for the synthesis of such data are the recent extension of the Mantel-Haenszel approach to NMA or the use of the noncentral hypergeometric distribution. In this article, we suggest a new common-effect NMA approach that can be applied even in networks of interventions with extremely low or even zero number of events without requiring study exclusion or arbitrary imputations. Our method is based on the implementation of the penalized likelihood function proposed by Firth for bias reduction of the maximum likelihood estimate to the logistic expression of the NMA model. A limitation of our method is that heterogeneity cannot be taken into account as an additive parameter as in most meta-analytical models. However, we account for heterogeneity by incorporating a multiplicative overdispersion term using a two-stage approach. We show through simulation that our method performs consistently well across all tested scenarios and most often results in smaller bias than other available methods. We also illustrate the use of our method through two clinical examples. We conclude that our "penalized likelihood NMA" approach is promising for the analysis of binary outcomes with rare events especially for networks with very few studies per comparison and very low control group risks.

Functional Enrichment Analysis of Regulatory Elements.

Statistical methods for enrichment analysis are important tools to extract biological information from omics experiments. Although these methods have been widely used for the analysis of gene and protein lists, the development of high-throughput technologies for regulatory elements demands dedicated statistical and bioinformatics tools. Here, we present a set of enrichment analysis methods for regulatory elements, including CpG sites, miRNAs, and transcription factors. Statistical significance is determined via a power weighting function for target genes and tested by the Wallenius noncentral hypergeometric distribution model to avoid selection bias. These new methodologies have been applied to the analysis of a set of miRNAs associated with arrhythmia, showing the potential of this tool to extract biological information from a list of regulatory elements. These new methods are available in GeneCodis 4, a web tool able to perform singular and modular enrichment analysis that allows the integration of heterogeneous information.

The conditional approach to evaluating detection performance

In many applied single-point Yes/No signal-detection studies, the main interest is to evaluate the observer’s sensitivity, based on the observed rates of hits and false alarms. For example, Kostopoulou, Nurek, Cantarella et al. (2019, Medical Decision Making, 39, 21–31) presented general practitioners (GPs) with clinical vignettes of patients showing various cancer-related symptoms, and asked them to decide if urgent referral was required; the standard discrimination index d′ was calculated for each GP. An alternative conditional approach to statistical inference emphasizes explicitly the conditional nature of the inferences drawn, and argues on the basis of the response marginal (the number of “yes” responses) that was actually observed. It is closely related to, for example, Fisher’s exact test or the Rasch model in item response theory which have long been valuable and prominent in psychology. The conditional framework applied to single-point Yes/No detection studies is based on the noncentral hypergeometric sampling distribution and permits, for samples of any size, exact inference because it eliminates nuisance (i.e., bias) parameters by conditioning. We describe in detail how the conditional approach leads to conditional maximum likelihood sample estimates of sensitivity, and to exact confidence intervals for the underlying (log) odds ratio. We relate the conditional approach to classical (logistic) detection models also leading to analyses of the odds ratio, compare its statistical power to that of the unconditional approach, and conclude by discussing some of its pros and cons.

Wallenius Bayes

This paper introduces a new event model appropriate for classifying (binary) data generated by a “destructive choice” process, such as certain human behavior. In such a process, making a choice removes that choice from future consideration yet does not influence the relative probability of other choices in the choice set. The proposed Wallenius event model is based on a somewhat forgotten non-central hypergeometric distribution introduced by Wallenius (Biased sampling: the non-central hypergeometric probability distribution. Ph.D. thesis, Stanford University, 1963). We discuss its relationship with models of how human choice behavior is generated, highlighting a key (simple) mathematical property. We use this background to describe specifically why traditional multivariate Bernoulli naive Bayes and multinomial naive Bayes each are suboptimal for such data. We then present an implementation of naive Bayes based on the Wallenius event model, and show experimentally that for data where we would expect the features to be generated via destructive choice behavior Wallenius Bayes indeed outperforms the traditional versions of naive Bayes for prediction based on these features. Furthermore, we also show that it is competitive with non-naive methods (in particular, support-vector machines). In contrast, we also show that Wallenius Bayes underperforms when the data generating process is not based on destructive choice.

An Approach to Enhanced Indexing

Investment managers often manage a portfolio of stocks with respect to a benchmark. Their primary concern is selecting the best stocks to outperform their benchmark, while constraining their tracking error. The most common way of doing this is to use an optimization framework to maximize the information ratio of their portfolio. In this paper, we develop an unconventional approach to determining how a portfolio should be managed with respect to the benchmark. In particular, given some assumptions, we determine what percentage of the benchmark stocks would be optimal for the portfolio manager to select. The optimal portfolio depends on Fisher’s noncentral hypergeometric distribution and the Wallenius noncentral hypergeometric distribution. We find that the optimal selectivity of a benchmark universe varies from 50% to 80%, when managers have selection ability. The information ratio and the selectivity percentage can vary as the selection ability of the portfolio manager changes and as the underlying size of the universe changes. These results are provocative, given that many enhanced index portfolio managers select a low a percentage of the benchmark universe.

A general non-central hypergeometric distribution

ABSTRACTWe construct a general non-central hypergeometric distribution, which models biased sampling without replacement. Our distribution is constructed from the combined order statistics of two samples: one of independent and identically distributed random variables with absolutely continuous distribution F and the other of independent and identically distributed random variables with absolutely continuous distribution G. The distribution depends on F and G only through F○G( − 1) (F composed with the quantile function of G), and the standard hypergeometric distribution and Wallenius’ non-central hypergeometric distribution arise as special cases. We show in efficient economic markets the quantity traded has a general non-central hypergeometric distribution.

Context-Based Scene Understanding

Context plays an important role in performance of object detection. There are two popular considerations in building context models for computer vision applications; type of context (semantic, spatial, scale) and scope of the relations (pairwise, high-order). In this paper, a new unified framework is presented that combines multiple sources of context in high-order relations to encode semantical coherence and consistency of the scenes. This framework introduces a new descriptor called context relevance score to model context-based distribution of the response variables and apply it to two distributions. First model incorporates context descriptor along with annotation response into a supervised Latent Dirichlet Allocation (LDA) built on multi-variate Bernoulli distribution called Context-Based LDA (CBLDA). The second model is based on multi-variate Wallenius' non-central Hyper-geometric distribution and is called Wallenius LDA (WLDA). WLDA incorporates context knowledge as bias parameter. Scene context is modeled as a graph and effectively used in object detection framework to maximize semantical consistency of the scene. The graph can also be used in recognition of out-of-context objects. Annotation metadata of Sun397 dataset is used to construct the context model. Performance of the proposed approaches was evaluated on ImageNet dataset. Comparison between proposed approaches and state-of-art multi-class object annotation algorithm shows superiority of presented approach in labeling of scene content.

English

It is a well known fact that race and ethnicity specificc variations exist in the treatment and survival of cancer patients. Studies based on breast cancer patients admitted to community hospitals in U.S depicted that there is significant difference in patterns of care between black and white breast cancer patients with blacks receiving lower quality and quantity of care. In this study, we look at this problem from a different perspective, treating the hospitals as small areas, and employing Bayesian techniques for parameter estimation. Two separate models are constructed to estimate the odds ratio of receiving liver scan (a pattern of care) for blacks and whites. The first model uses hospital-specific information while the second one uses pooled hospital data by borrowing strength from neighbouring hospitals. We have used the non-central hyper-geometric distribution as the basis for constructing the likelihood while estimation has been carried out using the griddy Metropolis-Hastings sampler. We apply our methodology on a National Cancer Institute (NCI) database. Although our results corroborate some of the observations from previous studies, it proposes a computationally attractive alternative to the established procedures in formulating and analyzing this problem.

A Permutation Procedure to Correct for Confounders in Case-Control Studies, Including Tests of Rare Variation

Many case-control tests of rare variation are implemented in statistical frameworks that make correction for confounders like population stratification difficult. Simple permutation of disease status is unacceptable for resolving this issue because the replicate data sets do not have the same confounding as the original data set. These limitations make it difficult to apply rare-variant tests to samples in which confounding most likely exists, e.g., samples collected from admixed populations. To enable the use of such rare-variant methods in structured samples, as well as to facilitate permutation tests for any situation in which case-control tests require adjustment for confounding covariates, we propose to establish the significance of a rare-variant test via a modified permutation procedure. Our procedure uses Fisher's noncentral hypergeometric distribution to generate permuted data sets with the same structure present in the actual data set such that inference is valid in the presence of confounding factors. We use simulated sequence data based on coalescent models to show that our permutation strategy corrects for confounding due to population stratification that, if ignored, would otherwise inflate the size of a rare-variant test. We further illustrate the approach by using sequence data from the Dallas Heart Study of energy metabolism traits. Researchers can implement our permutation approach by using the R package BiasedUrn.

Random effects meta-analysis of event outcome in the framework of the generalized linear mixed model with applications in sparse data

We consider random effects meta-analysis where the outcome variable is the occurrence of some event of interest. The data structures handled are where one has one or more groups in each study, and in each group either the number of subjects with and without the event, or the number of events and the total duration of follow-up is available. Traditionally, the meta-analysis follows the summary measures approach based on the estimates of the outcome measure(s) and the corresponding standard error(s). This approach assumes an approximate normal within-study likelihood and treats the standard errors as known. This approach has several potential disadvantages, such as not accounting for the standard errors being estimated, not accounting for correlation between the estimate and the standard error, the use of an (arbitrary) continuity correction in case of zero events, and the normal approximation being bad in studies with few events. We show that these problems can be overcome in most cases occurring in practice by replacing the approximate normal within-study likelihood by the appropriate exact likelihood. This leads to a generalized linear mixed model that can be fitted in standard statistical software. For instance, in the case of odds ratio meta-analysis, one can use the non-central hypergeometric distribution likelihood leading to mixed-effects conditional logistic regression. For incidence rate ratio meta-analysis, it leads to random effects logistic regression with an offset variable. We also present bivariate and multivariate extensions. We present a number of examples, especially with rare events, among which an example of network meta-analysis.

Estimation using non-central hypergeometric distributions in combining [formula omitted] tables

We consider estimation of the overall log odds ratio and the between-study variance or heterogeneity variance from a series of independent 2 × 2 tables. The estimates are obtained using the conditional distributions of the tables with fixed marginal totals, specifically the non-central hypergeometric distribution, along with an assumed normal distribution of the true study-specific log odds ratios. The method presented differs from the traditional random effects meta-analysis approach for 2 × 2 tables in two respects. First, it does not assume a normal distribution for the observed within-study log odds ratio statistics; and second, it does not require estimation of the heterogeneity variance as a first step for estimation of the overall log odds ratio. Furthermore, the method directly provides an estimate of the heterogeneity variance and an associated standard error. Several real examples are discussed to illustrate the approach outlined in this paper, and to compare it with other common approaches.

Calculation Methods for Wallenius' Noncentral Hypergeometric Distribution

Two different probability distributions are both known in the literature as “the” noncentral hypergeometric distribution. Wallenius' noncentral hypergeometric distribution can be described by an urn model without replacement with bias. Fisher's noncentral hypergeometric distribution is the conditional distribution of independent binomial variates given their sum. No reliable calculation method for Wallenius' noncentral hypergeometric distribution has hitherto been described in the literature. Several new methods for calculating probabilities from Wallenius' noncentral hypergeometric distribution are derived. Range of applicability, numerical problems, and efficiency are discussed for each method. Approximations to the mean and variance are also discussed. This distribution has important applications in models of biased sampling and in models of evolutionary systems.

Estimation of false discovery rates in multiple testing: application to gene microarray data.

Testing for significance with gene expression data from DNA microarray experiments involves simultaneous comparisons of hundreds or thousands of genes. If R denotes the number of rejections (declared significant genes) and V denotes the number of false rejections, then V/R, if R > 0, is the proportion of false rejected hypotheses. This paper proposes a model for the distribution of the number of rejections and the conditional distribution of V given R, V / R. Under the independence assumption, the distribution of R is a convolution of two binomials and the distribution of V / R has a noncentral hypergeometric distribution. Under an equicorrelated model, the distributions are more complex and are also derived. Five false discovery rate probability error measures are considered: FDR = E(V/R), pFDR = E(V/R / R > 0) (positive FDR), cFDR = E(V/R / R = r) (conditional FDR), mFDR = E(V)/E(R) (marginal FDR), and eFDR = E(V)/r (empirical FDR). The pFDR, cFDR, and mFDR are shown to be equivalent under the Bayesian framework, in which the number of true null hypotheses is modeled as a random variable. We present a parametric and a bootstrap procedure to estimate the FDRs. Monte Carlo simulations were conducted to evaluate the performance of these two methods. The bootstrap procedure appears to perform reasonably well, even when the alternative hypotheses are correlated (rho = .25). An example from a toxicogenomic microarray experiment is presented for illustration.

Fast and Stable Algorithms for Computing and Sampling From the Noncentral Hypergeometric Distribution

Although the noncentral hypergeometric distribution underlies conditional inference for 2 × 2 tables, major statistical packages lack support for this distribution. This article introduces fast and stable algorithms for computing the noncentral hypergeometric distribution and for sampling from it. The algorithms avoid the expensive and explosive combinatorial numbers by using a recursive relation. The algorithms also take advantage of the sharp concentration of the distribution around its mode to save computing time. A modified inverse method substantially reduces the number of searches in generating a random deviate. The algorithms are implemented in a Java class, Hypergeometric, available on the World Wide Web.

A note on the distribution of the number of vaccinated infected under non‐random mixing conditions

A common situation in vaccine efficacy (VE) estimation is dealing with non-randomly mixing populations, which may subject vaccinated and unvaccinated individuals to a different infection pressure. These conditions may lead to a bias in VE estimates. The derivation of the statistical distribution of the number of vaccinated and infected out of a sample of n infections in a VE trial is essential to develop estimates and their properties. For randomly mixing populations, it has been shown recently that this follows a hypergeometric distribution for ‘all/nothing’ vaccines, whereas it is a non-central hypergeometric distribution for ‘leaky’ ones. Here it is shown that these distributions still hold under non-random mixing conditions, provided that mixing preferences and contact rates are independent of vaccination status. These conditions are met when vaccine and placebo are randomized. Copyright © 2001 John Wiley & Sons, Ltd.

A note on the distribution of the number of vaccinated infected under non-random mixing conditions.

A common situation in vaccine efficacy (VE) estimation is dealing with non-randomly mixing populations, which may subject vaccinated and unvaccinated individuals to a different infection pressure. These conditions may lead to a bias in VE estimates. The derivation of the statistical distribution of the number of vaccinated and infected out of a sample of n infections in a VE trial is essential to develop estimates and their properties. For randomly mixing populations, it has been shown recently that this follows a hypergeometric distribution for 'all/nothing' vaccines, whereas it is a non-central hypergeometric distribution for 'leaky' ones. Here it is shown that these distributions still hold under non-random mixing conditions, provided that mixing preferences and contact rates are independent of vaccination status. These conditions are met when vaccine and placebo are randomized.

Exact tests of goodness of fit of log-linear models for rates.

We propose Metropolis-Hastings sampling methods for estimating the exact conditional p-value for tests of goodness of fit of log-linear models for mortality rates and standardized mortality ratios. We focus on two-way tables, where the required conditional distribution is a multivariate noncentral hypergeometric distribution with known noncentrality parameter. Two examples are presented: a 2 x 3 table, where the exact results, obtained by enumeration, are available for comparison, and a 9 x 7 table, where Monte Carlo methods provide the only feasible approach for exact inference.

A Hierarchical Bayesian Model for Combining Multiple 2 × 2 Tables Using Conditional Likelihoods

This paper introduces a hierarchical Bayesian model for combining multiple 2 x 2 tables that allows the flexibility of different odds ratio estimates for different tables and at the same time allows the tables to borrow information from each other. The proposed model, however, is different from a full Bayesian model in that the nuisance parameters are eliminated by conditioning instead of integration. The motivation is a more robust model and a faster and more stable Gibbs algorithm. We work out a Gibbs scheme using the adaptive rejection sampling for log concave density and an algorithm for the mean and variance of the noncentral hypergeometric distribution. The model is applied to a multicenter ulcer clinical trial.

Representations, bounds and approximations for tail probabilites of multivariate non-central hypergeometric and negative hypergeometric distributions

This paper presents representations, bounds and approximations for tail probabilities of the multivariate non-central hypergeometric distribution in terms of the corresponding generating functions and the convolution of truncated binomial random variables. The similar results are given for the multivariate non-central negative hypergeometric distribution. Numerical examples illustrate the great accuracy of the approximations.