Class Of Probabilistic Models Research Articles

Improving probabilistic information retrieval by modeling burstiness of words

The classical probabilistic models attempt to capture the ad hoc information retrieval problem within a rigorous probabilistic framework. It has long been recognized that the primary obstacle to the effective performance of the probabilistic models is the need to estimate a relevance model. The Dirichlet compound multinomial (DCM) distribution based on the Polya Urn scheme, which can also be considered as a hierarchical Bayesian model, is a more appropriate generative model than the traditional multinomial distribution for text documents. We explore a new probabilistic model based on the DCM distribution, which enables efficient retrieval and accurate ranking. Because the DCM distribution captures the dependency of repetitive word occurrences, the new probabilistic model based on this distribution is able to model the concavity of the score function more effectively. To avoid the empirical tuning of retrieval parameters, we design several parameter estimation algorithms to automatically set model parameters. Additionally, we propose a pseudo-relevance feedback algorithm based on the mixture modeling of the Dirichlet compound multinomial distribution to further improve retrieval accuracy. Finally, our experiments show that both the baseline probabilistic retrieval algorithm based on the DCM distribution and the corresponding pseudo-relevance feedback algorithm outperform the existing language modeling systems on several TREC retrieval tasks. The main objective of this research is to develop an effective probabilistic model based on the DCM distribution. A secondary objective is to provide a thorough understanding of the probabilistic retrieval model by a theoretical understanding of various text distribution assumptions.

A Bayesian network view on nested effects models.

Nested effects models (NEMs) are a class of probabilistic models that were designed to reconstruct a hidden signalling structure from a large set of observable effects caused by active interventions into the signalling pathway. We give a more flexible formulation of NEMs in the language of Bayesian networks. Our framework constitutes a natural generalization of the original NEM model, since it explicitly states the assumptions that are tacitly underlying the original version. Our approach gives rise to new learning methods for NEMs, which have been implemented in the R/Bioconductor package nem. We validate these methods in a simulation study and apply them to a synthetic lethality dataset in yeast.

A classification of hidden-variable properties

Hidden variables are extra components added to try to banish counterintuitive features of quantum mechanics. We start with a quantum-mechanical model and describe various properties that can be asked of a hidden-variable model. We present six such properties and a Venn diagram of how they are related. With two existence theorems and three no-go theorems (EPR, Bell and Kochen–Specker), we show which properties of empirically equivalent hidden-variable models are possible and which are not. Formally, our treatment relies only on classical probability models, and physical phenomena are used only to motivate which models to choose.

Entanglement in the Quantum Ising Model

We study the asymptotic scaling of the entanglement of a block of spins for the ground state of the one-dimensional quantum Ising model with transverse field. When the field is sufficiently strong, the entanglement grows at most logarithmically in the number of spins. The proof utilises a transformation to a model of classical probability called the continuum random-cluster model, and is based on a property of the latter model termed ratio weak-mixing. In an intermediate result, we establish an exponentially decaying bound on the operator norm of differences of the reduced density operator. Of special interest is the mathematical rigour of this work, and the fact that the proof applies equally to a large class of disordered interactions.

Is Weibull distribution the most appropriate statistical strength distribution for brittle materials?

Strength reliability, one of the critical factors restricting wider use of brittle materials in various structural applications, is commonly characterized by Weibull strength distribution function. In the present work, the detailed statistical analysis of the strength data is carried out using a larger class of probability models including Weibull, normal, log-normal, gamma and generalized exponential distributions. Our analysis is validated using the strength data, measured with a number of structural ceramic materials and a glass material. An important implication of the present study is that the gamma or log-normal distribution function, in contrast to Weibull distribution, may describe more appropriately, in certain cases, the experimentally measured strength data.

Information retrieval and applications of graphical models (IRGM 2007)

Probabilistic models constitute an important kind of Information Retrieval (IR) model. They have been long and widely used [2], and offer a principled way of managing the uncertainty that naturally appears in many elements within this field. Nowadays, the dominant approach for managing probability within the field of Artificial Intelligence is based on the use of Bayesian Networks [15, 9], and these have also been used within IR as extensions of classical probabilistic models.

Universal residuals: A multivariate transformation

Rosenblatt's transformation has been used extensively for the evaluation of model goodness-of-fit, but it only applies to models whose joint distribution is continuous. In this paper we generalize the transformation so that it applies to arbitrary probability models. The transformation is simple, but has a wide range of possible applications, providing a tool for exploratory data analysis and formal goodness-of-fit testing for a very general class of probability models. The method is demonstrated with specific examples.

Avaliação bayesiana da eficácia da manutenção via processo de renovação generalizado

O Processo de Renovação Generalizado (PRG) é uma classe de modelos probabilísticos que trata ações de reparo de acordo com a redução que proporcionam na idade real de um equipamento/sistema. O PRG é uma extensão do Processo de Renovação e do Processo Não Homogêneo de Poisson e será utilizado para avaliar ações de reparo quanto a seu grau de eficácia. Considerando que os tempos entre falhas seguem uma distribuição Weibull, tal avaliação será realizada através da estimação da distribuição dos parâmetros do PRG e da análise de incerteza sobre o número esperado de falhas a partir de simulação Monte Carlo. Devido à escassez de dados de falha, o procedimento de inferência probabilística será executado a partir do paradigma Bayesiano o qual permite que outras fontes de informação, além dos dados de falha, sejam utilizadas na avaliação da distribuição de probabilidade sobre algum parâmetro de interesse.

RELIABILITY OF ELECTRIC POWER GENERATION IN POWER SYSTEMS WITH THERMAL AND WIND POWER PLANTS

The principles of evaluation of the reliability of electric power generation in a power system including thermal and wind power plants are considered in this paper. Besides classical probabilistic models the use of uncertain probabilistic and fuzzy probabilistic models of reliability is recommended. Generation of electric power at wind power plants is treated as a nonstationary stochastic process controllable only to down. The paper presents numerical examples.

A Formula of Total Probability with the Interference Term and the Hilbert Space Representation of the Contextual Kolmogorovian Model

We compare the classical Kolmogorov and quantum probability models. We show that the gap between these models is not so huge as was commonly believed. The main structures of quantum theory (interference of probabilities, Born's rule, complex probabilistic amplitudes, Hilbert state space, representation of observables by operators) are present in a latent form in the Kolmogorov model. In particular, we obtain “interference of probabilities” without appealing to the Hilbert space formalism. We interpret “interference of probabilities” as a perturbation (by a cos-term) of the conventional formula of total probability. Our classical derivation of quantum probabilistic formalism can stimulate applications of quantum methods outside of the microworld, for instance, in psychology, biology, economy, and other domains of science.

Information Retrieval for short documents

The major problem of the most current approaches of information models lies in that individual words provide unreliable evidence about the content of the texts. When the document is short, e.g. only the abstract is available, the word-use variability problem will have substantial impact on the Information Retrieval (IR) performance. To solve the problem, a new technology to short document retrieval named Reference Document Model (RDM) is put forward in this letter. RDM gets the statistical semantic of the query/document by pseudo feedback both for the query and document from reference documents. The contributions of this model are three-fold: (1) Pseudo feedback both for the query and the document; (2) Building the query model and the document model from reference documents; (3) Flexible indexing units, which can be any linguistic elements such as documents, paragraphs, sentences, n-grams, term or character. For short document retrieval, RDM achieves significant improvements over the classical probabilistic models on the task of ad hoc retrieval on Text REtrieval Conference (TREC) test sets. Results also show that the shorter the document, the better the RDM performance.

On the infinite-horizon probability of (non)ruin for integer-valued claims

We consider a compound Poisson process whose jumps are modelled as a sequence of positive, integer-valued, dependent random variables, W1,W2,…, viewed as insurance claim amounts. The number of points up to time t of the stationary Poisson process which models the claim arrivals is assumed to be independent of W1,W2,…. The premium income to the insurance company is represented by a nondecreasing, nonnegative, real-valued function h(t) on [0,∞) such that limt→∞h(t) = ∞. The function h(t) is interpreted as an upper boundary. The probability that the trajectory of such a compound Poisson process will not cross the upper boundary in infinite time is known as the infinite-horizon nonruin probability. Our main result in this paper is an explicit expression for the probability of infinite-horizon nonruin, assuming that certain conditions on the premium-income function, h(t), and the joint distribution of the claim amount random variables, W1,W2,…, hold. We have also considered the classical ruin probability model, in which W1,W2,… are assumed to be independent, identically distributed random variables and we let h(t)=u + ct. For this model we give a formula for the nonruin probability which is a special case of our main result. This formula is shown to coincide with the infinite-horizon nonruin probability formulae of Picard and Lefèvre (2001), Gerber (1988), (1989), and Shiu (1987), (1989).

On the infinite-horizon probability of (non)ruin for integer-valued claims

We consider a compound Poisson process whose jumps are modelled as a sequence of positive, integer-valued, dependent random variables, W 1,W 2,…, viewed as insurance claim amounts. The number of points up to time t of the stationary Poisson process which models the claim arrivals is assumed to be independent of W 1,W 2,…. The premium income to the insurance company is represented by a nondecreasing, nonnegative, real-valued function h(t) on [0,∞) such that lim t→∞ h(t) = ∞. The function h(t) is interpreted as an upper boundary. The probability that the trajectory of such a compound Poisson process will not cross the upper boundary in infinite time is known as the infinite-horizon nonruin probability. Our main result in this paper is an explicit expression for the probability of infinite-horizon nonruin, assuming that certain conditions on the premium-income function, h(t), and the joint distribution of the claim amount random variables, W 1,W 2,…, hold. We have also considered the classical ruin probability model, in which W 1,W 2,… are assumed to be independent, identically distributed random variables and we let h(t)=u + ct. For this model we give a formula for the nonruin probability which is a special case of our main result. This formula is shown to coincide with the infinite-horizon nonruin probability formulae of Picard and Lefèvre (2001), Gerber (1988), (1989), and Shiu (1987), (1989).

Interference in the classical probabilistic model and its representation in complex Hilbert space

The notion of a context (complex of physical conditions, that is to say: specification of the measurement setup) is basic in this paper.We show that the main structures of quantum theory (interference of probabilities, Born's rule, complex probabilistic amplitudes, Hilbert state space, representation of observables by operators) are present already in a latent form in the classical Kolmogorov probability model. However, this model should be considered as a calculus of contextual probabilities. In our approach it is forbidden to consider abstract context independent probabilities: “first context and only then probability”. We construct the representation of the general contextual probabilistic dynamics in the complex Hilbert space. Thus dynamics of the wave function (in particular, Schrödinger's dynamics) can be considered as Hilbert space projections of a realistic dynamics in a “prespace”. The basic condition for representing of the prespace-dynamics is the law of statistical conservation of energy—conservation of probabilities. In general the Hilbert space projection of the “prespace” dynamics can be nonlinear and even irreversible (but it is always unitary). Methods developed in this paper can be applied not only to quantum mechanics, but also to classical statistical mechanics. The main quantum-like structures (e.g., interference of probabilities) might be found in some models of classical statistical mechanics. Quantum-like probabilistic behavior can be demonstrated by biological systems. In particular, it was recently found in some psychological experiments.

Interference of probabilities in the classical probabilistic framework

The notion of context (complex of physical conditions) is basic in this paper. We show that the main structures of quantum theory (interference of probabilities, Born's rule, complex probabilistic amplitudes, Hilbert state space, representation of observables by operators) are present in a latent form in the classical Kolmogorov probability model. However, this model should be considered as a calculus of contextual probabilities. In our approach, it is forbidden to consider abstract context-independent probabilities: “first context and then probability.” We start with the conventional formula of total probability for contextual (conditional) probabilities and then we rewrite it by eliminating combinations of incompatible contexts from consideration. In this way we obtain interference of probabilities without to appeal to the Hilbert space formalism or wave mechanics. Our contextual approach is important for demystification of quantum probabilistic formalism. This approach gives the possibility to apply quantum-like models in domains of science different from quantum theories, e.g., in economics, finance, social sciences, cognitive sciences, psychology.

Solution theorems for the standard eigenvalue problem of structures with uncertain-but-bounded parameters

Generalized eigenvalue problems from the modal analysis are often converted to the standard eigenvalue problems. In this paper, it evaluates the upper and lower bounds on the eigenvalues of the standard eigenvalue problem of structures subject to severely deficient information about the structural parameters. Here, we focus on non-probabilistic interval analysis models of uncertainty, which are adapted to the case of severe lack of information on uncertainty. Non-probabilistic, interval analysis method in which uncertainties are defined by interval numbers appears as an alternative to the classical probabilistic models. For the standard eigenvalue problem of structures with uncertain-but-bounded parameters, the vertex solution theorem, the positive semi-definite solution theorem and the parameter decomposition solution theorem for the standard eigenvalue problem are presented, and compared with Deif's solution theorem in numerical examples. It is shown that, for the upper and lower bounds on the eigenvalues of the standard eigenvalue problem with uncertain-but-bounded parameters, the presented vertex solution theorem is unconditional, and the positive semi-definite solution theorem and the parameter decomposition solution theorem have less limitary conditions compared with Deif's solution theorem. The effectiveness of the vertex solution theorem, the positive semi-definite solution theorem and the parameter decomposition solution theorem are illustrated by numerical examples

Efficient approximations for learning phylogenetic HMM models from data.

We consider models useful for learning an evolutionary or phylogenetic tree from data consisting of DNA sequences corresponding to the leaves of the tree. In particular, we consider a general probabilistic model described in Siepel and Haussler that we call the phylogenetic-HMM model which generalizes the classical probabilistic models of Neyman and Felsenstein. Unfortunately, computing the likelihood of phylogenetic-HMM models is intractable. We consider several approximations for computing the likelihood of such models including an approximation introduced in Siepel and Haussler, loopy belief propagation and several variational methods. We demonstrate that, unlike the other approximations, variational methods are accurate and are guaranteed to lower bound the likelihood. In addition, we identify a particular variational approximation to be best-one in which the posterior distribution is variationally approximated using the classic Neyman-Felsenstein model. The application of our best approximation to data from the cystic fibrosis transmembrane conductance regulator gene region across nine eutherian mammals reveals a CpG effect.

Concentration of Size and Path Length of Tries

We show that, for a certain class of probabilistic models, the number of internal nodes $S_n$ of a trie built from $n$ independent and identically distributed keys is concentrated around its mean, in the sense that $\Var S_n=\Oh(\EE S_n)$. Keys are sequences of symbols which may be taken from varying alphabets, and the choice of the alphabet from which the $k$th symbol is taken, as well as its distribution, may depend on all the preceding symbols. In the construction of the trie we also allow for bucket sizes greater than 1. The property that characterizes our models is the following: there is a constant $C$ such that for any word $v$ that may occur as a prefix of a key the size $S^v_n$ of a trie built from the suffixes of $n$ independent keys conditioned to have the common prefix $v$ has the property $\EE S^v_n\leq Cn$. This class of models contains memoryless and Markovian source models as well as the probabilistic dynamical source models that were recently introduced and thoroughly developed by Vallee [Algorithmica 29 (2001) 262–306], in particular the continued fraction source. Furthermore we study the external path length $L_n$, which obeys $\EE L_n=\Oh(n\ln n)$ and $\Var L_n=\Oh(n\ln^2 n)$.

Aspects Regarding the Use of Probabilistic Models for Isothermal Full Film Rough Line Contacts

This article presents a method to improve the classical probabilistic EHL models by incorporating deterministic EHL results. We first provide a review of the latest developments in the probabilistic approach for the solution of a mixed EHL problem, as well as a review of the latest deterministic EHL results that can impact the probabilistic formulations. Next, we present a probabilistic algorithm for the computations of the load supported by the fluid, the elastically deformed asperities, and the plastically deformed asperities in a mixed EHL contact. The Chang-Etsion-Bogy elastoplastic micro contact model is used together with a corresponding probabilistic fluid flow model. We also present an approach in which the elastic roughness deformation induced by the oil film can be incorporated into the probabilistic model via the use of the amplitude reduction technique.

The query complexity of order-finding

We consider the problem where π is an unknown permutation on {0,1,…,2n−1}, y0∈{0,1,…,2n−1}, and the goal is to determine the minimum r>0 such that πr(y0)=1. Information about π is available only via queries that yield πx(y) from any x∈{0,1,…,2m−1} and y∈{0,1,…,2n−1} (where m is polynomial in n). The main resource under consideration is the number of these queries. We show that the number of queries necessary to solve the problem in the classical probabilistic bounded-error model is exponential in n. This contrasts sharply with the quantum bounded-error model, where a constant number of queries suffices.