Measurement of incompatible probability in information retrieval: A case study with user clicks

Abstract

The incompatible probability represents an important non-classical phenomenon, and it describes conflicting observed marginal probabilities, which cannot be satisfied with a joint probability. First, the incompatibility of random variables was defined and discussed via the non-positive semi-definiteness of their covariance matrixes. Then, a method was proposed to verify the existence of incompatible probability for variables. A hypothesis testing was also applied to reexamine the likelihood of the observed marginal probabilities being integrated into a joint probability space, thus showing the statistical significance of incompatible probability cases. A case study with user click-through data provided the initial evidence of the incompatible probability in information retrieval (IR), particularly in user interaction. The experiments indicate that both incompatible and compatible cases can be found in IR data, and informational queries are more likely to be compatible than navigational queries. The results inspire new theoretical perspectives of modeling the complex interactions and phenomena in IR.