Identifying Complex Patterns in Online Information Retrieval Processes

Abstract

We define a computable benchmark framework that replicates the behavior of users as they proceed through the alternatives ranked by a search engine and highlights the problems faced by artificial intelligence techniques to categorize the retrieval patterns generated by users endowed with different information assimilation capacities. The research is developed through three specific stages.First, we define a decision theoretical model that relates the threshold values determining the retrieval behavior of decision makers (DMs) to the distance in terms of semantic similarity between the descriptions observed in the snippets and the ideal ones considered by the user. The model is designed to highlight the complexity of the search process defined by DMs, who must consider combinations of the variables defining the alternatives, both observed and expected, together with the number of satisficing alternatives aimed to be observed. Second, we design a set of heuristic algorithms that mimic the online information retrieval behavior of DMs as reflected in their click through rates (CTRs). We illustrate how requiring DMs to observe two satisficing alternatives provides a sufficient approximation to their CTRs. Adding a third alternative delivers an almost identical set of CTRs to those displayed by DMs ([1], [2]). The mimicking quality of the heuristic algorithms prevails as alternatives are added up to include the ten ranked within the first page of search results. Third, the set of heuristic algorithms provides two different strings of data, the pages clicked by the DMs and, more importantly, a numerical representation of each of the observations and evaluations that determine the retrieval behavior of DMs. We illustrate how, even when providing several artificial intelligence techniques with both strings of data, the models face considerable problems categorizing DMs correctly as their information assimilation capacities are enhanced.