Fact-Finding or Exploration: Identifying Latent Behavior Clusters in User’s Search Activities

Abstract

Understanding the search behavior of users based on their current interaction with an information retrieval system, such as a search engine, can be quite beneficial to develop advanced support mechanisms. Statistical models comprise a rich framework for characterizing user’s search behavior for known search activity classes but also for identifying unknown/latent search behavior clusters. To gather deeper insights of search activities, we apply Hidden Markov Models to classify search sessions of known search activities with an accuracy of 88.58% and derive behavioral properties of users from these models. Afterwards, we extend these models to Mixture Models and cluster the users’ search behavior to reveal strong associations to the known search activities with an agreement of 79.1%, but also to identify new search behavior. That is, instead of just classifying already known search activities, our methodology also allows to identify new search activities solely on behavior parameters independent of the known search activities. Hence, in addition to qualitatively determined and evaluated search patterns, this statistical approach can be used to reveal new search activities from the data and therefore to further understand the human information seeking process. The method is suitable to detect new search behaviors as generic patterns in a latent parameter space. This enables the identification of user’s search activities that might be triggered contradictory to the originally expected search behavior and/or can reveal behavioral sub patterns.