Reading Comprehension in Information Retrieval (RCIR) for Personalized Results

Abstract

ABSTRACTRecent research on personalized retrieval technology has been actively conducted to meet the needs of users for seeking adequate information. To refine the retrieval, researchers are considering user behavior patterns in a variety of ways. In this study, we use eye‐tracking metadata to predict users' levels of comprehension as textual evidence for IR processes. Furthermore, we incorporated eye‐tracking metadata on the Automated Readability Index (ARI), a readability assessment tool of an English text. Our research is largely divided into two tasks: i) comprehension evaluation task (CET) and ii) comprehension‐based retrieval task (CRT). In the CET task, for predicting the comprehension level, we applied various regression models. Among them, the Voting regressor demonstrated the highest performance with a Spearman's 𝜌 of 0.68. In the CRT task, we incorporated the level of comprehension predicted in the CET task and ARI score into the ranking results. We derived a sentenceBERT to find the relevant text for a query and the Normalized Discounted Cumulative Gain (nDCG) for evaluating the CRT task. The nDCG score for Comprehension Level only and that with ARI together were 0.65 and 0.78, respectively. Thus, applying ARI resulted in a higher nDCG score compared to comprehension level only.