Using the Characteristics of Documents, Users and Tasks to Predict the Situational Relevance of Health Web Documents

Melinda Oroszlányová,Cristina Ribeiro,Sérgio Nunes,Carla Teixeira Lopes

doi:10.20897/jisem.201725

Abstract

Relevance is usually estimated by search engines using document content, disregarding the user behind the search and the characteristics of the task. In this work, we look at relevance as framed in a situational context, calling it situational relevance, and analyze whether it is possible to predict it using documents, users and tasks characteristics. Using an existing dataset composed of health web documents, relevance judgments for information needs, user and task characteristics, we build a multivariate prediction model for situational relevance. Our model has an accuracy of 77.17%. Our findings provide insights into features that could improve the estimation of relevance by search engines, helping to conciliate the systemic and situational views of relevance. In a near future we will work on the automatic assessment of document, user and task characteristics.

Highlights

It is estimated that 3.5 billion individuals (47.3% of the population) were Internet users in 2016 worldwide (World Telecommunication 2016)
Several user studies have been conducted with the aim to learn how people use online resources for their health concerns (Fox 2011, Espanha et al 2008), and how internet users search for health information on the Web (Fox 2006, Fox et al 2013)
The stratified model is based on theoretical concepts of human-computer interaction (HCI), and the stratificational theory developed in linguistics

Summary

Introduction

It is estimated that 3.5 billion individuals (47.3% of the population) were Internet users in 2016 worldwide (World Telecommunication 2016). Several information retrieval (IR) models have been developed to predict documents’ relevance (e.g., the classical Boolean model, vector space model and probabilistic model) They consist of a framework including representations of documents, queries, relationships among them and, in some cases, a ranking function. IR models rely on evaluations which consider traditional user and task models Such models are though inadequate, as for example they do not capture all types of informationseeking tasks, activities, and situations (Kelly et al 2009). The stratified model is based on theoretical concepts of human-computer interaction (HCI), and the stratificational theory developed in linguistics It considers the contemporary reality of IR, and the nature of relevance in information science, and optimizes the strengths, and minimizes the weaknesses of both the systems-centered and user-centered approaches to IR (Saracevic 1996). Besides explicit relevance models (Vargas et al 2012), multidimensional relevance modeling has been as well studied (Zhang et al 2014)

Objectives

Methods

Results

Conclusion