Relevance judgment when browsing a health discussion forum : content analysis of eye fixations

Abstract

ABSTRACTIntroduction. People are increasingly searching and browsing for health information on social media sites. This is a small study of the relevance criteria used by laypersons when browsing a health discussion forum under three conditions-when seeking information for their own health issue, for a friend's or relative's health issue, and with no particular issue in mind.Method. An eye-tracker system was used to identify what text users' eyes were fixated on when browsing post surrogates and post content on a health discussion forum. Eye-fixations indicated the text segments that the user's attention was focused on when making relevance judgments.Analysis. Content analysis was performed on the text segments with eye-fixation, to identify the types of information they contain. These types of information are considered to be the direct relevance criteria used.Results. Users seeking information for their own health issue focused on case-based information: the poster's symptom and history of disease, demographic information, and feelings about the symptom. They also focused on descriptions of disease and treatments. Participants seeking for other people's health issue focused on factual information: terminology, etiology and description of disease, and description of treatments. Participants browsing with no particular issue focused on topics of general interest such as smoking, and rare or unusual issues. Conclusion. While the relevance criteria of topicality, accuracy, currency and authority are not unimportant, they are not upper-most in the minds of users when assessing information content.INTRODUCTIONMore and more people are seeking and accessing health information on the Internet. A telephone survey of adults in the United States conducted in 2012 by PEW Internet & American Life Project (Fox & Duggan, 2013) found that 59% of adults had sought health information on the Internet in the past year. 35% had accessed health information to find out what medical condition they or someone else had. Among these online diagnosers, 53% talked about what they found with their clinicians and 41% had their diagnosis confirmed by their clinician. Moreover, the survey found that among the health information seekers, 16% had sought other people who shared the same concerns on the Internet.Internet users sought various kinds of health information online. The PEW survey (Fox & Duggan, 2013) found that among the people who looked for health information, 53% had sought information about specific diseases or problems, 43% for particular medical treatments, 27% for weight loss and control information, 19% for food safety information and 15% for drug information. But it is not clear how people decide what health information is relevant for particular purposes (e.g., diagnosis) and in different situations, and what relevance criteria they use. Most of the previous studies on relevance judgments were in the context of information retrieval in bibliographic or full text databases (e.g., Shawn, 1995). More recently, researchers studied relevance judgments on Web search engines (e.g., Balatsoukas & Ruthven, 2012). They identified various types of relevance criteria used during the information search process. As social media sites become more and more popular among Internet users, it is important to know what relevance criteria people use on social media sites for relevance decisions, and whether the criteria are the same as for search engines and databases.There are different types of social media applications with different characteristics and information behaviour of their users. Our study focused on a health discussion forum- HealthBoards.com (http://www.healthboards.com/boards/index.php). On a discussion forum, users post health issues, and view and respond to other people's posts. The posts on discussion forums are organized by topics and sub-topics in a hierarchical structure, which is different from Facebook and Instagram where people post information sequentially like a news feed, without organizing their posts. …