Interactive Information Retrieval Research Articles

Understanding users' dynamic perceptions of search gain and cost in sessions: An expectation confirmation model

AbstractUnderstanding the roles of search gain and cost in users' search decision‐making is a key topic in interactive information retrieval (IIR). While previous research has developed user models based on simulated gains and costs, it is unclear how users' actual perceptions of search gains and costs form and change during search interactions. To address this gap, our study adopted expectation‐confirmation theory (ECT) to investigate users' perceptions of gains and costs. We re‐analyzed data from our previous study, examining how contextual and search features affect users' perceptions and how their expectation‐confirmation states impact their following searches. Our findings include: (1) The point where users' actual dwell time meets their constant expectation may serve as a reference point in evaluating perceived gain and cost; (2) these perceptions are associated with in situ experience represented by usefulness labels, browsing behaviors, and queries; (3) users' current confirmation states affect their perceptions of Web page usefulness in the subsequent query. Our findings demonstrate possible effects of expectation‐confirmation, prospect theory, and information foraging theory, highlighting the complex relationships among gain/cost, expectations, and dwell time at the query level, and the reference‐dependent expectation at the session level. These insights enrich user modeling and evaluation in human‐centered IR.

The 4Fs model of digital marketing: Its impact on brand positioning

Introduction/Objective: The 4Fs of digital marketing — flow, functionality, feedback, and loyalty — enhance effective and appealing communication across digital platforms. This study aims to evaluate the application of the 4Fs model in a Mexico-based ecotourism hotel, validating its measurement framework and analyzing the relationship between these variables and brand positioning. Methodology: A quantitative, non-experimental, cross-sectional, descriptive, and correlational approach was employed. The study surveyed 280 followers of the company’s Facebook page, using a 21-item questionnaire. Confirmatory factor analysis with maximum likelihood estimation assessed construct validity, while Cronbach’s alpha coefficient measured reliability. Results: Pearson correlation analysis revealed a strong, significant relationship between the 4Fs of digital marketing and brand positioning: flow (0.808), functionality (0.829), feedback (0.885), and loyalty (0.864). All correlations were statistically significant at the 0.01 level (two-tailed). Conclusions: The study affirmed that the company’s efforts on its websites to foster pleasant user interaction and effective information retrieval significantly contribute to creating memorable brand associations.

Multiple sampling schemes and deep learning improve active learning performance in drug-drug interaction information retrieval analysis from the literature

BackgroundDrug-drug interaction (DDI) information retrieval (IR) is an important natural language process (NLP) task from the PubMed literature. For the first time, active learning (AL) is studied in DDI IR analysis. DDI IR analysis from PubMed abstracts faces the challenges of relatively small positive DDI samples among overwhelmingly large negative samples. Random negative sampling and positive sampling are purposely designed to improve the efficiency of AL analysis. The consistency of random negative sampling and positive sampling is shown in the paper.ResultsPubMed abstracts are divided into two pools. Screened pool contains all abstracts that pass the DDI keywords query in PubMed, while unscreened pool includes all the other abstracts. At a prespecified recall rate of 0.95, DDI IR analysis precision is evaluated and compared. In screened pool IR analysis using supporting vector machine (SVM), similarity sampling plus uncertainty sampling improves the precision over uncertainty sampling, from 0.89 to 0.92 respectively. In the unscreened pool IR analysis, the integrated random negative sampling, positive sampling, and similarity sampling improve the precision over uncertainty sampling along, from 0.72 to 0.81 respectively. When we change the SVM to a deep learning method, all sampling schemes consistently improve DDI AL analysis in both screened pool and unscreened pool. Deep learning has significant improvement of precision over SVM, 0.96 vs. 0.92 in screened pool, and 0.90 vs. 0.81 in the unscreened pool, respectively.ConclusionsBy integrating various sampling schemes and deep learning algorithms into AL, the DDI IR analysis from literature is significantly improved. The random negative sampling and positive sampling are highly effective methods in improving AL analysis where the positive and negative samples are extremely imbalanced.

Optical character recognition quality affects subjective user perception of historical newspaper clippings

PurposeThis study aims to identify user perception of different qualities of optical character recognition (OCR) in texts. The purpose of this paper is to study the effect of different quality OCR on users' subjective perception through an interactive information retrieval task with a collection of one digitized historical Finnish newspaper.Design/methodology/approachThis study is based on the simulated work task model used in interactive information retrieval. Thirty-two users made searches to an article collection of Finnish newspaper Uusi Suometar 1869–1918 which consists of ca. 1.45 million autosegmented articles. The article search database had two versions of each article with different quality OCR. Each user performed six pre-formulated and six self-formulated short queries and evaluated subjectively the top 10 results using a graded relevance scale of 0–3. Users were not informed about the OCR quality differences of the otherwise identical articles.FindingsThe main result of the study is that improved OCR quality affects subjective user perception of historical newspaper articles positively: higher relevance scores are given to better-quality texts.Originality/valueTo the best of the authors’ knowledge, this simulated interactive work task experiment is the first one showing empirically that users' subjective relevance assessments are affected by a change in the quality of an optically read text.

The effects of work task difficulty on health information searching behaviour

Information needs and information seeking processes depend on contextual factors. Identifying and paying attention to contextual factors such as work tasks can be helpful for personalization of information retrieval. This study aimed to determine the relationship between work task difficulty and interactive health information searching behaviour (HISB). The study was performed through an analysis of transaction logs to identify relation between the work task difficulty and HISB of postgraduate student health ambassadors (PSHB). Thirty participants were purposefully sampled and invited to complete four simulated work tasks. User perception of work task difficulty, satisfaction, and success of search process were measured using questionnaires. A total of 120 log files were analysed. The results showed that with increasing tasks difficulty, PSHB see more results pages, review more information items and finally save more documents. Also, with increasing difficulty, PSHB enter more and longer queries in information retrieval systems and, in addition to changing the query more, they use more keywords and longer queries to search. As the task difficulty increases, the rate of satisfaction and success, as well as the navigational speed decrease. Understanding how work task attributes affect HISB can be effective for designers of interactive information retrieval systems, and developers of personalized health information retrieval systems and recommendation systems based on contextual information.

Clustering and Mining Algorithm of Factor Structure Data of Physical Education Teachers' Teaching Behavior Based on Internet Information Retrieval Algorithm

Integrating visual word list and Rocchio algorithm, constructs an implicit relevance feedback retrieval model in semantic space, and judges information retrieval preference. User demand mining is introduced, Jensen-Shannon divergence is used to calculate the relative entropy distance between the probability distributions of document sets, and similarity matching is calculated to complete interactive information retrieval. Teaching ability is gradually formed by teachers in long-term teaching practice, and is stably and comprehensively reflected in teaching viewpoints, teaching methods, teaching skills, teaching style and so on. The weight analysis of each index in the evaluation index system was carried out by using the analytic hierarchy process, and the selection of the evaluation subject was completed through the third round of expert questionnaires, and the evaluation criteria were formulated in combination with the opinions of tutors and some expert teachers.

Reflection on future directions: a systematic review of reported limitations and solutions in interactive information retrieval user studies

PurposeThe purpose of this paper is to understand how users behave and evaluate how systems with users are essential for interactive information retrieval (IIR) research. User study methodology serves as a primary approach to answering IIR research questions. In addition to designing user study procedures, understanding the limitations of varying study designs and discussing solutions to the limitations is also critical for improving the methods and advancing the knowledge in IIR.Design/methodology/approachGiven this unresolved gap, we apply the faceted framework developed by Liu and Shah (2019) in systematically reviewing 131 IIR user studies recently published (2016–2021) on multiple IR and information science venues.FindingsOur study achieve three goals: (1) extracting and synthesizing the reported limitations on multiple aspects of user study (e.g. recruitment, tasks, study procedures, system interfaces, data analysis methods) under associated facets; (2) summarizing the reported solutions to the limitations; (3) clarifying the connections between types of limitations and types of solutions.Practical implicationsThe bibliography of user studies can be used by students and junior researchers who are new to user-centered IR studies as references for study design. Our results can facilitate the reflection and improvement on IR research methodology and serve as a checklist for evaluating customized IIR user studies in varying problem spaces.Originality/valueTo our knowledge, this work is the first study that systematically reviews the study limitations and solutions reported by IIR researchers in articles and empirically examines their connections to different study components.

Predicting Search Task Difficulty through a Discrete‐Time Action Log Representation on Spectrum Kernel

ABSTRACTPredicting perceived difficulty on a web search task is an open problem in the interactive information retrieval field. A common approach to tackle it, is through features obtained from full search sessions, which are then used to train classification models. In this poster we attempt to predict perceived task difficulty at different stages of the search process. To do so, we use the spectrum kernel for support vector machine (SVM) classification. Our preliminary results suggest that by using behavioral data from the first query segment, it is possible to provide timely classifications of whether a search task is perceived as hard or easy.

Conversational Voice Assistants and a Case Study of Long-Term Users: A Human Information Behaviours Perspective

ABSTRACT This paper describes the human information behaviours of long-time users of voice assistants or chatbot software that focuses on voice interaction to retrieve information, and which operate on technologies that use artificial intelligence to understand and act upon the information requests of the user through a conversational interface. The paper reports on a case study based on 10 interviews with adult participants who have used voice assistants in their homes for at least six months. Interviews were semi-structured and focused on connecting to the users’ and their family’s experiences of using voice assistants in their daily lives. Insights from our findings focus on two areas, how the humanness of voice assistants and their perceived personality affect information retrieval, and secondly, the difficulties in transitioning from text-based information seeking to speech-based interactions. This paper makes a case for expanding human information behaviour studies to include interactive conversational information retrieval devices such as voice assistants.

Toward Cranfield-inspired reusability assessment in interactive information retrieval evaluation

Re-using research resources is essential for advancing knowledge and developing repeatable, empirically solid experiments in scientific fields, including interactive information retrieval (IIR). Despite recent efforts on standardizing research re-use and documentation, how to quantitatively measure the reusability of IIR resources still remains an open challenge. Inspired by the reusability evaluations on Cranfield experiments, our work proactively explores the problem of measuring IIR test collection reusability and makes threefold contributions: (1) constructing a novel usefulness-oriented framework with specific analytical methods for evaluating the reusability of IIR test collections consisting of query sets, document/page sets, and sets of task-document usefulness (tuse); (2) explaining the potential impacts of varying IIR-specific factors (e.g. search tasks, sessions, user characteristics) on test collection reusability; (3) proposing actionable methods for building reusable test collections in IIR and thereby amortizing the true cost of user-oriented evaluations. The Cranfield-inspired reusability assessment framework serves as an initial step towards accurately evaluating the reusability of IIR research resources and measuring the reproducibility of IIR evaluation results. It also demonstrates an innovative approach to integrating the insights from individual heterogeneous user studies with the evaluation techniques developed in standardized ad hoc retrieval experiments, which will facilitate the maturation of IIR fields and eventually benefits both sides of research.

Eye Gaze Relevance Feedback Indicators for Information Retrieval

There is a growing interest in the research on interactive information retrieval, particularly in the study of eye gaze-enhanced interaction. Feedback generated from user gaze features is important for developing an interactive information retrieval system. Generating these gaze features have become less difficult with the advancement of the eye tracker system over the years. In this work, eye movement as a source of relevant feedback was examined. A controlled user experiment was carried out and a set of documents were given to users to read before an eye tracker and rate the documents according to how relevant they are to a given task. Gaze features such as fixation duration, fixation count and heat maps were captured. The result showed a medium linear relationship between fixation count and user explicit ratings. Further analysis was carried out and three classifiers were compared in terms of predicting document relevance based on gaze features. It was found that the J48 decision tree classifier produced the highest accuracy.

Is anybody in there? Towards a model of affect and trust in human–AI information interactions

Advancements in search engines that utilize machine learning increase the likelihood that users will perceive these systems as worthy of trust.Methods. The nature and implications of trust in the context of algorithmic systems that utilize machine learning is examined and the resulting conception of trust is modelled. While current artificial intelligence does not meet the requirements of moral autonomy necessary to be considered trustworthy, people may still engage in misplaced trust based on the perception of moral autonomy. Users who place their trust in algorithmic systems limit their critical engagement with, and assessment of, the information interaction. A preliminary high-level model of trust’s role in information interactions adapting Ingwersen and Jarvelin’s Integrative Model for Interactive Information Seeking and Retrieval is proposed using the Google search engine as an example. We need to recognize that is it possible for users to react to information systems in a social manner that may lead to the formation of trust attitudes. As information professionals we want to develop interventions that will encourage users to stay critically engaged with their interactions with information systems, even when they perceive them to be autonomous.

Developing a Framework for Interactive Information Retrieval of Digital Libraries Based on Bandura's Social-Cognitive Theory

Developing a Framework for Interactive Information Retrieval of Digital Libraries Based on Bandura's Social-Cognitive Theory

Intellectual structure of information science 2011–2020: an author co-citation analysis

PurposeThis study continues a long history of author co-citation analysis of the intellectual structure of information science into the time period of 2011–2020. It also examines changes in this structure from 2006–2010 through 2011–2015 to 2016–2020. Results will contribute to a better understanding of the information science research field.Design/methodology/approachThe well-established procedures and techniques for author co-citation analysis were followed. Full records of research articles in core information science journals published during 2011–2020 were retrieved and downloaded from the Web of Science database. About 150 most highly cited authors in each of the two five-year time periods were selected from this dataset to represent this field, and their co-citation counts were calculated. Each co-citation matrix was input into SPSS for factor analysis, and results were visualized in Pajek. Factors were interpreted as specialties and labeled upon an examination of articles written by authors who load primarily on each factor.FindingsThe two-camp structure of information science continued to be present clearly. Bibliometric indicators for research evaluation dominated the Knowledge Domain Analysis camp during both fivr-year time periods, whereas interactive information retrieval (IR) dominated the IR camp during 2011–2015 but shared dominance with information behavior during 2016–2020. Bridging between the two camps became increasingly weaker and was only provided by the scholarly communication specialty during 2016–2020. The IR systems specialty drifted further away from the IR camp. The information behavior specialty experienced a deep slump during 2011–2020 in its evolution process. Altmetrics grew to dominate the Webometrics specialty and brought it to a sharp increase during 2016–2020.Originality/valueAuthor co-citation analysis (ACA) is effective in revealing intellectual structures of research fields. Most related studies used term-based methods to identify individual research topics but did not examine the interrelationships between these topics or the overall structure of the field. The few studies that did discuss the overall structure paid little attention to the effect of changes to the source journals on the results. The present study does not have these problems and continues the long history of benchmark contributions to a better understanding of the information science field using ACA.

The impact of reading fluency level on interactive information retrieval

Given an information need and the corresponding set of documents retrieved, it is known that user assessments for such documents differ from one user to another. One frequent reason that is put forward is the discordance between text complexity and user reading fluency. We explore this relationship from three different dimensions: quantitative features, subjective-assessed difficulty, and reader/text factors. In order to evaluate quantitative features, we wondered whether it is possible to find differences between documents that are evaluated by the user and those that are ignored according to the complexity of the document. Secondly, a task related to the evaluation of the relevance of short texts is proposed. For this end, users evaluated the relevance of these short texts by answering 20 queries. Documents complexity and relevance assessments were done previously by some human experts. Then, the relationship between participants assessments, experts assessments and document complexity is studied. Finally, a third experimentation was performed under the prism of neuro-Information Retrieval: while the participants were monitored with an electroencephalogram (EEG) headset, we tried to find a correlation among EEG signal, text difficulty and the level of comprehension of texts being read during the EEG recording. In light of the results obtained, we found some weak evidence showing that users responded to queries according to text complexity and user’s reading fluency. For the second and third group of experiments, we administered a sub-test from the Woodcock Reading Mastery Test to ensure that participants had a roughly average reading fluency. Nevertheless, we think that additional variables should be studied in the future in order to achieve a sound explanation of the interaction between text complexity and user profile.

Integrated Random Negative Sampling and Uncertainty Sampling in Active Learning Improve Clinical Drug Safety Drug–Drug Interaction Information Retrieval

Clinical drug–drug interactions (DDIs) have been a major cause for not only medical error but also adverse drug events (ADEs). The published literature on DDI clinical toxicity continues to grow significantly, and high-performance DDI information retrieval (IR) text mining methods are in high demand. The effectiveness of IR and its machine learning (ML) algorithm depends on the availability of a large amount of training and validation data that have been manually reviewed and annotated. In this study, we investigated how active learning (AL) might improve ML performance in clinical safety DDI IR analysis. We recognized that a direct application of AL would not address several primary challenges in DDI IR from the literature. For instance, the vast majority of abstracts in PubMed will be negative, existing positive and negative labeled samples do not represent the general sample distributions, and potentially biased samples may arise during uncertainty sampling in an AL algorithm. Therefore, we developed several novel sampling and ML schemes to improve AL performance in DDI IR analysis. In particular, random negative sampling was added as a part of AL since it has no expanse in the manual data label. We also used two ML algorithms in an AL process to differentiate random negative samples from manually labeled negative samples, and updated both the training and validation samples during the AL process to avoid or reduce biased sampling. Two supervised ML algorithms, support vector machine (SVM) and logistic regression (LR), were used to investigate the consistency of our proposed AL algorithm. Because the ultimate goal of clinical safety DDI IR is to retrieve all DDI toxicity–relevant abstracts, a recall rate of 0.99 was set in developing the AL methods. When we used our newly proposed AL method with SVM, the precision in differentiating the positive samples from manually labeled negative samples improved from 0.45 in the first round to 0.83 in the second round, and the precision in differentiating the positive samples from random negative samples improved from 0.70 to 0.82 in the first and second rounds, respectively. When our proposed AL method was used with LR, the improvements in precision followed a similar trend. However, the other AL algorithms tested did not show improved precision largely because of biased samples caused by the uncertainty sampling or differences between training and validation data sets.

Topic-independent modeling of user knowledge in informational search sessions

Web search is among the most frequent online activities. In this context, widespread informational queries entail user intentions to obtain knowledge with respect to a particular topic or domain. To serve learning needs better, recent research in the field of interactive information retrieval has advocated the importance of moving beyond relevance ranking of search results and considering a user’s knowledge state within learning oriented search sessions. Prior work has investigated the use of supervised models to predict a user’s knowledge gain and knowledge state from user interactions during a search session. However, the characteristics of the resources that a user interacts with have neither been sufficiently explored, nor exploited in this task. In this work, we introduce a novel set of resource-centric features and demonstrate their capacity to significantly improve supervised models for the task of predicting knowledge gain and knowledge state of users in Web search sessions. We make important contributions, given that reliable training data for such tasks is sparse and costly to obtain. We introduce various feature selection strategies geared towards selecting a limited subset of effective and generalizable features.

Orientation tactics and associated factors in the digital library environment: Comparison between blind and sighted users

AbstractThis is the first study that compares types of orientation tactics that blind and sighted users applied in their initial interactions with a digital library (DL) and the associated factors. Multiple methods were employed for data collection: questionnaires, think‐aloud protocols, and transaction logs. The paper identifies seven types of orientation tactics applied by the two groups of users. While sighted users focused on skimming DL content, blind users concentrated on exploring DL structure. Moreover, the authors discovered 13 types of system, user, and interaction factors that led to the use of orientation tactics. More system factors than user factors affect blind users' tactics in browsing DL structures. The findings of this study support the social model that the sight‐centered design of DLs, rather than blind users' disability, prohibits them from effectively interacting with a DL. Simultaneously, the results reveal the limitation of existing interactive information retrieval models that do not take people with disabilities into consideration. DL design implications are discussed based on the identified factors.

Deconstructing search tasks in interactive information retrieval: A systematic review of task dimensions and predictors

Increasingly, users are interacting with information retrieval (IR) systems with the goal of addressing complex, ill-defined search tasks. As a result, the interest in deconstructing and predicting search tasks has grown among IR researchers. Given the multidimensionality of search tasks, researchers usually focus on one or multiple dimensions and study the associations between implicit task dimensions and observable predictors. Synthesizing the knowledge of tasks and predictors learned from individual user studies can clarify the progresses we have made as a research community and provide an intellectual benchmark for further explorations of task-based search interactions. This article presents an overview of 76 task-based interactive IR (IIR) studies published between 2000 and 2020 and systematically coded the papers using features such as task dimensions (definitions and operationalizations), task-predictor associations, as well as task prediction models. Results include 1) data illustrating the growth and interdisciplinarity of IIR studies; 2) a comprehensive typology of task dimensions along with the associated measures; 3) a summary of the statistically significant correlations between task dimensions and predictors; 4) a list of the task dimensions being predicted, ground truth labels, and the feature sets employed. In addition, our bibliography of IIR works can be used by students and junior researchers who are new to the area. The results of our review can facilitate the growth of knowledge in IIR community and serve as the basis for future research on new modalities of user-task interactions.

An empirical study of interest, task complexity, and search behaviour on user engagement

User engagement has become an important outcome measure in interactive information retrieval (IIR) research, as commercial (e.g., search engines and e-commerce companies) and educational (e.g., libraries) enterprises focus on capturing and retaining customers. User engagement pertains to the kind of investment – emotional, cognitive, behavioural – the user is willing to make in an application. While research has shown how characteristics of users (e.g., individual differences and preferences) and the systems and content with which they interact influence engagement, less is understood about how the tasks people perform using digital applications affect their engagement. Drawing upon a wealth of literature in IIR, this study examined the effects of task on search engagement in a within-subjects Amazon Mechanical Turk (MTurk) experiment. Participants completed six search tasks on different task topics using task versions that included or excluded items and dimensions in the task descriptions. Items refer to things being compared (alternatives) and dimensions correspond to attributes by which items may differ. The task topics were meant to influence user interest in the task, and the versions were intended to manipulate the task doer's degree of certainty as they planned and performed the task, with the expectation that these factors would affect their self-reported engagement. We captured self-reported task perceptions (e.g., complexity, difficulty, interest) and logged search behaviours (e.g., querying, bookmarking) to both validate our manipulations and to understand how these variables related to engagement. Using multi-level modelling (MLM) we discovered that task topic affected user engagement, whereas task version had limited effects. However, participants’ perceptions of the tasks as interesting, difficult, and so on affected their engagement. Through the self-report and behavioural data, we observed that effort (more search engine results page exploration, greater perceived task difficulty) had a negative effect on engagement, while bookmarking pages and the ability to understand the task and how to complete it was associated with positive engagement. These results have implications for designing search tasks, deciphering the relationship between user experience and task complexity in IIR experiments, and aligning self-reports and search behaviours in evaluating online search engagement.