Non-Uniform Information Access in Collaborative Information Retrieval

Abstract

Although there has been a great deal of research into Collaborative Information Retrieval/Seeking (CIR/S), the majority has assumed that team members in a collaborative search team always have the same level of unrestricted access to information. However, observations from different domains such as healthcare, business, government, etc. have suggested that collaboration sometimes involves people with differing levels of access to information. This type of scenario has been referred to as Multi-Level Collaborative Information Retrieval (MLCIR). As well as a support for collaboration, MLCIR must consider information flow, security and shareability between collaborators. Therefore, existing CIR/S interface components and concepts (e.g. awareness, division of labour, persistence, etc.) may not be directly applicable to MLCIR. To the best of our knowledge, there has not been any systematic evaluations to investigate the impact of MLCIR on search outcomes and to provide user-centred design recommendations for MLCIR interfaces. To address this gap, a number of studies were conducted using real-world MLCIR examples as scenarios. This thesis highlights four research objectives; these are: 1) to investigate the impact of different collaborative search strategies and MLCIR scenarios on search outcomes using a user simulation approach, 2) to develop a preliminary MLCIR interface and investigate the impact of MLCIR scenarios on search outcomes using a user study, 3) to investigate the impact of existing CIR concepts on MLCIR search outcomes using a user study, and 4) to develop and evaluate a working prototype of an MLCIR interface that supports users for asynchronous collaboration. These investigations followed an iterative approach until a final MLCIR interface could be produced. The first study was a user simulated study which allowed us to test a large number of variables and to pick prominent ones for further user studies. It was found that when relevance feedback search strategies were used, search performance of non-full access (where team members had 90% access to collection) outperformed full access (where team members had 100% access to collection). This simulated study also fulfilled the first research objective. Next, a preliminary user study was conducted using variables selected based on the findings of the simulated study. This study uncovered a number of interface components, and awareness concepts such as query awareness, result awareness and team awareness that could be used to support users in MLCIR scenarios. This user study addressed the second research objective. Based on findings from this user study, two further user studies were conducted where we evaluated a number of interface components implemented to support query awareness, result awareness and team awareness. Results suggested that having access to query properties and result properties in MLCIR scenarios can leverage team members? search performance. These two user studies addressed the third research objective. Finally, a working MLCIR interface was built based on the findings elicited from our previous user studies. The interface was then evaluated using the web and non-uniform information access scenarios. The results showed that blacklisting search terms had a significant negative impact on search outcome than blacklisting documents (f-measure: 0.04 vs 0.197). This study fulfilled the final research objective. All five studies presented in this thesis provided us with an increasing understanding of how to develop an MLCIR interface that is usable and helpful for users. As a final product, a working MLCIR interface was produced which could provide collaborating users with useful awareness information without disclosing any sensitive data. Overall, this thesis provided important new design suggestions and showed reusability of CIR components for MLCIR interfaces, something that has never been demonstrated before. The interfaces, experimental designs and findings including design suggestions that are presented in this thesis serve as novel guidelines for further investigations into MLCIR.