Proceedings of the 2015 Workshop on Evaluation on Collaborative Information Retrieval and Seeking

Abstract

The publication date is one day earlier then the EST date to provide the proceedings to attendees in Australian on the first day of the conference It is our great pleasure to welcome you to the ECol 2015 Workshop on the Evaluation on Collaborative Information Seeking and Retrieval in conjunction with CIKM 2015. These last years and particularly since 2005, CIS and CIR have became emerging topics that have been addressed in several IR and IS conferences including CIKM and SIGIR conferences. While the potential of collaboration has been highlighted with respect to individual settings, other challenges remain and need to be thoroughly explored. Despite most of experimental evaluations have been done with the objective of highlighting the synergic effect of the proposed contributions, there is an important need for the future to discuss about what should be evaluated in terms of collaboration aspects (e.g. cognitive effort, mutual beneficial goal satisfaction, collective relevance...). Moreover, it does not exist standard framework as proposed in ad-hoc information retrieval through the evaluation campaign, as those proposed by TREC, INEX, CLEF... Previous related workshops (e.g. CIR at CIKM 2011, CIS at CSCW 2010-2013 and CIB at GROUP 2009) and special issues on "Collaborative Information Seeking" (IP&M, 2010; IEEE, 2014) gathered researchers exploring theoretical frameworks and applications focusing on CIS/CIR. More particularly, these workshops and special issues allowed opening relevant research directions dealing mainly with the modeling of collaborative search behavior, the design of appropriate collaborative search interfaces, and the formalization of retrieval models. However, the major issue of evaluation in CIS/CIR is still underexplored, and the understanding of how well the current evaluation protocols and metrics capture the dimension of collaboration still remains poor. Therefore, the mission of this workshop is to discuss the impact of collaboration features on evaluation as well as evaluation metrics and protocols. Finally, our goal is to open new directions in the CIS/CIR field. Our workshop would both interest and benefit from researchers with a complementary expertise that would cover all the aspects dealing with the evaluation in CIS and CIR. The call for papers attracted submissions from Asia, Australia, Europe, and the United States. We also encourage attendees to attend the keynote. This insightful talk can and will guide us to a better understanding of the future: Social and Collaborative Information Seeking: State of the Union, Chirag Shah (School of Communication and Information, Rutgers University, USA).