Abstract
This paper reports on the Evaluation Methodologies in Information Retrieval Seminar [1] held from 27 October to 1 November 2013 at the Schloss Dagstuhl Leibniz Center for Informatics that is a world-wide renowned venue for informatics where scientists come together to exchange their knowledge and to discuss their research findings. Dagstuhl offers modern facilities and is located in the beautiful countryside of Saarland, Germany. Schloss Dagstuhl has previously hosted other seminars on information retrieval topics, e.g. the Seminar 09101 on Interactive Information Retrieval which took place in 2009, but this one was the first one devoted to the specific topic of information retrieval evaluation. The seminar was attended by 42 participants from thirteen different countries, including a large number of established researchers as well as some some promising young researchers, and also practitioners from industry. As it is well-known evaluation of information retrieval (IR) systems has a long tradition (see, for example, [2]), however, the test-collection based evaluation paradigm is of limited value for assessing today’s IR applications, since it fails to address major aspects of the IR process. Thus there is a need for new evaluation methodologies, which are able to deal with the following issues:
Highlights
Social media refers to the interaction among people who share different types of information in a particular Internet service
“Task-Based information retrieval (IR)” typically refers to research focusing on the task or goal motivating a person to invoke an IR system, calling for systems being able to recognize the nature of the task and to support the accompanying search process
There is a wide variety of IR measures, but many of them are defined in an ad-hoc way, Basically, the definition of a new metric should consist of the following steps: Starting from the chosen criterion, assume a specific user behavior Define preferences
Summary
Social media refers to the interaction among people who share different types of information in a particular Internet service. When researchers and practitioners invoke social media, it is usually in the context of social networks like Facebook or Twitter. All these services have a strong focus on personal connections (e.g., friends, followers) and on user-generated content that is shaped at least in part by those social connections. While search engines like Google and Bing receive millions of queries per day, information dissemination and consumption is a prominent feature of services with a focus on social characteristics. This phenomenon is changing the landscape of how users access and share information. Bing introduced the annotations of Web links with social connections from Facebook, and Google implemented a similar feature using Google+
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Free) 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a call
