Information Retrieval: A Predictive Science

Abstract

Information Retrieval is the discipline that studies how recorded information is retrieved, as well as organized for use, and how the use and related needs develop and serve to drive the design and implementation of Information Retrieval systems. Studying how a document is retrieved, as well as the quality of the Information Retrieval process, can be performed using several different methods. One simple method is to perform experiments, trying different techniques—including different algorithms—with a given dataset, looking for performance improvements in the Information Retrieval processes. The TREC (Text REtrieval Conference) studies developed by the National Institute of Standards and Technology of the U.S. government provide datasets incorporating documents and queries that are used in the study of Information Retrieval performance, ideally allowing generalization to similar datasets, users, and domains. Psychology and sociology have developed techniques in survey design, interviewing, focus groups, and other techniques that can be applied to the social scientific study of searchers and contributors to Information Retrieval databases. Computer science has developed a range of research methods, such as those used in software engineering, to analyze the various aspects of algorithms, software modules, and development techniques. These methods can help place variables into their context as foundations upon which future evaluation may be conducted. Other non–TREC experimental studies often advance Information Retrieval, either for academic knowledge or for corporate advantage. Google, Microsoft, and Facebook, for example, conduct many “A/B” experiments, where almost all of their many machines continue to use the default algorithms that the company has developed, while a few machines use a variant of the algorithm to see how performance might improve with this variant. Because large technology companies function with so many users, one can determine the effectiveness of new techniques within brief time periods.