Testing and Using Quantitative Methods in Science Policy Contexts: A Response to Hicks

Abstract

Diana Hicks' recent essayl provides a critique of co-citation bibliometric modelling as an information tool for science policy applications. The objective to examine the limitations of a quantitative method is entirely appropriate. Moreover, the essay raises a number of interesting questions about the testing and use of quantitative methods, particularly science mapping techniques, in science policy contexts. However, the arguments and assumptions on which the critique relies are seriously flawed. Beyond these problems, Hicks fails to ask the most pertinent questions about the technique she examines and about the use of quantitative methods for science policy purposes. Hicks' starting premise that there has been a paucity of critical discussion about possible policy uses of bibliometric models is well founded. Her general objective, to raise a critical debate about a quantitative method, is commendable. Such a debate is essential to the development of any research methodology, and perhaps particularly important for advancing understanding of the capabilities and limitations of one intended for use in policy situations. Too often the debate goes on outside of the open literature. Hicks is also correct in her observation that the methodological choices made in building bibliometric models, and the assumptions underlying analysts' interpretations of data from them, often have not been made explicit.2 A forthcoming paper addresses this problem and details the methodological nuances involved in building bibliometric models, and the types of interpretive uses that have been explored in national science policy contexts.3 The neglect of these issues by the producers and users of bibliometric models can be attributed in part to the economic competition in the