The Case of the Disappearing Caveat: A Critique of Irvine and Martin's Methodology

Abstract

The work of Irvine and Martin (IM) has provided a wealth of statistical data to be taken into account by future policy makers and historians. Those analysts will interpret the data in terms of the significance of what has been counted. This need for careful interpretation was sensitively recognized by IM in the methodological introduction to their paper on progress in radio astronomy (which I will refer to as RA).1 Even then they were to underestimate a fundamental problem, and its difficulties have been magnified as the sensitivity shown in the methodological introduction has been abandoned. The authors have failed to avoid the question: What is research in science for? That is a black hole into which their analysis is irretrievably sucked. First they impute unrealistic goals to science, and then they use indicators which do not indicate levels of achievement of those hypothetical goals. This Response focuses on their recent paper on 'Basic Research in the East and West' (EW).2 The object of that work is to evaluate the scientific outputs of high-energy accelerators in the Eastern bloc compared to those in the United States and Western Europe. But if one is to 'evaluate' science, or indeed anything else, performance must be matched to goals. When these are embedded in three such different cultural systems the issue is particularly salient. The objectives of science are explored by IM themselves in their recent book, Foresight in Science (FS).3 Here it is explained that the conventional differentiation of 'basic research', 'applied research' and 'experimental development' may be useful for statistical purposes, but not for science policy (FS,2). Within 'basic research' one can distinguish in terms of the patron's objectives between 'pure or curiosity oriented research' and 'strategic research' (FS, 4). The