Performance evaluation

Abstract

What is experimental computer science? This question has been widely discussed ever since the Feldman Report was published (1979 [18]). Many computer scientists believe that survival of their discipline is intimately linked to their ability to rejuvenate experimentation. The National Science Foundation instituted the Coordinated Experimental Research Program (CERP) in 1979 to help universities set up facilities capable of supporting experimental research. Other agencies of government are considering similar programs. Some industrial firms are offering similar help through modest cash grants and equipment discounts.What is experimental computer science? Surprisingly, computer scientists disagree on the answer. A few believe that computer science is in flux—making a transition from theoretical to experimental science—and, hence, no operational definition is yet available. Some believe that it is all the non-theoretical activities of computer science, especially those conferring “hands-on” experience. Quite a few believe that it is large system development projects—i.e., computer and software engineering—and they cite MIT's Multics, Berkeley's version of Bell Labs' UNIX, the ARPAnet, IBM's database System R, and Xerox's Ethernet-based personal computer network as examples.These beliefs are wrong. There are well-established standards for experimental science. The field of performance evaluation meets these standards and provides examples of experimental science for the rest of the computing field.