Evaluating collaborative R&D programmes in information technology: the case of the U.K. Alvey programme

Abstract

Increasing attention is being paid to the evaluation of the economic and technological impact of large, government-sponsored, collaborative research and development (R&D) programmes in information technology. However, the practice of evaluation is still in its infancy and, as yet, there exist few guidelines as to how to evaluate major information technology (IT) programmes with multiple objectives. This paper describes the evaluation of the U.K. Alvey programme, focusing on how the technological and economic impact of the programme on the U.K. economy is being evaluated. The various methods adopted to evaluate the programme arc described and related to the multiplicity of goals contained within the Alvey programme. Also, a concrete example is provided of ‘real-time’ evaluation. It is argued that ongoing, or real-time, evaluation has a range of advantages over the conventional ex-post (after-the-event) evaluation, including the ability to feed back evaluation results to those responsible for directing the programme. Although IT programme evaluations will inevitably differ according to the aims, strategies, and rationale of each specific programme, by demonstrating the range of methods and techniques used to evaluate Alvey, the paper hopes to contribute to the general field of IT programme evaluation.