Comparative analysis on the performance evaluation of national R&D projects1)

Abstract

Summary This paper reviews and compares the performance evaluation system of research and development (R&D) programs in advanced countries and provides suggestions for its improvement. The programs reviewed are the Advanced Technology Program (ATP) in the United States, the Framework Program in the European Union (EU), technology development programs in Japan, and industrial technology development programs in Korea. The evaluation of the Korea Institute of Industrial Technology Evaluation and Planning (ITEP) mainly focuses on the economic effects of the project, therefore detailed information on commercialization such as sales volume, cost reduction, increase in market share, costs of commercialization, changes in price, changes in demand, causes of suspended commercialization, and employment effects is required. ATP's Performance Report is interested not only in commercialization but also in creation and dissemination of knowledge. The performance report in EU measures three dimensions of performance: the achievement of goals, the outputs and outcomes, and the macro impacts of the research projects. The evaluation in Japan, both in project and in program, balances between the achievement of national policy goals and the economic effects. The results suggest that qualitative evaluation factors indicating creation and dissemination of knowledge and the macro impacts of the research projects/programs should be considered in the evaluation system of ITEP. In conclusion, R&D performance evaluation system is necessary to measure the achievement of goals, outputs, outcomes, and impacts using a multidimensional performance indicator. This paper provides awareness and access to the best evaluation methods available for project managers.