Monitoring and evaluation of transformative innovation policy: Suggestions for Improvement

Abstract

Transformative Innovation Policy (TIP) has an important role in the sudden transition that our economies require to face up to today's grand challenges (climate change, sustainable development goals). In the European Union (EU), Cohesion policy funds are one of the main financing instruments to support innovation and a fair transition. This paper focuses on TIP's monitoring and evaluation (M&E). To begin with, we discuss the various degrees of sophistication that can be found in monitoring and evaluation exercises (i.e. Storey's “6 steps to heaven” scale), ranging from interviews asking recipients whether they are happy to receive funding, to full-blown causal econometric analyses. We then provide a survey of causal inference techniques that reach the 6th step on this scale, and analyse the degree of sophistication of recent EU Cohesion project evaluations. We conclude that evaluation completed by EU Member States using causal inference techniques only represents 8% of the total evaluations conducted for period 2014–2020, and this percentage is even lower when we look at innovation or environmental-related programmes. We identify some gaps in the observed M&E of EU Member States and we provide some recommendations for how to set up M&E, contrasting traditional M&E with modern M&E, and highlighting the need for real-time data. In sum, we state that M&E needs improving, and we suggest how this might be done.