Research assessment in the UK and Italy: Costly and difficult, but probably worth it (at least for a while)

Abstract

This paper provides a comparative analysis of the development of the UK and Italian university research funding systems with a special focus on Peer Review-Based Research Assessment (PRBRA) and its cost. Much of the debate surrounding the value of performance-based allocation systems hinges on the disadvantages versus the benefits of their implementation, and there is very little evidence on either their absolute cost or their cost relative to other allocation systems. Our objective is to fill this gap, collating the best possible estimates of the costs of alternative research funding methods to inform the ongoing policy debate. First, we compare funding in the UK and Italy during the period 2005–2012 and analyze the development of performance-based allocation in the two systems. Second, based on public reports and documents collected from universities, we discuss the public agency and university costs of RAE2008 and REF2014 and provide some estimates for VQR2012. We find that RAE2008 costs accounted for less than 1% of the total performance allocation in the related period while the VQR2012 efficiency ratio is estimated at around 2.5%. Finally, we compare the costs and efficiency ratios of PRBRA with metrics-based assessment and Research Council allocations and show that costs increase going from metrics to PRBRA to Research Council allocation.