Abstract
The intensification of an audit culture in higher education is made no more apparent than with the growing prevalence of performance-based research funding systems like the UK’s Research Excellence Framework (REF) and the introduction of new measures of assessment like ‘impact’ or more specifically, the economic and societal impacts of research. Detractors of this regulatory intervention, however, question the legitimacy and credibility of such a system for, and focus within, the evaluation of research performance. Within this study, we specifically sought to understand the process of evaluating the impact of research by gaining unique access as observers of a simulated impact evaluation exercise populated by aproximately n = 90 senior academic peer reviewers and user assessors, undertaken within one UK research-intensive university prior to and in preparation of its submission to REF2014. Over an intensive two-day period, we observed how peer reviewers and user assessors grouped into four overarching disciplinary panels went about deliberating and scoring impact, presented in the form of narrative-based case studies. Among other findings, our observations revealed that in their efforts to evaluate impact, peer reviewers were indirectly promoting a kind of impact mercantilism, where case studies that best sold impact were those rewarded with the highest evaluative scores.
Highlights
In the milieu of higher education’s marketisation; massification; globalisation and neoliberalization (Peck and Tickell 2002; see Giroux 2014), academics are challenged to become more gregarious even promiscuous, entrepreneurial and implicated in their external relationships
The first point to note about our observations and what they might tell us about how the academic assessment of impact takes place is that these data were generated at a very specific point in time, in very specific circumstances
Post-Research Excellence Framework (REF)-2014, academics and university administrators have a far clearer idea of what counts as impact, how to evidence impact claims and many of the other aspects of this process, which were highly uncertain for the mock-REF panels we observed
Summary
Following a successful application to the University’s REF Committee to observe the exercise and having secured ethical approval, we undertook observations across two days of intensive impact deliberations, comprising the full range of the University’s research activity, represented across four mock panels across which we rotated on an individual basis These observations were instrumental to our developing empirically based understandings of how impact case studies, across all disciplines, are scrutinised and valued by assessors. A further condition was that any submission of findings for publication occur in good time after the completion of REF2014 and on the basis that our delayed publication would mitigate against any lost competiveness suffered by the university in the terms of its impact submission by, albeit indirectly, openly declaring locally sourced knowledge related to the production of (potentially high-scoring) impact case studies This hiatus has been useful in neutralising any sensitivities potentially arising from the more critical dimensions of our discussion. For the purpose of clarity, we have distinguished ‘reviewers’ as those we observed in the assessment days from ‘panellists’ as the official REF impact evaluators
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
