Measuring the Impacts of Research Investments: Beyond the Economic Approach

Abstract

Research is funded from public coffers to solve problems and create opportunities. Those government officers and legislators funding research do so in anticipation of results, but these results can be difficult to measure. Policy officials have good intentions to respond to public calls for accountability; but the complexity of cause-and-effect relationships . Economic measures are better known, but social impact measures are growing in number. This paper reports on a scanning done of existing policy papers and academic literature to identify possible measures for social benefits of research. The review finds that, when measuring for social impact, a combination of measures, linked to an understanding of intent, is required to provide a full assessment of benefit. Many types of measures are presented in this report with the goal of identifying concrete methods that can assess societal benefits, to reveal what we can expect from research, and which tools provide input into a broader understanding. Tangible effects are the target, since diffuse social effects are beyond measures. Findings include the following six key points: 1. No single quantitative measure can reveal social impacts of research 2. Quantitative (numerical) measures can be useful when research is narrowly focused on a specific social problem. Measures can include the following items: a. Size and capabilities of the newly trained workforce b. Percent of the population impacted by research results c. Enhanced reputation of national researchers d. Growth in capabilities of universities e. Publications, patents, and knowledge mapping 3. Qualitative and quantitative approaches should be combined to provide a fuller assessment of benefit than would come from just numerical measures 4. Stakeholders, such as the general public, should define expectations early in the process of defining social benefits; public expectations should guide evaluation 5. The key actions required from stakeholders are the following: a. Tie measures to social values and relevance b. Consult experts and collect available data c. Specify outcomes of interest with references to goals d. Define ‘impact’ in terms of desired social benefits 6. The literature cautions against putting too much emphasis on quantitative accountability, since this can drive the research process towards near-term, incentivized behavior rather than the desirable, but broader, societal goals.