Evaluating Research Impact: The Development of a Research for Impact Tool.

Komla Tsey,Kenny Lawson,Roxanne Bainbridge,Felecia Watkin,Janya Mccalman,Allison Rossetto,Irina Kinchin,Yvonne Cadet-James

doi:10.3389/fpubh.2016.00160

Abstract

This paper examines the process of developing a Research for Impact Tool in the contexts of general fiscal constraint, increased competition for funding, perennial concerns about the over-researching of Aboriginal and Torres Strait Islander issues without demonstrable benefits as well as conceptual and methodological difficulties of evaluating research impact. The aim is to highlight the challenges and opportunities involved in evaluating research impact to serve as resource for potential users of the research for impact tool and others interested in assessing the impact of research. A combination of literature reviews, workshops with researchers, and reflections by project team members and partners using participatory snowball techniques. Assessing research impact is perceived to be difficult, akin to the so-called "wicked problem," but not impossible. Heuristic and collaborative approach to research that takes the expectations of research users, research participants and the funders of research offers a pragmatic solution to evaluating research impact. The logic of the proposed Research for Impact Tool is based on the understanding that the value of research is to create evidence and/or products to support smarter decisions so as to improve the human condition. Research is, therefore, of limited value unless the evidence created is used to make smarter decisions for the betterment of society. A practical way of approaching research impact is, therefore, to start with the decisions confronting decision makers whether they are government policymakers, industry, professional practitioners, or households and the extent to which the research supports them to make smarter policy and practice decisions and the knock-on consequences of doing so. Embedded at each step in the impact planning and tracking process is the need for appropriate mix of expertise, capacity enhancement, and collaborative participatory learning-by-doing approaches. The tool was developed in the context of Aboriginal and Torres Strait Islander research but the basic idea that the way to assess research impact is to start upfront with the information needs of decisions makers is equally applicable to research in other settings, both applied (horizontal) and basic (vertical) research. The tool will be further tested and evaluated with researchers over the next 2 years (2016/17). The decision by the Australian Government to include 'industry engagement' and 'impact' as additions to the Excellence in Research for Australia (ERA) quality measures from 2018 makes the Research for Impact Tool a timely development. The wider challenge is to engage with major Australian research funding agencies to ensure consistent alignment and approaches across research users, communities, and funders in evaluating impact.

Highlights

Good decisions flow from great research -Lowitja Institute: http://www.lowitja.org.au/lowitja-videoAboriginal and Torres Strait Islander health literature has a long tradition of identifying and describing difficulties facing Indigenous peoples and proposing solutions
The stakeholder workshops revealed considerable consensus, interest, and support for a Lowitja Institute-led research impact agenda designed to improve the value of research for Indigenous people
It was agreed that as Lowitja Institutefunded research constituted only a small proportion of the total Indigenous health research output, a future research evaluation tool, if it is to have a meaningful impact on practice and, improve research value, must have buy-in from the major research funding bodies, such as the National Health and Medical Research Council (NHMRC) and the Australian Research Council (ARC)

Summary

Introduction

Aboriginal and Torres Strait Islander (hereafter, Indigenous) health literature has a long tradition of identifying and describing difficulties facing Indigenous peoples and proposing solutions. It repeatedly calls on researchers to show how their research contributes meaningfully to improving Indigenous health and wellbeing. The literature has often identified the “over-researching” of Indigenous Australians without demonstrable benefits [2,3,4,5,6]. It regularly details the significant ongoing health and social disparities between Indigenous peoples and other Australians. Some authors have pointed out the difficulties awaiting those who enter largely uncharted territory to assess the extent to which research contributes to improving Indigenous health. The assessment of research impact suffers from the age-old problem of attribution – how to differentiate the relative contribution of research from contextual and other factors [8, 9]

Methods

Results

Conclusion