Using formative evaluation in an implementation project to increase vaccination rates in high-risk veterans: QUERI Series

Carolyn M Wallace,Marcia W Legro

doi:10.1186/1748-5908-3-22

Abstract

BackgroundImplementation of research into practice in health care systems is a challenging and often unsuccessful endeavor. The United States Department of Veterans Affairs (VA) Quality Enhancement Research Initiative (QUERI) research teams include formative evaluations (FE) in their action-oriented VA implementation projects to identify critical information about the processes of implementation that can guide adjustments to project activities, in order to better meet project goals. This article describes the development and use of FE in an action-oriented implementation research project.MethodsThis two-year action-oriented implementation research project was conducted at 23 VA Spinal Cord Injury (SCI) Centers, and targeted patients, staff and the system of care, such as administration and information technology. Data for FE were collected by electronic and paper surveys, semi-structured and open-ended interviews, notes during conference calls, and exchange of e-mail messages. Specific questions were developed for each intervention (designed to improve vaccination rates for influenza in veterans with spinal cord injury and disorder); informants were selected for their knowledge of interventions and their use in SCI Centers.ResultsData from FE were compiled separately for each intervention to describe barriers to progress and guide adjustments to implementation activities. These data addressed the processes of implementing the interventions, problem-solving activities and the status of interventions at SCI Centers.ConclusionFormative evaluations provided the project team with a broad view of the processes of implementing multi-targeted interventions as well as the evolving status of the related best practice. Using FE was useful, although the challenges of conducting FE for non-field researchers should be addressed. Work is needed to develop methods for conducting FE across multiple sites, as well as acknowledging variations in local contexts that affect implementation of interventions.

Highlights

Implementation of research into practice in health care systems is a challenging and often unsuccessful endeavor
Reminder letter and information formative evaluations (FE) data from Year 1 were used immediately to provide staff at Spinal Cord Injury (SCI) Centers with specific assistance to generate the lists of patients
The project team had expected that the nationally-developed and distributed clinical reminder (CCR) for influenza would be used at all SCI Centers, we did not anticipate the variation in access to this CCR, variation in availability of information technology (IT) support to SCI Center staff, nor other barriers to its use that we found through FE

Summary

Introduction

Implementation of research into practice in health care systems is a challenging and often unsuccessful endeavor. The United States Department of Veterans Affairs (VA) Quality Enhancement Research Initiative (QUERI) research teams include formative evaluations (FE) in their action-oriented VA implementation projects to identify critical information about the processes of implementation that can guide adjustments to project activities, in order to better meet project goals. Implementation of research into practice in health care systems is a challenging and often unsuccessful endeavor, when those persons introducing or researching change fail to adequately understand and modify the context and progress of implementation or make appropriate adjustments to achieve goals. This article is one in a Series of articles documenting implementation science frameworks and approaches developed by the U.S Department of Veterans Affairs (VA) Quality Enhancement Research Initiative (QUERI). The Series' introductory article [3] highlights aspects of QUERI that are related to implementation science, and describes additional types of articles contained in the QUERI Series

Methods

Results

Discussion

Conclusion