A Critique of an Evaluation of the Impact of Hospital Bed Closures in Winnipeg, Canada: Lessons to Be Learned from Evaluation Research Methods

Abstract

In the last few years, the continuing increases in health care expenditures have led to a call for greater accountability in health care and have spurred evaluative research into the area of health care policy. Yet the challenge has been to develop health care policy evaluations that maximize the rigor of the evaluation process within the constraints and limitations of evaluation milieus. This paper describes the principles of evaluation research and, using the example of a study evaluating the impact of hospital bed closures on community health status in Winnipeg, Canada, demonstrates the epistemological, methodological and interpretive problems that can occur when these principles are not followed. The conclusions are that studies which fail to outline the causal links between policy actions and outcomes, and use designs and methods which threaten internal and external validity, are limited in their abilities to elucidate the impact of health care policy changes.