What Kind of Evidence do we Need for Evidence-Based Mental Health Policy? The Case of the Better Access Initiative

Abstract

In this issue of the journal, Jane Pirkis and colleagues report some of the core fi ndings from the evaluation of the Better Access scheme that was funded by the Commonwealth Department of Health and Aging [1]. This report, along with others [2 – 5], provides some of the fi rst evaluations of this innovative government program. As is noted in the companion editorial by Tony Jorm [6], the program is signifi cantly more expensive than was projected; the Department of Health and Ageing initially estimated forward expenditure on the program of AU $ 538 million over 4 years. The cost over this period is now understood to be about triple that fi gure. In a context where every dollar of public mental health spending is hard won by the sector, the effectiveness of this program becomes a critical matter. If it is effective, then it represents one of the most signifi cant, and praiseworthy, expansions of mental health services ever instigated in public policy. Indeed, if the program is effective, the fact that it has become much more costly than expected refl ects not only the popularity of the program with consumers and referrers, but will presumably result in an even more benefi cial impact on the nation ’ s mental health. However, if it is not effective, then the program represents misspent public funds and an enormous opportunity cost in the sense that the money could be spent on other programs that might be much more infl uential in improving the mental health of Australians. Perhaps because of these considerations, along with the fact that the scheme has the potential to aggravate professional rivalries, both between and within the professions, the program has attracted some controversy – often from commentators who lack any data to back up either their criticism or defence of the scheme. For this reason the recent evaluations of the scheme, including the paper in the current issue, are to be welcomed as they have the potential to serve as a vital resource in this important public debate. Indeed, in the companion editorial by Professor Jorm [6], he clearly lays out a series of criticisms that have been levelled at the scheme, and gives a generally positive scorecard for Better Access, largely based on these recent evaluations. Our interest here is not so much to comment on whether a positive or negative evaluation of the scheme can be made, but rather to comment on the limitations as to what conclusions can be drawn, given the very considerable methodological shortcomings of the evaluation. Indeed, we believe there are many key questions that are simply not answered by the current evaluation, in large part due to inadequate methodology. We fully appreciate the reasons why it would not have been possible to conduct a ‘ methodologically pristine ’ evaluation of the scheme, such as a traditional randomized controlled trial, in the current environment. Clearly signifi cant limitations apply to the evaluation that can be conducted of a government scheme that has already been rolled out on a nationwide basis (as noted by Prof Jorm [6]), and where limited fi nancial resources are made available for the evaluation (as alluded to by Pirkis and colleagues in their discussion of limitations [1]). It is also true to say, as Pirkis and colleagues do, that this evaluation is more rigorous than that applied to most government health spending – a clearly lamentable state of affairs. However, none of these limitations addresses the fundamental issue of whether the relevant questions have actually been answered. Just because an evaluation has been conducted