Evaluation of a national quality use of medicines service in Australia: an evolving model

Abstract

To describe the first phase of a global evaluation framework for the National Prescribing Service (NPS), with a focus on services for health professionals, and in particular highlight the lessons learnt from evaluation around the establishment and implementation of this national program. The agreed evaluation framework used mixed methods focused around a series of evaluation questions, aimed at measuring the overall effect of this new organization as well as the individual programs within it. The evaluation questions were determined a priori and were based on the objectives established by the organization in its first year of operation. A detailed analyses has been completed of: the process, scope and reach of program delivery using both quantitative and qualitative measures; changes in attitudes and knowledge measured through key informant interviews and surveys of professional groups and consumers; and changes in prescribing behaviour and savings to the Pharmaceutical Benefits Scheme (PBS) through analysis of prescription data. The evaluation period for this report was mid-1998 to mid-2004. The NPS has successfully implemented a complex, multi-faceted program across Australia. From 1998 to 2004, in addition to print material provided to all general practitioners (GPs) and pharmacists, 90% of all GPs have been actively involved in one or more educational activity; 116 of 120 divisions of general practice have coordinated local NPS programs; and 9% of pharmacists have actively participated in at least one educational activity. Sixty per cent of GPs and pharmacists rated the printed educational materials as good or very good. In the last three years, the NPS activities have generated savings in the range of $121-163 million to the PBS, owing to changes in prescribing practices. The national evaluation framework has informed program delivery and ongoing design and development. Continued refinement of existing evaluation methods and further exploration of new techniques will remain a priority for the organization.