Performance Measures for Schizophrenia Research: In Reply

Abstract

Back to table of contents Previous article Next article LettersFull AccessPerformance Measures for Schizophrenia Research: In ReplyDonald Emile Addington, M.B.B.S., Emily McKenzie, M.Sc., and JianLi Wang, Ph.D.Donald Emile AddingtonSearch for more papers by this author, M.B.B.S., Emily McKenzieSearch for more papers by this author, M.Sc., and JianLi WangSearch for more papers by this author, Ph.D.Published Online:1 Jul 2012https://doi.org/10.1176/appi.ps.20120p721aAboutSectionsPDF/EPUB ToolsAdd to favoritesDownload CitationsTrack Citations ShareShare onFacebookTwitterLinked InEmail In Reply: Burns has made three general points in his letter about our article on performance measures for schizophrenia services. First, he has endorsed the use of the Delphi process for engaging multiple stakeholder groups. Second, he has made pertinent observations about the process of using the Delphi based on his considerable experience working with multiple groups, including consumers. More specifically, he has pointed out the potential problem that occurs when groups rate many items as essential, and he has made several practical suggestions about how to encourage the group to be more focused and selective. We recognize the value of his suggestions and, in retrospect, we could have improved our Delphi process.We have a different perspective on his third suggestion, to return to the Delphi group with improved directions in order to obtain a more focused group of performance measures with the goal of changing the practice of individual clinicians with a set of “performance measures that nobody could ignore.” The reason underlying our perspective is that we see the identification of measures as only a small step on the path to implementing performance measures in mental health services. In a prior systematic review, we examined the facilitators and barriers to implementing performance measures (1). The review identified seven broad factors that influenced the implementation of performance measures, including indicator characteristics, promotional strategies, implementation strategies, resources, individual factors, organizational factors, and external factors. In our article on performance measures, which is the subject of Burns' letter, we focused on one factor only, indicator characteristics.We have previous experience with use of the Delphi process as part of the process of identifying performance measures for first-episode psychosis services (2). When we moved to implement these measures, we found that some measures rated as essential were, in practice, hard to specify with precision, and some were difficult to collect. When we studied the use of the available measures to compare services, only one measure, hospitalization, was sufficiently robust and widely available to be considered suitable for setting standards (3). We have since sought to increase the potential value of hospitalization as a performance measure for first-episode psychosis services. First, we developed a risk adjustment formula for hospitalization (4), and second, we tested the validity of hospitalization as a performance measure by demonstrating that it correlated with two other important outcome measures, quality of life and global psychopathology (5).In conclusion, we see the systematic review and rating of evidence as essential first steps in developing performance measures. The Delphi is a useful next step to narrow the number of measures, but it reflects only opinion evidence, one of the weaker levels of evidence in evidence-based medicine. Further practical steps include the rigorous definition of measures and further research to validate the measures. All these steps would still address only one of the seven factors (indicator characteristics) that we identified as facilitators and barriers to implementing performance measures in mental health services.References1 Addington D , Kyle T , Desai S , et al.: Facilitators and barriers to implementing quality measurement in primary mental health care: systematic review. Canadian Family Physician 56:1322–1331, 2010 Medline, Google Scholar2 Addington D , McKenzie E , Addington J , et al.: Performance measures for early psychosis treatment services. Psychiatric Services 56:1570–1582, 2005 Link, Google Scholar3 Addington D , Norman R , Adair C , et al.: A comparison of early psychosis treatment services using consensus and evidence-based performance measures: moving towards setting standards. Early Intervention in Psychiatry 3:274–281, 2009 Crossref, Google Scholar4 Addington DE , Beck C , Wang J , et al.: Predictors of admission in first-episode psychosis: developing a risk adjustment model for service comparisons. Psychiatric Services 61:483–488, 2010 Link, Google Scholar5 Addington DE , McKenzie E , Wang JL : Validity of hospital admission as an outcome measure of services for first-episode psychosis. Psychiatric Services 63:280–282, 2012 Link, Google Scholar FiguresReferencesCited byDetailsCited ByNone Volume 63Issue 7 July 2012Pages 721-722 Metrics PDF download History Published online 1 July 2012 Published in print 1 July 2012