Who’s Your Expert? Use of an Expert Opinion Survey to Inform Development of American Psychiatric Association Practice Guidelines

Abstract

For many clinical questions in psychiatry, high-quality evidence is lacking. Credible practice guidelines for such questions depend on transparent, reproducible, and valid methods for assessing expert opinion. The objective of this study was to develop and demonstrate the feasibility of a method for assessing expert opinion to aid in the development of practice guidelines by the American Psychiatric Association (APA). A "snowball" process initially soliciting nominees from three sets of professional leaders was used to identify experts on a guideline topic (psychiatric evaluation). In a Web-based survey, the experts were asked to rate their level of agreement that specific assessments improve specific outcomes when they are included in an initial psychiatric evaluation. The experts were also asked about their own practice patterns with respect to the doing of the assessments. The main outcome measures are the following: number of nominated experts, number of experts who participated in the survey, and number and nature of quantitative and qualitative responses. The snowball process identified 1,738 experts, 784 (45 %) of whom participated in the opinion survey. Participants generally, but not always, agreed or strongly agreed that the assessments asked about would improve specified outcomes. Participants wrote 716 comments explaining why they might not typically include some assessments in an initial evaluation and 1,590 comments concerning other aspects of the topics under consideration. The snowball process based on initial solicitation of Psychiatry's leaders produced a large expert panel. The Web-based survey systematically assessed the opinions of these experts on the utility of specific psychiatric assessments, providing useful information to substantiate opinion-based practice guidelines on how to conduct a psychiatric evaluation. The considerable engagement of respondents shows promise for using this methodology in developing future APA practice guidelines.