PRM136 - A PILOT OF ONLINE SURVEY SOFTWARE FOR EXPERT ELICITATION AT NICE

Abstract

Clinical experts are routinely relied on to fill evidence gaps during development of health and care guidance. Use of online surveys to elicit information could help improve engagement, reduce time constraints and standardise methods within health technology assessment (HTA) agencies. This project assessed the use of online surveys for expert elicitation using published National Institute of Health and Care Excellence (NICE) guidance as a case study. We administered online surveys to 12 clinicians, using an evidence based protocol to develop questions and conduct the exercise. Respondents were asked to estimate means and 95% confidence intervals (CIs) for economic model input parameters where high-quality evidence was lacking. A second round of elicitation was conducted by email, where experts were invited to amend their estimates in light of aggregated responses (with no aim to achieve consensus). Seven (58%) clinical experts completed the online survey. Response rates were poor for some questions, particularly where respondents were asked to give 95% CIs around their estimates. Values derived from elicited information were suitable for use as model input parameters but often only represented responses from one or two individuals. The second round of elicitation had little impact on the results. Use of online surveys did not appear to improve expert engagement compared with current methods (typically use of email). Responses were often poor for questions requiring estimation of means and CIs, possibly reflecting areas of high uncertainty. However, use of online surveys did allow a standardised set of questions to be presented to experts remotely, in a user-friendly format, and generated outputs suitable for use in guidance-making. Use of standardised question sets may be beneficial to support expert elicitation in the context of HTA, but the approach will require further development to fit the time and resource constraints precluding widespread use by NICE.