ROMPER: The RAND/USC OPTIC Method for Policy Expert Ratings

Abstract

We developed an expert panel approach for identifying expert views on the effectiveness and implementability of population-level policy interventions. ROMPER—the RAND/USC OPTIC Method for Policy Expert Ratings—involves an online, three-round, modified-Delphi process:•Experts rate and comment on policies according to domains of the Grading of Recommendations Assessment, Development and Evaluation (GRADE) Evidence-to-Decision framework.•To identify consensus on policy effectiveness and implementability, expert ratings are analyzed using the Inter-Percentile Range Adjusted for Symmetry (IPRAS) technique from the RAND/UCLA Appropriateness Method and visualized using a forest plot. To explain consensus, expert comments are analyzed using reflexive thematic analysis and reported following the Standards for Reporting Qualitative Research.•To provide actionable information for decisionmakers, each policy is summarized in a “Policy Profile” adapted from GRADEPro Evidence-to-Decision tables.We validated ROMPER in two studies that successfully recruited the targeted sample size, retained experts through all three rounds, and examined consensus on which policies are (not) effective and implementable. ROMPER protocols, materials, data, and code are openly available on the Open Science Framework with Creative Commons licensing for replication and reuse. ROMPER provides a validated, replicable, open access approach for eliciting expert views on both policy effectiveness and implementability—and for summarizing (lack of) consensus specifically for policymakers.