Toward a Science of Research Prioritization? The Use of Value of Information by Multidisciplinary Stakeholder Groups

Abstract

In this issue of Medical Decision Making, Carlson and others describe the process of convening an external group of diverse stakeholders to identify 3 high-priority cancer genomics tests for further research and to rank these in order of priority for conducting further research. The external multidisciplinary stakeholder advisory group involved 13 members, including patient advocates, payers, test developers, a regulator, policy makers, and practicing oncologists. This group committed to meet over 2 years to help the South West Oncology Group (SWOG) leadership and investigators identify and prioritize future research areas and to help assess where scarce research funds would be best spent. The stakeholder group began by identifying 3 topics for research from a larger number of possible options in the area of cancer genomic tests. They were provided with a landscape review, and they developed priority-setting criteria in collaboration with the SWOG investigators to undertake this process. They identified 3 genomic tests for which further research would be useful: 1) ERCC1 expression testing for platinum-based adjuvant therapy in fully resected earlystage non–small cell lung cancer (NSCLC); 2) epidermal growth factor receptor (EGFR) mutation testing for erlotinib maintenance therapy after firstline chemotherapy in advanced NSCLC, and 3) breast cancer tumor markers (BC markers) for detection of recurrence after primary breast cancer therapy. The stakeholder group was subsequently provided with value of information (VOI) analyses to quantify the value of conducting further research on each of these genomic tests. More specifically, they were provided with expected value of perfect information (EVPI) analyses, which estimate the upper level dollar value of reducing the uncertainty about which strategy is the optimal intervention. Providing the stakeholder group with these analyses involved a number of complex steps: developing simulation models for each genomic test, calculating the EVPI, training the external group of stakeholders in understanding what these results mean and how to use them, and discussing the results and reranking their choices based on what they may have learned from the VOI analyses. Indeed, Carlson and colleagues describe how the stakeholder group ranking of the tests changed from 1) ERCC1, 2) EGFR, 3) BC markers to 1) ERCC1, 2) BC markers, 3) EGFR. The BC markers ranked higher relative to EGFR compared with the initial prioritization vote in part because the VOI analyses helped the stakeholders appreciate the benefits of studying a technology associated with high uncertainty. Of the stakeholders, 69% reported finding these analyses helpful and 53% reported that they changed their priority ranking because of the information provided by the VOI results. The science of research prioritization has made important advances over the past 20 years. Research prioritization has moved from an approach largely based on convening groups of experts for unstructured discussions to a scientific process that studies who and how to engage in priority-setting decisions and provides measurements of the value provided Received 11 March 2013 from Patient Centered Outcomes Research Institute (RLF); Department of Medicine, Department of Economics, and the Harris School of Public Policy Studies at the University of Chicago (DOM). Accepted for publication 20 March 2013.