A Learning Program for Treatment Recommendations by Molecular Tumor Boards and Artificial Intelligence

Abstract

Substantial heterogeneity exists in treatment recommendations across molecular tumor boards (MTBs), especially for biomarkers with low evidence levels; therefore, the learning program is essential. To determine whether a learning program sharing treatment recommendations for biomarkers with low evidence levels contributes to the standardization of MTBs and to investigate the efficacy of an artificial intelligence (AI)-based annotation system. This prospective quality improvement study used 50 simulated cases to assess concordance of treatment recommendations between a central committee and participants. Forty-seven participants applied from April 7 to May 13, 2021. Fifty simulated cases were randomly divided into prelearning and postlearning evaluation groups to assess similar concordance based on previous investigations. Participants included MTBs at hub hospitals, treating physicians at core hospitals, and AI systems. Each participant made treatment recommendations for each prelearning case from registration to June 30, 2021; participated in the learning program on July 18, 2021; and made treatment recommendations for each postlearning case from August 3 to September 30, 2021. Data were analyzed from September 2 to December 10, 2021. The learning program shared the methodology of making appropriate treatment recommendations, especially for biomarkers with low evidence levels. The primary end point was the proportion of MTBs that met prespecified accreditation criteria for postlearning evaluations (approximately 90% concordance with high evidence levels and approximately 40% with low evidence levels). Key secondary end points were chronological enhancements in the concordance of treatment recommendations on postlearning evaluations from prelearning evaluations. Concordance of treatment recommendations by an AI system was an exploratory end point. Of the 47 participants who applied, 42 were eligible. The accreditation rate of the MTBs was 55.6% (95% CI, 35.3%-74.5%; P < .001). Concordance in MTBs increased from 58.7% (95% CI, 52.8%-64.4%) to 67.9% (95% CI, 61.0%-74.1%) (odds ratio, 1.40 [95% CI, 1.06-1.86]; P = .02). In postlearning evaluations, the concordance of treatment recommendations by the AI system was significantly higher than that of MTBs (88.0% [95% CI, 68.7%-96.1%]; P = .03). The findings of this quality improvement study suggest that use of a learning program improved the concordance of treatment recommendations provided by MTBs to central ones. Treatment recommendations made by an AI system showed higher concordance than that for MTBs, indicating the potential clinical utility of the AI system.