Abstract S6-07: Double blinded validation study to assess performance of IBM artificial intelligence platform, Watson for oncology in comparison with Manipal multidisciplinary tumour board – First study of 638 breast cancer cases

Abstract

Abstract Background: IBM Watson for Oncology (WFO) developed in collaboration with Memorial Sloan Kettering Cancer Centre is a cognitive computing system able to extract structured data from free text documents using natural language processing (NLP). It is a technology platform that uses NLP and machine learning to reveal insights from large amounts of unstructured data. Currently WFO provides treatment options only for breast, lung and colorectal cancers. In the present study we try to evaluate concordance of WFO treatment recommendations with Manipal multidisciplinary tumor board (MMDT),a quaternary health care centre for 638 breast cancer cases. Materials and Methods: MMDT treatment recommendation and relevant clinic-pathological data of 638 breast cancer cases both localised (514) and metastatic (124) disease which were treated in last 3 years was collected and the collected data of all the patients was entered in WFO. The treatment recommendations & the time required to enter the data and the time taken by WFO to give recommendations after analyzing the data was recorded. Treatment recommendation by WFO came in three categories with colour coding green, orange and red . Green is the RECommended standard treatment (REC), Orange is For Consideration (FC) and Red is Not RECommended (NREC). Concordance between MMDT and WFO was analysed for all cases as per stage and receptor status. Results: Of the treatment recommendations given by MMDT, WFO provided 46.4% in REC, 26.1% in FC, 21.5% in NREC. Nearly 73% of the MMDT treatment recommendations were in WFO REC and WFO-FC group. However 6% of the treatment provided by MMDT was not available with WFO. Treatment recommendations from WFO were concordant with MMDT in nearly 80% of time in non-metastatic and 45% in metastatic disease. In subset analysis of breast cancer with respect to receptor status the WFO-REC treatment was high with triple negative disease with 67.9% and least with Her 2neu negative disease with 35%. Across all subsets with respect to receptor status concordance was better with non-metastatic disease over metastatic disease which was statistically significant. The mean time taken to collect the data and enter was 20 minutes which gradually decreased with acquaintance after 10 cases to 12 minutes. Metastatic disease took longer time (5-7 minutes more) over localized disease in all the groups. WFO took a median of 40 seconds to capture, analyze and give the treatment recommendations. Conclusions: WFO-REC and WFO-FC together were in 73 % of time concordant with the MMDT treatment recommendations. However with respect to metastatic disease and harmone positive Her 2 neu –ve disease there still needs to be lots of improvement from WFO. WFO is a step towards personalized medicine. It should be kept in mind that WFO will be only an assisting tool and it will never be able to replace the patient-doctor relationship which is a very essential component of treating a patient suffering with cancer. WFO will be a reliable artificial intelligence tool for every cancer center and its multidisciplinary tumor board and will change the quality of care in oncology. Citation Format: Somashekhar SP, Kumarc R, Rauthan A, Arun KR, Patil P, Ramya YE. Double blinded validation study to assess performance of IBM artificial intelligence platform, Watson for oncology in comparison with Manipal multidisciplinary tumour board – First study of 638 breast cancer cases [abstract]. In: Proceedings of the 2016 San Antonio Breast Cancer Symposium; 2016 Dec 6-10; San Antonio, TX. Philadelphia (PA): AACR; Cancer Res 2017;77(4 Suppl):Abstract nr S6-07.