Leaving cancer diagnosis to the computers

Abstract

Recent reports of artificial intelligence (AI)-based diagnosis of brain cancer, breast cancer, lung cancer, skin cancer, and others profess greater accuracy and faster diagnosis in patients than human specialists, giving prominence to the potential of using deep learning AI tools to improve cancer diagnoses. According to Cancer Research UK, 27·5 million new cancer cases could be diagnosed globally each year by 2040, meaning that radiological imaging data will continue to grow at an inordinate rate when compared with the number of radiologists. These factors have contributed to a dramatic increase in workloads and a study reports that an average radiologist must interpret one image every 3–4 seconds in an 8-hour workday to meet workload demands, making human error inevitable. Despite the existence of breast cancer screening programmes worldwide, the interpretation of mammograms is affected by high rates of false positives and false negatives. On Jan 1, 2020, Nature reported a breakthrough AI algorithm that could retrospectively outperform radiologists in the diagnosis of breast cancer using large mammogram data sets from the USA and UK, and in a controlled study of six expert radiologists who interpreted 500 randomly selected cases. Although this is not the first high performing deep learning algorithm to show promise, the study brought excitement to the field in part due to the large scale of the data sets used for training and subsequently validating the AI algorithm; the detailed description of how the AI algorithm could make decisions; and the superiority of the AI to human radiologists. This ultimately begs the question: will artificial intelligence be making our cancer diagnosis in the near future? Unfortunately, the short answer is no. The Nature study has numerous limitations that are common to many AI studies. Despite positive results in a retrospective controlled test, this AI has not been shown to work in a real-world setting. The Lancet Digital Health recently published a study analysing the current AI medical diagnostic field to show that less than 0·1% (14 of 20 000) studies were of sufficient methodological quality for clinical implementation. Just like drugs or medical devices, AI algorithms must show efficacy in robust clinical trials and in the real world to move into the next phase of clinical implementation. Furthermore, the performance of AI algorithms is highly dependent on the population used in the training sets, but in the Nature study, the ill-defined population means that we cannot be sure that the results are broadly applicable. Although the authors described how the AI algorithm could be making the decisions, because the tool was not generalised to different imaging hardware or scanning protocols, it is still not possible to fully understand the decision-making process of the AI. If AI systems are to be developed and used widely, the use of diverse population data will be critical. Sharing data between institutes and regions is becoming widely accepted as a necessity to AI research, with pledges from the US National Institutes of Health (NIH), US National Cancer Institute, Wellcome, and the Bill & Melinda Gates Foundation, which will require data gathered from any funded project to be made available to the scientific community. However, algorithm sharing is not subject to the data principles of FAIR: findable, accessible, interoperable, and reusable. Recent AI studies in cancer diagnosis have had proprietary algorithms, which are not publicly or commercially available, and which are developed by large corporations, such as Google Health. This is a major limitation in developing the next generation of AI health products, as it reduces the possibility that others can reproduce, validate, and build on the results, or monitor how algorithms are updated, although some have expressed concern that AI research may be too dangerous to share. Although the tools and training that radiologists receive is likely to change as a result of AI algorithms in the future, a major concern is a shortage of people with both medical knowledge and computer proficiency. With new and impressive algorithms published constantly, health-care workers need the necessary education to understand the strengths and weaknesses of the technology and the ability to rigorously assess its benefits in terms of clinical outcomes. With comprehensive education for our health-care workforce and openness to AI research in medicine, AI should make an impact sooner than we think.