Deploying artificial intelligence to find the needle in the haystack: deep learning for video capsule endoscopy

Abstract

Wireless video capsule endoscopy (CE) is a noninvasive endoscopic technique developed in the mid 1990s that has been used to visualize a wide spectrum of pathologic conditions of the small intestine, from inflammatory enteropathies such as inflammatory bowel disease and celiac disease to suspected small-bowel bleeding, to polyposis syndromes. Recent clinical practice guidelines put forth by the American Gastroenterological Association have recommended CE in a growing number of clinical scenarios, ranging from GI bleeding to polyposis syndromes.1Enns R.A. Hookey L. Armstrong D. et al.Clinical practice guidelines for the use of video capsule endoscopy.Gastroenterology. 2017; 152: 497-514Abstract Full Text Full Text PDF PubMed Scopus (197) Google Scholar Reviewing CE results in clinical practice, however, can be challenging and labor-intensive. A typical video capsule records 2 images per second and 50,000 to 60,000 still images in a single video recording. Finding a subtle abnormality among this number of images can feel akin to looking for a needle in the proverbial haystack, with average physicians’ reading times ranging from 45 to 120 minutes.2Shiotani A. Honda K. Kawakami M. et al.Analysis of small-bowel capsule endoscopy reading by using Quickview mode: training assistants for reading may produce a high diagnostic yield and save time for physicians.J Clin Gastroenterol. 2012; 46: e92-e95Crossref Scopus (32) Google Scholar Several strategies have been used that aim to reduce physicians’ reading times and increase accuracy. These strategies have included using an endoscopy nurse as a preliminary reader3Dokoutsidou H. Karagiannis S. Giannakoulopoulou E. et al.A study comparing an endoscopy nurse and an endoscopy physician in capsule endoscopy interpretation.Eur J Gastroenterol Hepatol. 2011; 23: 166-170Crossref Scopus (21) Google Scholar and using software designed to highlight images of potential interest.2Shiotani A. Honda K. Kawakami M. et al.Analysis of small-bowel capsule endoscopy reading by using Quickview mode: training assistants for reading may produce a high diagnostic yield and save time for physicians.J Clin Gastroenterol. 2012; 46: e92-e95Crossref Scopus (32) Google Scholar Artificial intelligence (AI), specifically deep learning, may offer a promising approach to streamlining the CE reading process. Over the past decade, machine learning and deep learning (a subset of machine learning) have been applied broadly to GI endoscopy, in applications ranging from colon polyp detection to dysplasia detection in Barrett’s esophagus.4Sharma P. Pante A. Gross S.A. Artificial intelligence in endoscopy.Gastrointest Endosc. 2019; 91: 925-931Abstract Full Text Full Text PDF Scopus (15) Google Scholar In the past year, several important studies have examined the role of deep learning in the automatic detection of erosions, ulcerations, and angioectasias in CE.5Aoki T. Yamada A. Aoyama K. et al.Automatic detection of erosions and ulcerations in wireless capsule endoscopy images based on a deep convolutional neural network.Gastrointest Endosc. 2019; 89 (357-63 e352)Abstract Full Text Full Text PDF Scopus (132) Google Scholar, 6Leenhardt R. Vasseur P. Li C. et al.A neural network algorithm for detection of GI angiectasia during small-bowel capsule endoscopy.Gastrointest Endosc. 2019; 89: 189-194Abstract Full Text Full Text PDF PubMed Scopus (107) Google Scholar, 7Tsuboi A. Oka S. Aoyama K. et al.Artificial intelligence using a convolutional neural network for automatic detection of small-bowel angioectasia in capsule endoscopy images.Dig Endosc. 2020; 32: 382-390Crossref Scopus (58) Google Scholar In this issue of Gastrointestinal Endoscopy, Saito et al8Saito H. Aoki T. Aoyama K. et al.Automatic detection and classification of protruding lesions in wireless capsule endoscopy images based on a deep convolutional neural network.Gastrointest Endosc. 2020; 92: 144-151.e1Abstract Full Text Full Text PDF PubMed Scopus (64) Google Scholar examine the role of a deep learning algorithm in the detection of protruding lesions during wireless CE. The authors developed a deep learning algorithm based on a single-shot multibox detector deep convolutional neural network to specifically detect protruding lesions in the small bowel during CE. The algorithm was developed using 30,584 images of protruding lesions from 292 patients from 3 hospitals in Japan: Sendai Kousei Hospital, the University of Tokyo, and Hiroshima University Hospital. The authors then tested or validated the algorithm using a separate set of 17,507 images (7507 “lesion-positive” and 10,000 “lesion-negative” images) from 93 patients. The area under the receiver operating characteristic curve for detection of protruding lesions was 0.911 (95% confidence interval, 0.9069-0.9155). The sensitivity and specificity of the convolutional neural network were 90.7% (95% confidence interval, 90.0%-91.4%) and 79.8% (95% confidence interval, 79.0%-80.6%).8Saito H. Aoki T. Aoyama K. et al.Automatic detection and classification of protruding lesions in wireless capsule endoscopy images based on a deep convolutional neural network.Gastrointest Endosc. 2020; 92: 144-151.e1Abstract Full Text Full Text PDF PubMed Scopus (64) Google Scholar This study has several strengths. To our knowledge, it is the first of its kind to specifically evaluate the role of computer-aided detection of polypoid lesions in the small bowel; in addition, the authors show the beginnings of accurate in-situ computer-aided diagnosis during CE, although the concordance of diagnosis between the AI algorithm and the expert endoscopist reader varied greatly depending on the lesion type, with a 42% concordance for polyps and an 82% to 83% concordance for nodules and epithelial tumors. As represented by the present work, the typical arc of research for AI applications in GI endoscopy begins with reporting on the performance of algorithms, which are trained and subsequently tested on retrospectively collected still images and/or video images. Once satisfactory performance standards are attained, the next critical step is a prospective clinical trial to determine whether application of the AI algorithm in clinical practice actually translates into a measurable clinical benefit for patients (or measurable efficiency benefits for physicians). In any case, the study by Saito et al8Saito H. Aoki T. Aoyama K. et al.Automatic detection and classification of protruding lesions in wireless capsule endoscopy images based on a deep convolutional neural network.Gastrointest Endosc. 2020; 92: 144-151.e1Abstract Full Text Full Text PDF PubMed Scopus (64) Google Scholar is a necessary first step along this path. In addition, as the authors note, this study specifically focuses on the detection of protruding lesions in the small bowel, which is a small subset of total lesions detected during CE. The authors plan to integrate the current deep learning algorithm with their prior work5Aoki T. Yamada A. Aoyama K. et al.Automatic detection of erosions and ulcerations in wireless capsule endoscopy images based on a deep convolutional neural network.Gastrointest Endosc. 2019; 89 (357-63 e352)Abstract Full Text Full Text PDF Scopus (132) Google Scholar,7Tsuboi A. Oka S. Aoyama K. et al.Artificial intelligence using a convolutional neural network for automatic detection of small-bowel angioectasia in capsule endoscopy images.Dig Endosc. 2020; 32: 382-390Crossref Scopus (58) Google Scholar to develop a more complete diagnostic system. In parallel, work by other groups has shown promise in the detection of all types of abnormalities encountered during CE. For example, recently Ding et al9Ding Z. Shi H. Zhang H. et al.Gastroenterologist-level identification of small-bowel diseases and normal variants by capsule endoscopy using a deep-learning model.Gastroenterology. 2019; 157 (1044-54 e1045)Abstract Full Text Full Text PDF Scopus (115) Google Scholar published the development and validation data of a deep learning algorithm designed to differentiate normal from abnormal images in CE. Their algorithm showed a high sensitivity and specificity for the detection of abnormal lesions (eg, inflammation, ulcer, polyps, vascular disease, blood, parasite, or diverticula) and of normal variants (lymphangiectasia, lymphatic follicular hyperplasia, and others). We are at the cusp of an AI revolution within GI endoscopy. Deep learning promises to change the landscape of video CE, and we predict that such algorithms will soon serve as an important adjunct to improve the detection and classification of lesions in the small bowel while reducing physicians’ reading times. Dr Berzin is a consultant for Wision AI, Fujifilm, and Medtronic. The other author disclosed no financial relationships. Automatic detection and classification of protruding lesions in wireless capsule endoscopy images based on a deep convolutional neural networkGastrointestinal EndoscopyVol. 92Issue 1PreviewProtruding lesions of the small bowel vary in wireless capsule endoscopy (WCE) images, and their automatic detection may be difficult. We aimed to develop and test a deep learning–based system to automatically detect protruding lesions of various types in WCE images. Full-Text PDF