Polyp Images Research Articles

INVESTIGATION OF POLYPS IN ENDOSCOPY IMAGES BY USING DEEP LEARNING ALGORITHM

Recent advances in machine learning, particularly with regard to deep learning, help to recognize and classify objects in medical images. In this study, endoscopy images were examined and deep learning method was used to classify healthy and polyp cells. For the proposed system, a database was created from the archives of General Surgery Department Endoscopy Unit in Kutahya Evliya Celebi Training and Research Hospital. The database contains 93 polyps and 216 normal images from 54 archive records. For image multiplexing, a total of 1236 images were obtained by rotating each image 90 degrees around its axis. While 2/3 of the randomly selected data from this obtained data was used for training the model, the rest of the data was reserved for testing. K-fold Cross Validation method was used to reduce the variability of performance results. In this study, 48 different models were created by using different activation and optimization functions to find the best classification model in deep learning. According to the experimental results, it was observed that accuracy of the models depends on the selected parameters; the best model with the accuracy rate of 91% was obtained with 64 neurons in the hidden layer, ReLU activation function and RmsProp optimization method whereas the worst model with the accuracy rate of 76% was obtained with 32 neurons in the hidden layer, Tanh activation and PmsProp optimization functions. Accordingly, classification performance of polyp images can be optimized by utilizing different activation and optimization methods during the design of deep learning models.

S26 Development of an Artificial Intelligence Tool for Detection of Polypoid Lesions in Inflammatory Bowel Disease (IBD-CADe)

Background: There are multiple developed and developing artificial intelligence (AI) tools for real-time detection of colorectal polyps. However, this technology has yet to be developed for identification of polypoid and flat lesions in patients with inflammatory bowel disease (IBD). Not all polypoid lesions in IBD are dysplastic, with serrated epithelial changes and pseudopolyps frequently occurring in this patient population. There are also challenges identifying dysplasia in the setting of background inflammation. This study aims to train and test a YOLO (“You Only Look Once”) computer aided detection system (CADe) for automated detection of polypoid lesions in patients with IBD. Methods: An original CADe model was pretrained using 8,000 endoscopic images and trained with 2,268 annotated white light (WL) colon polyps from patients without IBD. First, this original CADe system was tested in our entire dataset of 2016 unlabeled WL IBD polypoid lesions which were annotated into 5 categories based on pathology: dysplastic, non-dysplastic, pseudopolyps, serrated changes (SECs) and sessile serrate adenomas (SSAs). Then, all IBD polyps were hand-labeled with bounding boxes by 8 physicians. A training dataset comprising 80% of the total pool of IBD polypoid lesions was used to re-train the original model and the additional 20% was used for iterations and testing. We measured sensitivity, positive predictive value (PPV), false positive rate (FPR), and accuracy (F1 score) before and after re-training the model with IBD lesions. The object detection model used was YOLO version 4 with a scaling cross stage network approach (ScaledYOLOv4). Results: When the original CADe model was first tested in IBD polyp images, the performance metrics were highest for dysplastic polyps and lowest for pseudopolyps and SECs likely because these IBD lesions were never seen before by the original CADe trained with non-IBD polyps only. The overall sensitivity, PPV, FPR and F1 was as follows: 0.50, 0.97, 1.7% and 0.64. After re-training the system with IBD polypoid lesions, the sensitivity improved for all polyp types while maintaining a high PPV and low FPR of 0.95, 0.95, 5% and 0.95 respectively. From the 9 lesions missed by IBD-CADe in the test set, 3 had a corresponding Mayo Score of 0, 2 had a score of 1, 2 had a score of 2 and 2 had a score of 3 indicating no significant difference in performance detection per level of background inflammation. Conclusion(s): We successfully developed the first IBD-CADe system capable of detecting polypoid lesions in IBD including serrated lesions and pseudopolyps even with background inflammation. The development of this tool is the first step to the creation of several technologies that are inclusive to patients with IBD, increasing polyp detection in this patient cohort and aiding the endoscopist in automated detection of polypoid dysplasia.

Colon Cancer Diagnosis Based on Machine Learning and Deep Learning: Modalities and Analysis Techniques

The treatment and diagnosis of colon cancer are considered to be social and economic challenges due to the high mortality rates. Every year, around the world, almost half a million people contract cancer, including colon cancer. Determining the grade of colon cancer mainly depends on analyzing the gland's structure by tissue region, which has led to the existence of various tests for screening that can be utilized to investigate polyp images and colorectal cancer. This article presents a comprehensive survey on the diagnosis of colon cancer. This covers many aspects related to colon cancer, such as its symptoms and grades as well as the available imaging modalities (particularly, histopathology images used for analysis) in addition to common diagnosis systems. Furthermore, the most widely used datasets and performance evaluation metrics are discussed. We provide a comprehensive review of the current studies on colon cancer, classified into deep-learning (DL) and machine-learning (ML) techniques, and we identify their main strengths and limitations. These techniques provide extensive support for identifying the early stages of cancer that lead to early treatment of the disease and produce a lower mortality rate compared with the rate produced after symptoms develop. In addition, these methods can help to prevent colorectal cancer from progressing through the removal of pre-malignant polyps, which can be achieved using screening tests to make the disease easier to diagnose. Finally, the existing challenges and future research directions that open the way for future work in this field are presented.

Enhanced segmentation of gastrointestinal polyps from capsule endoscopy images with artifacts using ensemble learning.

Endoscopy artifacts are widespread in real capsule endoscopy (CE) images but not in high-quality standard datasets. To improve the segmentation performance of polyps from CE images with artifacts based on ensemble learning. We collected 277 polyp images with CE artifacts from 5760 h of videos from 480 patients at Guangzhou First People's Hospital from January 2016 to December 2019. Two public high-quality standard external datasets were retrieved and used for the comparison experiments. For each dataset, we randomly segmented the data into training, validation, and testing sets for model training, selection, and testing. We compared the performance of the base models and the ensemble model in segmenting polyps from images with artifacts. The performance of the semantic segmentation model was affected by artifacts in the sample images, which also affected the results of polyp detection by CE using a single model. The evaluation based on real datasets with artifacts and standard datasets showed that the ensemble model of all state-of-the-art models performed better than the best corresponding base learner on the real dataset with artifacts. Compared with the corresponding optimal base learners, the intersection over union (IoU) and dice of the ensemble learning model increased to different degrees, ranging from 0.08% to 7.01% and 0.61% to 4.93%, respectively. Moreover, in the standard datasets without artifacts, most of the ensemble models were slightly better than the base learner, as demonstrated by the IoU and dice increases ranging from -0.28% to 1.20% and -0.61% to 0.76%, respectively. Ensemble learning can improve the segmentation accuracy of polyps from CE images with artifacts. Our results demonstrated an improvement in the detection rate of polyps with interference from artifacts.

HMA-Net: A deep U-shaped network combined with HarDNet and multi-attention mechanism for medical image segmentation.

Automatic segmentation of lesion, organ, and tissue from the medical image is an important part of medical image analysis, which are useful for improving the accuracy of disease diagnosis and clinical analysis. For skin melanomas lesions, the contrast ratio between lesions and surrounding skin is low and there are many irregular shapes, uneven distribution, and local and boundary features. Moreover, some hair covering the lesions destroys the local context. Polyp characteristics such as shape, size, and appearance vary at different development stages. Early polyps with small sizes have no distinctive features and could be easily mistaken for other intestinal structures, such as wrinkles and folds. Imaging positions and illumination conditions would alter polyps' appearance and lead to no visible transitions between polyps and surrounding tissue. It remains a challenging task to accurately segment the skin lesions and polyps due to the high variability in the location, shape, size, color, and texture of the target object. Developing a robust and accurate segmentation method for medical images is necessary. To achieve better segmentation performance while dealing with the difficulties above, a U-shape network based on the encoder and decoder structure is proposed to enhance the segmentation performance in target regions. In this paper, a novel deep network of the encoder-decoder model that combines HarDNet, dual attention (DA), and reverse attention (RA) is proposed. First, HarDNet68 is employed to extract the backbone features while improving the inference speed and computational efficiency. Second, the DA block is adopted to capture the global feature dependency in spatial and channel dimensions, and enrich the contextual information on local features. At last, three RA blocks are exploited to fuse and refine the boundary features to obtain the final segmentation results. Extensive experiments are conducted on a skin lesion dataset which consists of ISIC2016, ISIC2017, and ISIC 2018, and a polyp dataset which consists of several public datasets, that is, Kvasir, CVC-ClinicDB, CVC-ColonDB, ETIS, Endosece. The proposed method outperforms some state-of-art segmentation models on the ISIC2018, ISIC2017, and ISIC2016 datasets, with Jaccard's indexes of 0.846, 0.881, and 0.894, mean Dice coefficients of 0.907, 0.929, and 03939, precisions of 0.908, 0.977, and 0.968, and accuracies of 0.953, 0.975, and 0.972. Additionally, the proposed method also performs better than some state-of-art segmentation models on the Kvasir, CVC-ClinicDB, CVC-ColonDB, ETIS, and Endosece datasets, with mean Dice coefficients of 0.907, 0.935, 0.716, 0.667, and 0.887, mean intersection over union coefficients of 0.850, 0.885, 0.644, 0.595, and 0.821, structural similarity measures of 0.918, 0.953, 0.823, 0.807, and 0.933, enhanced alignment measures of 0.952, 0.983, 0.850, 0.817, and 0.957, mean absolute errors of 0.026, 0.007, 0.037, 0.030, and 0.009. The proposed deep network could improve lesion segmentation performance in polyp and skin lesion images. The quantitative and qualitative results show that the proposed method can effectively handle the challenging task of segmentation while revealing the great potential for clinical application.

Performance of a novel computer-aided diagnosis system in the characterization of colorectal polyps, and its role in meeting Preservation and Incorporation of Valuable Endoscopic Innovations standards set by the American Society of Gastrointestinal Endoscopy.

There has been an increasing role of artificial intelligence (AI) in the characterization of colorectal polyps. Recently, a novel AI algorithm for the characterization of polyps was developed by NEC Corporation (Japan). The aim of our study is to perform an external validation of this algorithm. The study was a video-based evaluation of the computer-aided diagnosis (CADx) system. Patients undergoing colonoscopy were recruited to record videos of colonic polyps. The frozen polyp images extracted from these videos were used for real-time histological prediction by the endoscopists and by the CADx system, and the results were compared. A total of 115 polyp images were extracted from 66 patients. Sensitivity, negative predictive valueand accuracy for diminutive polyps on white light imaging (WLI) and image-enhanced endoscopy (IEE) when assessed by CADx was 90.9% [95% confidence interval (CI) 77.3-100] and 95.8% [95% CI 87.5-100], 80% [95% CI 44.4-97.5] and 90.9% [95% CI 58.7-99.8], 84.8% [95% CI 72.7-97] and 84.6% [95%CI 71.8-94.9], respectively, compared to 48.1% [95%CI 37.7-59.1] and 72% [95% CI 62.5-81], 37.5% [95% CI 28.8-46.8] and 55% [95% CI 44.7-65.0], 53.7% [95% CI 44.2-63.2] and 66.7% [95% CI 59.7-73.3] when assessed by endoscopists. Concordance between histology and CADx-based post-polypectomy surveillance intervals was 93.02% on WLI and 96% on IEE. AI-based optical diagnosis is promising and has the potential to be better than the performance of general endoscopists. We believe that AI can help make real-time optical diagnoses of polyps meeting the Preservation and Incorporation of Valuable endoscopic Innovations standards set by the American Society of Gastrointestinal Endoscopy.

DAN-PD: Domain adaptive network with parallel decoder for polyp segmentation.

Endoscopy is essential for polyp diagnosis and prevention of colorectal cancer. Many deep learning methods have been proposed to perform automatic semantic segmentation of polyps in endoscopic images. However, labeled training images are always scarce, and the styles of endoscopic images from different medical centers vary greatly. The annotation of medical images requires much effort, and how to make more efficient utilization of the existing labeled data is becoming an increasingly critical issue. Considering the characteristics of polyp segmentation tasks and the need for generalization, we proposed a novel method named DAN-PD based on the Vision Transformer. Moreover, we devised the Teacher Parallel Encoder (TPE) and the Domain-Aware Parallel Decoder (DAPD) for the model. Our design innovatively introduces Unsupervised Domain Adaptation (UDA) methods and adversarial learning strategies to the polyp segmentation task. We conducted four transfer learning experiments with three public polyp image datasets to examine the model's performance. The results shows that our proposed method is ahead of other methods in all experiments and reaches the state-of-the-art level.

Polyp segmentation with consistency training and continuous update of pseudo-label

Polyp segmentation has accomplished massive triumph over the years in the field of supervised learning. However, obtaining a vast number of labeled datasets is commonly challenging in the medical domain. To solve this problem, we employ semi-supervised methods and suitably take advantage of unlabeled data to improve the performance of polyp image segmentation. First, we propose an encoder-decoder-based method well suited for the polyp with varying shape, size, and scales. Second, we utilize the teacher-student concept of training the model, where the teacher model is the student model’s exponential average. Third, to leverage the unlabeled dataset, we enforce a consistency technique and force the teacher model to generate a similar output on the different perturbed versions of the given input. Finally, we propose a method that upgrades the traditional pseudo-label method by learning the model with continuous update of pseudo-label. We show the efficacy of our proposed method on different polyp datasets, and hence attaining better results in semi-supervised settings. Extensive experiments demonstrate that our proposed method can propagate the unlabeled dataset’s essential information to improve performance.

Impact of the Volume and Distribution of Training Datasets in the Development of Deep-Learning Models for the Diagnosis of Colorectal Polyps in Endoscopy Images.

Background: Establishment of an artificial intelligence model in gastrointestinal endoscopy has no standardized dataset. The optimal volume or class distribution of training datasets has not been evaluated. An artificial intelligence model was previously created by the authors to classify endoscopic images of colorectal polyps into four categories, including advanced colorectal cancer, early cancers/high-grade dysplasia, tubular adenoma, and nonneoplasm. The aim of this study was to evaluate the impact of the volume and distribution of training dataset classes in the development of deep-learning models for colorectal polyp histopathology prediction from endoscopic images. Methods: The same 3828 endoscopic images that were used to create earlier models were used. An additional 6838 images were used to find the optimal volume and class distribution for a deep-learning model. Various amounts of data volume and class distributions were tried to establish deep-learning models. The training of deep-learning models uniformly used no-code platform Neuro-T. Accuracy was the primary outcome on four-class prediction. Results: The highest internal-test classification accuracy in the original dataset, doubled dataset, and tripled dataset was commonly shown by doubling the proportion of data for fewer categories (2:2:1:1 for advanced colorectal cancer: early cancers/high-grade dysplasia: tubular adenoma: non-neoplasm). Doubling the proportion of data for fewer categories in the original dataset showed the highest accuracy (86.4%, 95% confidence interval: 85.0–97.8%) compared to that of the doubled or tripled dataset. The total required number of images in this performance was only 2418 images. Gradient-weighted class activation mapping confirmed that the part that the deep-learning model pays attention to coincides with the part that the endoscopist pays attention to. Conclusion: As a result of a data-volume-dependent performance plateau in the classification model of colonoscopy, a dataset that has been doubled or tripled is not always beneficial to training. Deep-learning models would be more accurate if the proportion of fewer category lesions was increased.

Automatic Polyp Detection by Combining Conditional Generative Adversarial Network and Modified You-Only-Look-Once

Recent years have seen deep learning algorithms such as Convolutional Neural Networks (CNNs) and their variants achieving competitive performance in the application of medical image processing. Yet their shortcomings are: (i) the need of a large collection of annotated data, and (ii) the involvement of careful design of CNN layers. In this paper, we presented a novel polyp detection architecture based on two ideas: (i) to use Conditional Generative Adversarial Network (CGAN) to expand the training datasets, specifically the Generator takes advantage of the Efficient Spatial Pyramid (ESP) and the Discriminator is based on the PatchGAN; (ii) to modify the architecture of YOLOv4 using dilated convolution and skip connections. Experiments were performed using three publicly available datasets, i.e., CVC-ClinicDB, CVC-ColonDB, and ETIS-Larib Polyp DB. Experimental results showed that our generative network outperformed U-Net and can synthesize more realistic polyp images. The modified YOLOv4 significantly improved the performance of polyp detection using the expanded dataset, with an accuracy of 92.37% and a detection rate of 17.2 frames per second.

Non-equivalent images and pixels: Confidence-aware resampling with meta-learning mixup for polyp segmentation.

Automatic segmentation of polyp regions in endoscope images is essential for the early diagnosis and surgical planning of colorectal cancer. Recently, deep learning-based approaches have achieved remarkable progress for polyp segmentation, but they are at the expense of laborious large-scale pixel-wise annotations. In addition, these models treat samples equally, which may cause unstable training due to polyp variability. To address these issues, we propose a novel Meta-Learning Mixup (MLMix) data augmentation method and a Confidence-Aware Resampling (CAR) strategy for polyp segmentation. MLMix adaptively learns the interpolation policy for mixup data in a data-driven way, thereby transferring the original soft mixup label to a reliable hard label and enriching the limited training dataset. Considering the difficulty of polyp image variability in segmentation, the CAR strategy is proposed to progressively select relatively confident images and pixels to facilitate the representation ability of model and ensure the stability of the training procedure. Moreover, the CAR strategy leverages class distribution prior knowledge and assigns different penalty coefficients for polyp and normal classes to rebalance the selected data distribution. The effectiveness of the proposed MLMix data augmentation method and CAR strategy is demonstrated through comprehensive experiments, and our proposed model achieves state-of-the-art performance with 87.450% dice on the EndoScene test set and 86.453% dice on the wireless capsule endoscopy (WCE) polyp dataset.

Artificial intelligence for disease diagnosis: the criterion standard challenge

Artificial intelligence for disease diagnosis: the criterion standard challenge

Weakly Supervised Polyp Segmentation in Colonoscopy Images Using Deep Neural Networks.

Colorectal cancer (CRC) is a leading cause of mortality worldwide, and preventive screening modalities such as colonoscopy have been shown to noticeably decrease CRC incidence and mortality. Improving colonoscopy quality remains a challenging task due to limiting factors including the training levels of colonoscopists and the variability in polyp sizes, morphologies, and locations. Deep learning methods have led to state-of-the-art systems for the identification of polyps in colonoscopy videos. In this study, we show that deep learning can also be applied to the segmentation of polyps in real time, and the underlying models can be trained using mostly weakly labeled data, in the form of bounding box annotations that do not contain precise contour information. A novel dataset, Polyp-Box-Seg of 4070 colonoscopy images with polyps from over 2000 patients, is collected, and a subset of 1300 images is manually annotated with segmentation masks. A series of models is trained to evaluate various strategies that utilize bounding box annotations for segmentation tasks. A model trained on the 1300 polyp images with segmentation masks achieves a dice coefficient of 81.52%, which improves significantly to 85.53% when using a weakly supervised strategy leveraging bounding box images. The Polyp-Box-Seg dataset, together with a real-time video demonstration of the segmentation system, are publicly available.

Simple U-net based synthetic polyp image generation: Polyp to negative and negative to polyp

Synthetic polyp generation is a good alternative to overcome the privacy problem of medical data and the lack of various polyp samples. In this study, we propose a deep learning-based polyp image generation framework that generates synthetic polyp images that are similar to real ones. We suggest a framework that converts a given polyp image into a negative image (image without a polyp) using a simple conditional GAN architecture and then converts the negative image into a new-looking polyp image using the same network. In addition, by using the controllable polyp masks, polyps with various characteristics can be generated from one input condition. The generated polyp images can be used directly as training images for polyp detection and segmentation without additional labeling. To quantitatively assess the quality of generated synthetic polyps, we use public polyp image and video datasets combined with the generated synthetic images to examine the performance improvement of several detection and segmentation models. Experimental results show that we obtain performance gains when the generated polyp images are added to the training set.

Identifying and Quantifying the Difference between Adenomatous Colon Polyps and Normal Colon Tissue from Clinical Histological Images

This study aimed to evaluate automated image analysis as a tool for differentiating normal tissue from adenomatous polyp lesions. Images of colon surgical pathology samples from 140 patients (70 normal subjects and 70 polyp subjects) were captured on a personal computer using an Olympus DP20 digital photomicrography apparatus mounted on an Olympus light microscope (trinocular) at 1200×1800 dots per inch resolution, with the high power magnification of ×400. The complexity of image patterns was studied. In particular, image features of Fractal Dimension (FD) and Lacunarity (Lac) were extracted by using non-overlapping and sliding box-counting techniques. There were highly significant differences between the two clinical groups for both techniques, with polyp images in comparison with normal tissue images having significantly greater FD (1.83±0.04 v 1.73±0.08 p<0.0001) and significantly smaller Lac (0.28±0.08 v 0.42±0.12 p<0.0001) by using the non-overlapping box-counting method, and similar results by using the sliding box-counting method (1.86±0.09 v 1.78±0.08 p<0.0001, and 0.26±0.08 v 0.37±0.12 p<0.0001, respectively). These results encourage the use of computer automation, as normal colon tissue and adenomatous polyp tissue can be significantly differentiated

Dual U-Net with Resnet Encoder for Segmentation of Medical Images

Segmentation of medical images has been the most demanding and growing area currently for analysis of medical images. Segmentation of polyp images is a huge challenge because of the variability of color depth and morphology in polyps throughout colonoscopy imaging. For segmentation, in this work, we have used a dataset of images of the gastrointestinal polyp. The algorithms used in this paper for segmentation of gastrointestinal polyp images depend on profound deep convolutional neural network architectures: FCN, Dual U-net with Resnet Encoder, U-net, and Unet_Resnet. To improve the performance, data augmentation is performed on the dataset. The efficiency of the algorithms is measured by using metrics such as Dice Similarity Coefficient (DSC) and Intersection Over Union (IOU). The algorithm Dual U-net with Resnet Encoder obtains a higher DSC of 0.87 and IOU of 0.80 and beats the other algorithms U-net, FCN, and Unet_Resnet in segmentation of gastrointestinal polyp images.

A New Automated Polyp Detection Network MP-FSSD in WCE and Colonoscopy Images Based Fusion Single Shot Multibox Detector and Transfer Learning

Small polyp region detection in wireless capsule endoscopy (WCE) images is a challenging task in computer vision owing to two major problems: its variation in terms of shape, texture, and size, and the low illumination in the gastrointestinal tract. This study proposes a multiscale pyramidal fusion single-shot multibox detector network (MP-FSSD) to detect small polyp regions in WCE or colonoscopy frames, or both, with respect to the precision-vs-speed trade-off as the base architecture. We investigated deep transfer learning by transferring knowledge to polyp images, thereby enabling the extraction of highly representative features and contextual information from the FSSD. First, an edge-pooling layer was embedded in the shallow part of the network. Subsequently, the feature maps from different layers and scales were transformed to match their sizes. A concatenation module was introduced to integrate the feature maps from different layers, which were delivered to the next layer, followed by downsampling blocks to generate new pyramidal layers. Finally, the feature maps were fed to the multibox detectors to predict the final detection results. Experimentally, we maintained the same hyperparameters for both datasets for a fair comparison. The proposed MP-FSSD network exceeded FSSD by 3.62% in terms of mean average precision (mAP). The testing speed of 62.5 FPS is superior to that of the competitor detection methods. The proposal demonstrates that deep learning has much room for development in the field of gastrointestinal image detection.

Polyp Detection from Colorectum Images by Using Attentive YOLOv5.

Background: High-quality colonoscopy is essential to prevent the occurrence of colorectal cancers. The data of colonoscopy are mainly stored in the form of images. Therefore, artificial intelligence-assisted colonoscopy based on medical images is not only a research hotspot, but also one of the effective auxiliary means to improve the detection rate of adenomas. This research has become the focus of medical institutions and scientific research departments and has important clinical and scientific research value. Methods: In this paper, we propose a YOLOv5 model based on a self-attention mechanism for polyp target detection. This method uses the idea of regression, using the entire image as the input of the network and directly returning the target frame of this position in multiple positions of the image. In the feature extraction process, an attention mechanism is added to enhance the contribution of information-rich feature channels and weaken the interference of useless channels; Results: The experimental results show that the method can accurately identify polyp images, especially for the small polyps and the polyps with inconspicuous contrasts, and the detection speed is greatly improved compared with the comparison algorithm. Conclusions: This study will be of great help in reducing the missed diagnosis of clinicians during endoscopy and treatment, and it is also of great significance to the development of clinicians’ clinical work.

Detecting colon polyps in endoscopic images using artificial intelligence constructed with automated collection of annotated images from an endoscopy reporting system.

Artificial intelligence (AI) has made considerable progress in image recognition, especially in the analysis of endoscopic images. The availability of large-scale annotated datasets has contributed to the recent progress in this field. Datasets of high-quality annotated endoscopic images are widely available, particularly in Japan. A system for collecting annotated data reported daily could aid in accumulating a significant number of high-quality annotated datasets. We assessed the validity of using daily annotated endoscopic images in a constructed reporting system for a prototype AI model for polyp detection. We constructed an automated collection system for daily annotated datasets from an endoscopy reporting system. The key images were selected and annotated for each case only during daily practice, not to be performed retrospectively. We automatically extracted annotated endoscopic images of diminutive colon polyps that had been diagnosed (study period March-September 2018) using the keywords of diagnostic information, and additionally collect the normal colon images. The collected dataset was devised into training and validation to build and evaluate the AI system. The detection model was developed using a deep learning algorithm, RetinaNet. The automated system collected endoscopic images (47,391) from colonoscopies (745), and extracted key colon polyp images (1356) with localized annotations. The sensitivity, specificity, and accuracy of our AI model were 97.0%, 97.7%, and 97.3% (n = 300), respectively. The automated system enabled the development of a high-performance colon polyp detector using images in endoscopy reporting system without the efforts of retrospective annotation works.

A novel Joint-Net model for recognizing small-bowel polyp images

Introduction To automatically recognize polyps of enteroscopy images and avoid pathological change, a novel Joint-Net has been proposed. Material and methods The left half of the Joint-Net is constructed by transfer learning VGG16 and its right half is deepened based on the U-Net. In the previous two skip connections, a 3 × 3 convolution layer is added and the original two convolutions are replaced by the identity blocks. To connect the left and the right half part, the asymmetric convolution layer is used. In the output, the loophole-like structure is used. Results The enteroscopy images were obtained in Changhai Hospital of Shanghai. The mean values of Dice and intersection over union were 90.05% and 82.71%. The classification accuracy of normal images and polyp images was 93.50%. Conclusions The experiments show that the Joint-Net can segment and recognize the polyps successfully.