Colonoscopy Images Research Articles

Conversational LLM Chatbot ChatGPT-4 for Colonoscopy Boston Bowel Preparation Scoring: An Artificial Intelligence-to-Head Concordance Analysis

Background/objectives:To date, no studies have evaluated Chat Generative Pre-Trained Transformer (ChatGPT) as a large language model chatbot in optical applications for digestive endoscopy images. This study aimed to weigh the performance of ChatGPT-4 in assessing bowel preparation (BP) quality for colonoscopy. Methods: ChatGPT-4 analysed 663 anonymised endoscopic images, scoring each according to the Boston BP scale (BBPS). Expert physicians scored the same images subsequently. Results: ChatGPT-4 deemed 369 frames (62.9%) to be adequately prepared (i.e., BBPS > 1) compared to 524 frames (89.3%) assessed by human assessors. The agreement was slight (κ: 0.099, p = 0.0001). The raw human BBPS score was higher at 3 (2–3) than that of ChatGPT-4 at 2 (1–3), demonstrating moderate concordance (W: 0.554, p = 0.036). Conclusions: ChatGPT-4 demonstrates some potential in assessing BP on colonoscopy images, but further refinement is still needed.

Artificial intelligence-aided colonoscopic differential diagnosis between Crohn's disease and gastrointestinal tuberculosis.

Differentiating between Crohn's disease (CD) and gastrointestinal tuberculosis (GITB) is challenging. We aimed to evaluate the clinical applicability of an artificial intelligence (AI) model for this purpose. The AI model was developed and assessed using an internal dataset comprising 1,132 colonoscopy images of CD and 1,045 colonoscopy images of GITB at a tertiary referral center. Its stand-alone performance was further evaluated in an external dataset comprising 67 colonoscopy images of 17 CD patients and 63 colonoscopy images of 14 GITB patients from other institutions. Additionally, a crossover trial involving three expert endoscopists and three trainee endoscopists compared AI-assisted and unassisted human interpretations. In the internal dataset, the sensitivity, specificity, and accuracy of the AI model in distinguishing between CD and GITB were 95.3%, 100.0%, and 97.7%, respectively, with an area under the ROC curve of 0.997. In the external dataset, the AI model exhibited a sensitivity, specificity, and accuracy of 77.8%, 85.1%, and 81.5%, respectively, with an area under the ROC curve of 0.877. In the human endoscopist trial, AI assistance increased the pooled accuracy of the six endoscopists from 86.2% to 88.8% (P=0.010). While AI did not significantly enhance diagnostic accuracy for the experts (96.7% with AI vs 95.6% without, P=0.360), it significantly improved accuracy for the trainees (81.0% vs 76.7%, P=0.002). This AI model shows potential in aiding the accurate differential diagnosis between CD and GITB, particularly benefiting less experienced endoscopists.

Morphometric analysis and tortuosity typing of the large intestine segments on computed tomography colonography with artificial intelligence

Background:Morphological properties such as length and tortuosity of the large intestine segments play important roles, especially in interventional procedures like colonoscopy. Objective:Using computed tomography (CT) colonoscopy images, this study aimed to examine the morphological features of the colon's anatomical sections and investigate the relationship of these sections with each other or with age groups. The shapes of the transverse colon were analyzed using artificial intelligence. Materials and Methods:The study was conducted as a two- and three-dimensional examination of CT colonography images of people between 40 and 80 years old, which were obtained retrospectively. An artificial intelligence algorithm (YOLOv8) was used for shape detection on 3D colon images. Results:160 people with a mean age of 89 men and 71 women included in the study was 57.79±8.55 and 56.55±6.60, respectively, and there was no statistically significant difference (p=0.24). The total colon length was 166.11±25.07 cm for men and 158.73±21.92 cm for women, with no significant difference between groups (p=0.12). As a result of the training of the model Precision, Recall, and mAP were found to be 0.8578, 0.7940, and 0.9142, respectively. Conclusion:The study highlights the importance of understanding the type and morphology of the large intestine for accurate interpretation of CT colonography results and effective clinical management of patients with suspected large intestine abnormalities. Furthermore, this study showed that 88.57% of the images in the test data set were detected correctly and that AI can play an important role in colon typing.

Treatment of chronic sinus tract leakage at rectal anastomosis with anal fistula endoscopy

Objective: To introduce the method of using anal fistula endoscope to treat chronic sinus tract leakage at rectal anastomosis site. Methods: We used anal fistula endoscopy to treat a patient with chronic sinus tract leakage after radical resection of rectal cancer, mainly including the following 5 steps: (1) establishing a water injection circulation system through the anus; (2) scraping off purulent coating and mucosa on the surface of the sinus tract with the brush; (3) hemostasis and removal of necrotic tissue with electrocoagulation rods; (4) filling the sinus tract with bioprotein gel; (5) compressing the sinus tract with transanal drainage tube. Results: The patient is a 70 year old male with rectal cancer. After undergoing 3D laparoscopic assisted radical resection of rectal cancer via abdominal anterior resection (Dixon's procedure) and diverting ileostomy surgery for more than 3 months, leakage of the rectal anastomosis was found through colonoscopy and anal iodine water contrast imaging .The patient started eating and flowing juice 6 hours after surgery, got out of bed 24 hours after surgery, and was discharged 48 hours after the removal of the anal canal. Three months after surgery, colonoscopy and transanal iodine hydrography showed that the sinus repair was intact. The diverting ileostomy was reduced 4 months after surgery. Conclusion: Anal fistula endoscope is safe and feasible for the treatment of chronic sinus tract anastomotic leakage in selected patients.

Highlighted Diffusion Model as Plug-in Priors for Polyp Segmentation.

Automated polyp segmentation from colonoscopy images is crucial for colorectal cancer diagnosis. The accuracy of such segmentation, however, is challenged by two main factors. First, the variability in polyps' size, shape, and color, coupled with the scarcity of well-annotated data due to the need for specialized manual annotation, hampers the efficacy of existing deep learning methods. Second, concealed polyps often blend with adjacent intestinal tissues, leading to poor contrast that challenges segmentation models. Recently, diffusion models have been explored and adapted for polyp segmentation tasks. However, the significant domain gap between RGB-colonoscopy images and grayscale segmentation masks, along with the low efficiency of the diffusion generation process, hinders the practical implementation of these models. To mitigate these challenges, we introduce the Highlighted Diffusion Model Plus (HDM+), a two-stage polyp segmentation framework. This framework incorporates the Highlighted Diffusion Model (HDM) to provide explicit semantic guidance, thereby enhancing segmentation accuracy. In the initial stage, the HDM is trained using highlighted ground-truth data, which emphasizes polyp regions while suppressing the background in the images. This approach reduces the domain gap by focusing on the image itself rather than on the segmentation mask. In the subsequent second stage, we employ the highlighted features from the trained HDM's U-Net model as plug-in priors for polyp segmentation, rather than generating highlighted images, thereby increasing efficiency. Extensive experiments conducted on six polyp segmentation benchmarks demonstrate the effectiveness of our approach.

Advancements in the use of AI in the diagnosis and management of inflammatory bowel disease.

Inflammatory bowel disease (IBD) causes chronic inflammation of the colon and digestive tract, and it can be classified as Crohn's disease (CD) and Ulcerative colitis (UC). IBD is more prevalent in Europe and North America, however, since the beginning of the 21st century it has been increasing in South America, Asia, and Africa, leading to its consideration as a worldwide problem. Optical colonoscopy is one of the crucial tests in diagnosing and assessing the progression and prognosis of IBD, as it allows a real-time optical visualization of the colonic wall and ileum and allows for the collection of tissue samples. The accuracy of colonoscopy procedures depends on the expertise and ability of the endoscopists. Therefore, algorithms based on Deep Learning (DL) and Convolutional Neural Networks (CNN) for colonoscopy images and videos are growing in popularity, especially for the detection and classification of colorectal polyps. The performance of this system is dependent on the quality and quantity of the data used for training. There are several datasets publicly available for endoscopy images and videos, but most of them are solely specialized in polyps. The use of DL algorithms to detect IBD is still in its inception, most studies are based on assessing the severity of UC. As artificial intelligence (AI) grows in popularity there is a growing interest in the use of these algorithms for diagnosing and classifying the IBDs and managing their progression. To tackle this, more annotated colonoscopy images and videos will be required for the training of new and more reliable AI algorithms. This article discusses the current challenges in the early detection of IBD, focusing on the available AI algorithms, and databases, and the challenges ahead to improve the detection rate.

A lighter hybrid feature fusion framework for polyp segmentation

Colonoscopy is widely recognized as the most effective method for the detection of colon polyps, which is crucial for early screening of colorectal cancer. Polyp identification and segmentation in colonoscopy images require specialized medical knowledge and are often labor-intensive and expensive. Deep learning provides an intelligent and efficient approach for polyp segmentation. However, the variability in polyp size and the heterogeneity of polyp boundaries and interiors pose challenges for accurate segmentation. Currently, Transformer-based methods have become a mainstream trend for polyp segmentation. However, these methods tend to overlook local details due to the inherent characteristics of Transformer, leading to inferior results. Moreover, the computational burden brought by self-attention mechanisms hinders the practical application of these models. To address these issues, we propose a novel CNN-Transformer hybrid model for polyp segmentation (CTHP). CTHP combines the strengths of CNN, which excels at modeling local information, and Transformer, which excels at modeling global semantics, to enhance segmentation accuracy. We transform the self-attention computation over the entire feature map into the width and height directions, significantly improving computational efficiency. Additionally, we design a new information propagation module and introduce additional positional bias coefficients during the attention computation process, which reduces the dispersal of information introduced by deep and mixed feature fusion in the Transformer. Extensive experimental results demonstrate that our proposed model achieves state-of-the-art performance on multiple benchmark datasets for polyp segmentation. Furthermore, cross-domain generalization experiments show that our model exhibits excellent generalization performance.

MLFA-UNet: A multi-level feature assembly UNet for medical image segmentation

Medical image segmentation is crucial for accurate diagnosis and treatment in medical image analysis. Among the various methods employed, fully convolutional networks (FCNs) have emerged as a prominent approach for segmenting medical images. Notably, the U-Net architecture and its variants have gained widespread adoption in this domain. This paper introduces MLFA-UNet, an innovative architectural framework aimed at advancing medical image segmentation. MLFA-UNet adopts a U-shaped architecture and integrates two pivotal modules: multi-level feature assembly (MLFA) and multi-scale information attention (MSIA), complemented by a pixel-vanishing (PV) attention mechanism. These modules synergistically contribute to the segmentation process enhancement, fostering both robustness and segmentation precision. MLFA operates within both the network encoder and decoder, facilitating the extraction of local information crucial for accurately segmenting lesions. Furthermore, the bottleneck MSIA module serves to replace stacking modules, thereby expanding the receptive field and augmenting feature diversity, fortified by the PV attention mechanism. These integrated mechanisms work together to boost segmentation performance by effectively capturing both detailed local features and a broader range of contextual information, enhancing both accuracy and resilience in identifying lesions. To assess the versatility of the network, we conducted evaluations of MFLA-UNet across a range of medical image segmentation datasets, encompassing diverse imaging modalities such as wireless capsule endoscopy (WCE), colonoscopy, and dermoscopic images. Our results consistently demonstrate that MFLA-UNet outperforms state-of-the-art algorithms, achieving dice coefficients of 91.42%, 82.43%, 90.8%, and 88.68% for the MICCAI 2017 (Red Lesion), ISIC 2017, PH2, and CVC-ClinicalDB datasets, respectively.

DCSS-UNet: UNet based on State Space Model for Polyp Segmentation

Early and accurate segmentation of medical images can provide valuable information for medical treatment. In recent years, the automatic and accurate segmentation of polyps in colonoscopy images has received extensive attention from the research community of artificial intelligence and computer vision. Many researchers have conducted in-depth research on models based on CNN and Transformer. However, CNN have limited ability to model remote dependencies, which makes it challenging to fully utilize semantic information in images. On the other hand, the complexity of the secondary computation poses a challenge to the transformer. Recently, state-space models (SSMS), such as Mamba, have been recognized as a promising approach. They not only show superior performance in remote interaction, but also maintain linear computational complexity. Inspired by Mamba, we propose DCSS-UNet, where we utilize visual state space (VSS) blocks in VMamba to capture a wide range of contextual information. In the Skip connection phase, we propose Skip Connects Feature Attention modules(SFA) to better communicate information from the encoder. In the decoder stage, we innovatively combined the Temporal Fusion Attention Module(TFAM) to effectively fuse the feature information. In addition, we introduced a custom Loss calculation method, Tversky Loss, for the model to achieve faster convergence and improve segmentation along polyp boundaries. Our model was trained on the Kvasir-SEG and CVC-ClinicDB datasets, and validated on datasets Kvasir-SEG, CVC-ColonDB, CVC-300, and ETIS. The results show that the model achieves good segmentation accuracy and generalization performance with a low number of parameters. We are 6.1% ahead in the Kavirs-SEG dataset and 3.1% ahead in the CVC-ClinicDB dataset compared to VM-UNet.

An Edge-Enhanced Network for Polyp Segmentation.

Colorectal cancer remains a leading cause of cancer-related deaths worldwide, with early detection and removal of polyps being critical in preventing disease progression. Automated polyp segmentation, particularly in colonoscopy images, is a challenging task due to the variability in polyp appearance and the low contrast between polyps and surrounding tissues. In this work, we propose an edge-enhanced network (EENet) designed to address these challenges by integrating two novel modules: the covariance edge-enhanced attention (CEEA) and cross-scale edge enhancement (CSEE) modules. The CEEA module leverages covariance-based attention to enhance boundary detection, while the CSEE module bridges multi-scale features to preserve fine-grained edge details. To further improve the accuracy of polyp segmentation, we introduce a hybrid loss function that combines cross-entropy loss with edge-aware loss. Extensive experiments show that the EENet achieves a Dice score of 0.9208 and an IoU of 0.8664 on the Kvasir-SEG dataset, surpassing state-of-the-art models such as Polyp-PVT and PraNet. Furthermore, it records a Dice score of 0.9316 and an IoU of 0.8817 on the CVC-ClinicDB dataset, demonstrating its strong potential for clinical application in polyp segmentation. Ablation studies further validate the contribution of the CEEA and CSEE modules.

Artificial intelligence based real time colorectal cancer screening study: Polyp segmentation and classification using multi-house database

PurposeThis work explores the potential of artificial intelligence (AI) for real-time colorectal cancer (CRC) screening. We utilized a multi-house dataset to achieve a three-step approach (i) automated polyp segmentation, (ii) analysis of machine learning (ML) for classification and (iii) to compare deep learning (DL) exploration for automated classification. MethodsTo achieve our objectives, we leveraged the CRPU-Net architecture for automated polyp segmentation within the multi-house colonoscopy images. Subsequently, a ML framework was employed for binary classification of the segmented regions. Feature extraction was performed using SIFT and ORB descriptors, followed by the feature fusion and dimensionality reduction through Principal Component Analysis (PCA) to create an optimal feature representation for classification. We further explored the potential of DL by investigating the use of SOTA and vision transformer (ViT) model. Additionally, we considered the development of a hybrid model (CRP-ViT) by potentially combining the ViT model with DL architectures. ResultsCRPU-Net achieved outstanding segmentation performance of 96.56% accuracy, 95.40% IoU, and 97.39% Dice coefficient. Additionally, the CRP-ViT model demonstrated promising results in binary classification of 96.59% accuracy, 96.38% sensitivity, 96.80% specificity, 96.82% precision, and 96.36% NPV. ConclusionThis study implemented CRPU-Net and CRP-ViT, AI systems designed for real-time polyp segmentation, and potential precancerous polyp classification. This comprehensive approach offers the potential to expedite colonoscopies, minimize unnecessary biopsies, improve patient care, and optimize healthcare resource allocation.

A complete benchmark for polyp detection, segmentation and classification in colonoscopy images.

Colorectal cancer (CRC) is one of the main causes of deaths worldwide. Early detection and diagnosis of its precursor lesion, the polyp, is key to reduce its mortality and to improve procedure efficiency. During the last two decades, several computational methods have been proposed to assist clinicians in detection, segmentation and classification tasks but the lack of a common public validation framework makes it difficult to determine which of them is ready to be deployed in the exploration room. This study presents a complete validation framework and we compare several methodologies for each of the polyp characterization tasks. Results show that the majority of the approaches are able to provide good performance for the detection and segmentation task, but that there is room for improvement regarding polyp classification. While studied show promising results in the assistance of polyp detection and segmentation tasks, further research should be done in classification task to obtain reliable results to assist the clinicians during the procedure. The presented framework provides a standarized method for evaluating and comparing different approaches, which could facilitate the identification of clinically prepared assisting methods.

Dataset-level color augmentation and multi-scale exploration methods for polyp segmentation

Automatic segmentation of polyps from colonoscopy images plays a critical role in early screening and treatment of colorectal cancer. Although deep learning methods have made significant progress, precise polyp segmentation faces two challenges: (1) the imbalance of color appearances in the limited training dataset hinders the generalization of the model, and (2) polyps have the diverse scales, locations and shapes with blurred boundary. To address the issues, Dataset-Level Color Augmentation (DLCA) and Convolutional Multi-scale Attention Module (CMAM) are proposed. DLCA employs the dataset-level color knowledge and generate new color appearances, to avoid the model learning false associations with colors. Meanwhile, CMAM simultaneously explores the region and boundary clues and model multi-scale context, which improves the accuracy of polyp localization and fine-grained segmentation. We conduct comprehensive experiments and compare our network with state-of-the-art methods. The proposed model is superior on multiple polyp datasets, especially on ETIS, where mDice and mIoU reach 0.839 and 0.766. Furthermore, it is validated that DLCA can be widely applied to most polyp segmentation methods, and CMAM is practical and plug-and-play. Finally, the model demonstrates promising generalization to breast ultrasound and skin lesion segmentation.

Unmasking colorectal cancer: A high-performance semantic network for polyp and surgical instrument segmentation

Colorectal cancer (CRC) remains a significant health concern, with colonoscopy serving as the gold standard for diagnosis. Accurately segmenting polyps from colonoscopy images is crucial for detecting polyps and preventing CRC. However, challenges such as varying polyp sizes, blurred edges, and uneven brightness hinder segmentation accuracy. Leveraging artificial intelligence (AI) and robot-assisted surgery mechanisms can aid surgeons and physicians in detecting and treating polyps. To address these challenges, we propose a Colorectal Network (CR-Net), an AI-based encoder-decoder network for precise polyp and surgical instrument segmentation. CR-Net incorporates a pre-trained Visual Geometry Group model with 16 convolution layers (VGG16), attention mechanisms, redesigned skip connections, and horizontal dense connections within a U-Net architecture. The VGG16 encoder captures robust visual features, while redesigned skip connections accommodate complex data dimensions, leading to enhanced segmentation outcomes. Horizontal dense connections transfer overlooked features from the encoder to subsequent layers, further improving segmentation accuracy. Additionally, a spatial attention block enhances spatial features and ensures compatibility during upsampling. Evaluation of datasets including the Kvasir segmentation (Kvasir-SEG) dataset, Computer Vision Center Clinic Database (CVC-ClinicDB), Kvasir-Instrument dataset, and University of Washington Sinus Surgery Live (UW-Sinus-Surgery-Live) dataset demonstrates CR-Net's superior performance, achieving Dice Similarity Coefficients of 96.21%, 96.54%, 96.32%, and 92.84%, respectively, surpassing previous methods. These results highlight CR-Net's potential in empowering healthcare professionals through advanced AI-driven engineering applications. By bridging AI techniques with engineering innovations, CR-Net represents a significant advancement in CRC diagnosis and treatment.

FP4.2 - Does colonic endoscopic tattooing affect radiological staging?

Abstract Aim Evidence exists of radiological upstaging of rectal cancers due to fibrosis caused by endoscopic tattooing. We review investigated whether endoscopic tattooing for colonic lesions resulted in subsequent radiological upstaging. Methods A retrospective review was performed of patients endoscopically diagnosed with colonic cancer and undergoing surgery over three years at a single centre. This included patients who underwent both CT imaging prior to, and after colonoscopy with biopsy +/- tattoo. Radiological and operative staging was subsequently compared. Emergencies, those with no colonoscopy or pre-operative imaging and rectal cancers were excluded. Data were obtained from Somerset register and electronic care records. Data were analysed using chi squared test. Results A total of 175 patients were included. 49 patients had CT imaging prior to their colonoscopy, 51 patients had colonoscopy and biopsy prior to imaging; a further 75 patients had biopsy and tattoo prior to imaging. Patients with pre-endoscopy imaging were considered the control group, and patients with biopsy only were calculated separately as they were considered potential confounders (Table 1).Table 1:UpstagedDownstagedSame stagingPre-endoscopy imaging191713Biopsy only131622Tattoo and biopsy212727 Overall, there was no significant difference between groups (p=.429), although 7 reports specifically mentioned potential impact of recent biopsy and tattoo affecting image interpretation. Conclusion Our results do not suggest biopsy or tattooing alters pre-operative staging, although it is possible that radiologists may have evolved their interpretation to account for any potential differences.

Locally advanced rectal cancer in a young adult affected with dyskeratosis congenita (Zinsser–Cole–Engman syndrome): a case report

BackgroundDyskeratosis congenita (DKC), also known as Zinsser–Cole–Engman syndrome, is a progressive genetic disease with a triad of reticulate skin pigmentation, nail dystrophy, and leukoplakia. Approximately 8–10% of patients with DKC develop malignancies, and cases of colorectal cancer with DKC in young people have been reported previously.Case presentationA 25-year-old man with DKC since approximately 10 years of age developed fever and lower abdominal discomfort. Diagnostic imaging revealed locally advanced rectal cancer with lymph node metastasis, direct invasion of the prostate, and pelvic abscess due to tumor microperforation (cT4bN2M0 cStage IIIC). Biopsy showed well to moderately differentiated ductal adenocarcinoma. Genetic testing was negative for RAS and BRAF gene mutations, and microsatellite instability (MSI) testing was also negative. After sigmoid colostomy, the patient was treated with total neoadjuvant therapy (TNT) with systemic chemotherapy (six courses of FOLFOX + panitumumab) followed by chemoradiation therapy (50.4 Gy with capecitabine). After TNT, the primary tumor and metastatic lymph nodes shrank. According to the findings of colonoscopy and magnetic resonance image (MRI), we diagnosed near complete response (near-CR) and decided to follow the patient without surgery by every 3 months re-evaluation. However, 5 months after TNT, tumor regrowth was detected on colonoscopy and imaging, and the patient underwent total pelvic exenteration. He developed paralytic ileus as a postoperative complication, and was discharged on the 38th postoperative day. Pathological examination revealed a residual tumor with invasion of the periprostatic tissue. There was no metastasis in the pararectal and lateral pelvic lymph nodes, but one extramural non-contiguous cancerous extension (tumor deposit) was observed (ypT4bN1cM0 ypStage IIIC). The patient has been free of recurrence for one year after surgery.ConclusionsDKC often develops into various tumors in the digestive system at an early age; therefore, appropriate surveillance may be required. In addition, considering that cancers in patients with DKC occur at a young age, fertility preservation and survivorship are also important, and adequate explanations and care should be provided to patients before and after treatment.

PolySegNet: improving polyp segmentation through swin transformer and vision transformer fusion.

Colorectal cancer ranks as the second most prevalent cancer worldwide, with a high mortality rate. Colonoscopy stands as the preferred procedure for diagnosing colorectal cancer. Detecting polyps at an early stage is critical for effective prevention and diagnosis. However, challenges in colonoscopic procedures often lead medical practitioners to seek support from alternative techniques for timely polyp identification. Polyp segmentation emerges as a promising approach to identify polyps in colonoscopy images. In this paper, we propose an advanced method, PolySegNet, that leverages both Vision Transformer and Swin Transformer, coupled with a Convolutional Neural Network (CNN) decoder. The fusion of these models facilitates a comprehensive analysis of various modules in our proposed architecture.To assess the performance of PolySegNet, we evaluate it on three colonoscopy datasets, a combined dataset, and their augmented versions. The experimental results demonstrate that PolySegNet achieves competitive results in terms of polyp segmentation accuracy and efficacy, achieving a mean Dice score of 0.92 and a mean Intersection over Union (IoU) of 0.86. These metrics highlight the superior performance of PolySegNet in accurately delineating polyp boundaries compared to existing methods. PolySegNet has shown great promise in accurately and efficiently segmenting polyps in medical images. The proposed method could be the foundation for a new class of transformer-based segmentation models in medical image analysis.

AI support for colonoscopy quality control using CNN and transformer architectures

BackgroundConstruct deep learning models for colonoscopy quality control using different architectures and explore their decision-making mechanisms.MethodsA total of 4,189 colonoscopy images were collected from two medical centers, covering different levels of bowel cleanliness, the presence of polyps, and the cecum. Using these data, eight pre-trained models based on CNN and Transformer architectures underwent transfer learning and fine-tuning. The models’ performance was evaluated using metrics such as AUC, Precision, and F1 score. Perceptual hash functions were employed to detect image changes, enabling real-time monitoring of colonoscopy withdrawal speed. Model interpretability was analyzed using techniques such as Grad-CAM and SHAP. Finally, the best-performing model was converted to ONNX format and deployed on device terminals.ResultsThe EfficientNetB2 model outperformed other architectures on the validation set, achieving an accuracy of 0.992. It surpassed models based on other CNN and Transformer architectures. The model’s precision, recall, and F1 score were 0.991, 0.989, and 0.990, respectively. On the test set, the EfficientNetB2 model achieved an average AUC of 0.996, with a precision of 0.948 and a recall of 0.952. Interpretability analysis showed the specific image regions the model used for decision-making. The model was converted to ONNX format and deployed on device terminals, achieving an average inference speed of over 60 frames per second.ConclusionsThe AI-assisted quality system, based on the EfficientNetB2 model, integrates four key quality control indicators for colonoscopy. This integration enables medical institutions to comprehensively manage and enhance these indicators using a single model, showcasing promising potential for clinical applications.

A Hessian-Based Technique for Specular Reflection Detection and Inpainting in Colonoscopy Images.

In the field of Computer-Aided Detection (CADx), the use of AI-based algorithms for disease detection in endoscopy images, especially colonoscopy images, is on the rise. However, these algorithms often encounter performance issues due to obstructions like specular reflection, resulting in false positives. This paper presents a novel algorithm specifically designed to tackle the challenges posed by high specular reflection regions in colonoscopy images. The proposed algorithm identifies these regions and applies precise inpainting for restoration. The process entails converting the input image from RGB to HSV color space and focusing on the Saturation (S) component in convex regions detected using a Hessian-based method. This step creates a binary mask that pinpoints areas of specular reflection. The inpainting function then uses this mask to guide the restoration of these identified regions and their borders. To ensure a seamless blend of the restored regions with the background and adjacent pixels, a feathering process is applied to the repaired regions. This enhances both the accuracy and aesthetic coherence of the inpainted images. The performance of our algorithm was rigorously tested on five unique colonoscopy datasets and various endoscopy images from the Kvasir dataset, using an extensive set of evaluation metrics and a comparative analysis with existing methods consistently highlighted the superior performance of our algorithm.

Rethinking encoder-decoder architecture using vision transformer for colorectal polyp and surgical instruments segmentation

Accurate polyp segmentation from colonoscopy images is important for the immediate diagnosis and effective treatment of colon cancer. While significant progress has been made in the polyps segmentation task, there are various challenges that need to be addressed. Polyps can vary greatly in size and shape, and often has no clear boundary between surrounding tissues. Furthermore, surgical instrument segmentation can aid surgeons with the precise positioning and orientation of the instruments, helping them to plan the next steps in the robot-assisted surgery. The proposed colorectal polyp segmentation Transformer (CPS-Former) uses innovative attention blocks in a network that encodes-decodes features like classic Semantic Segmentation Network (SegNet). However, it has special self-attention modules with small convolutional kernels that efficiently extract information from different feature-channels. Moreover, it is equipped with an effective positional embedding to capture information from a large area of context for long distance interactions. Additionally, a fusion block is embedded for scaling-attention that combines the outputs from the encoder-decoder blocks to enhance the semantic features and reduce the non-semantic ones. Transformer encoder blocks also modified by adding a local feedforward layer and skips connections, and adjust the channel sizes to reduce the model trainable parameters. We evaluate our colorectal polyp segmentation network (CPS-Former) on four colorectal polyp public datasets and one surgical instrument segmentation dataset, which show its superiority over other state-of-the-art polyp segmentation models. Our implementation source code and network weights are available at GitHub: https://github.com/ahmedeqbal/CPS-Former.