Wireless Capsule Endoscopy Video Research Articles

A Deep Learning Application of Capsule Endoscopic Gastric Structure Recognition Based on a Transformer Model.

Gastric structure recognition systems have become increasingly necessary for the accurate diagnosis of gastric lesions in capsule endoscopy. Deep learning, especially using transformer models, has shown great potential in the recognition of gastrointestinal (GI) images according to self-attention. This study aims to establish an identification model of capsule endoscopy gastric structures to improve the clinical applicability of deep learning to endoscopic image recognition. A total of 3343 wireless capsule endoscopy videos collected at Nanfang Hospital between 2011 and 2021 were used for unsupervised pretraining, while 2433 were for training and 118 were for validation. Fifteen upper GI structures were selected for quantifying the examination quality. We also conducted a comparison of the classification performance between the artificial intelligence model and endoscopists by the accuracy, sensitivity, specificity, and positive and negative predictive values. The transformer-based AI model reached a relatively high level of diagnostic accuracy in gastric structure recognition. Regarding the performance of identifying 15 upper GI structures, the AI model achieved a macroaverage accuracy of 99.6% (95% CI: 99.5-99.7), a macroaverage sensitivity of 96.4% (95% CI: 95.3-97.5), and a macroaverage specificity of 99.8% (95% CI: 99.7-99.9) and achieved a high level of interobserver agreement with endoscopists. The transformer-based AI model can accurately evaluate the gastric structure information of capsule endoscopy with the same performance as that of endoscopists, which will provide tremendous help for doctors in making a diagnosis from a large number of images and improve the efficiency of examination.

P293 Safety and diagnostic yield of wireless video capsule endoscopy in early onset inflammatory bowel disease (EOIBD) in children under the age of 6 years in a tertiary paediatric IBD centre

Abstract Background Wireless video capsule endoscopy (VCE) is a well-established diagnostic tool used in the evaluation of small bowel disease in paediatric IBD. Patients usually swallow the capsule. Young children are rarely able to swallow the capsule and these therefore need to be placed with the help of an OGD and an ACORN device. 2 capsule systems were used: PillCam and MiroCam. The aim of this study was to look at the safety of capsule insertion in children less than 6 years of age and to also look at the diagnostic yield. Methods We conducted a retrospective review of our VCE data base over a 7 year period from April 2016 to August 2023. Results We conducted a retrospective review of our VCE data base over a 7 year period from April 2016 to August 2023. 22 patients (14 male) and 27 capsule episodes were included into the study with a an age range at point of capsule insertion between 2 years and 5 months and 5 years and 8 months (median 5 years). All capsules were inserted with an ACORN device. All patients had a histologically proven diagnosis of EOIBD; during this period 4 patients had 2 VCEs and 1 patient 3 VCEs placed. All patient had genetic tests performed, only 1/24 patients had a monogenic mutation confirmed. 15/27 (56%) of all capsules were normal, 8/27 (30%) showed small bowel ulcerations only, 2/27 (7%) showed both small bowel and colonic ulcerations and 5/27 (19%) showed colitis only. We were able to insert the capsule in all 27 capsule episodes without any complications. All studies were complete with the capsule entering the colon and no capsule got stuck. Conclusion Wireless video capsule endoscopy in early onset inflammatory bowel disease in children under the age of 6 years is safe with a diagnostic yield of around 50%. This modality should be used in the diagnostic pathway of EOIBD.

CoInNet: A Convolution-Involution Network With a Novel Statistical Attention for Automatic Polyp Segmentation.

Polyps are very common abnormalities in human gastrointestinal regions. Their early diagnosis may help in reducing the risk of colorectal cancer. Vision-based computer-aided diagnostic systems automatically identify polyp regions to assist surgeons in their removal. Due to their varying shape, color, size, texture, and unclear boundaries, polyp segmentation in images is a challenging problem. Existing deep learning segmentation models mostly rely on convolutional neural networks that have certain limitations in learning the diversity in visual patterns at different spatial locations. Further, they fail to capture inter-feature dependencies. Vision transformer models have also been deployed for polyp segmentation due to their powerful global feature extraction capabilities. But they too are supplemented by convolution layers for learning contextual local information. In the present paper, a polyp segmentation model CoInNet is proposed with a novel feature extraction mechanism that leverages the strengths of convolution and involution operations and learns to highlight polyp regions in images by considering the relationship between different feature maps through a statistical feature attention unit. To further aid the network in learning polyp boundaries, an anomaly boundary approximation module is introduced that uses recursively fed feature fusion to refine segmentation results. It is indeed remarkable that even tiny-sized polyps with only 0.01% of an image area can be precisely segmented by CoInNet. It is crucial for clinical applications, as small polyps can be easily overlooked even in the manual examination due to the voluminous size of wireless capsule endoscopy videos. CoInNet outperforms thirteen state-of-the-art methods on five benchmark polyp segmentation datasets.

Feature Extraction for Polyp Detection in Wireless Capsule Endoscopy Video Frames

Wireless capsule endoscopy (WCE) is a technology for filming the gastrointestinal (GI) tract to find abnormalities such as tumors, polyps, and bleeding. This paper proposes a new method based on hand-crafted features to detect polyps in WCE frames. A polyp has a convex surface containing pixel values with a specified Gaussian distribution. If a polyp exists in the WCE image, edges will be seen at the border of the occupied area. Since WCE images often suffer from low illumination, a histogram equalization (HE) technique can be used to enhance the image. In this paper, we initially find probable polyp edges via thresholding. Then, we use the edges to find the region of interest (ROI). Then, the mean, standard deviation (STD), and division of mean by STD from the ROI are computed as features to discriminate between polyp and nonpolyp using a support vector machine (SVM). The evaluation results on the Kvasir-Capsule dataset show 99% accuracy for the proposed method in polyp detection. Furthermore, the proposed method runs at a real-time speed of ∼0.031 seconds detection for each image.

Federated Deep Learning for Wireless Capsule Endoscopy Analysis: Enabling Collaboration Across Multiple Data Centers for Robust Learning of Diverse Pathologies

Wireless capsule endoscopy (WCE) is a revolutionary diagnostic method for small bowel pathology. The manual perusal of the resulting lengthy and redundant videos is cumbersome. Automated analysis of WCE video frames is an intricate data modeling task because of the diverse representations of anomalies caused by inappropriate capture conditions. Deep neural networks require training to learn diverse pathological manifestations utilizing heterogeneous data collected from multiple institutions. However, the accessibility of WCE data poses privacy concerns for multiple centers. The efficient learning of heterogeneous data distributed over multiple institutions in a privacy-preserving fashion has become a challenge hampering the adoption of AI-based diagnoses in clinical practice. Prior studies have contrived extensive data augmentation and the generation of synthetic images from the same center. However, models trained at one center are at risk of a lack of generalization for a global deployment. Federated learning (FL) is a novel paradigm in which models learn from distributed data and share knowledge without accessing the data themselves. This study proposes an FL framework for multiple anomaly classifications of WCE frames, elaborating on the potential of collaborative learning from multiple data centers on the edge. Our empirical results prove that the proposed decentralized approach can learn the generalized features of WCE frames. Validating heterogeneous test sets revealed a 10–12% improvement in performance for decentralized models based on FL compared to the best-case performance of centralized models, demonstrating the potential of the federated framework to support multiple anomaly classification of WCE frames while preserving data privacy across various clinical setups.

Video Analysis of Small Bowel Capsule Endoscopy Using a Transformer Network.

Although wireless capsule endoscopy (WCE) detects small bowel diseases effectively, it has some limitations. For example, the reading process can be time consuming due to the numerous images generated per case and the lesion detection accuracy may rely on the operators' skills and experiences. Hence, many researchers have recently developed deep-learning-based methods to address these limitations. However, they tend to select only a portion of the images from a given WCE video and analyze each image individually. In this study, we note that more information can be extracted from the unused frames and the temporal relations of sequential frames. Specifically, to increase the accuracy of lesion detection without depending on experts' frame selection skills, we suggest using whole video frames as the input to the deep learning system. Thus, we propose a new Transformer-architecture-based neural encoder that takes the entire video as the input, exploiting the power of the Transformer architecture to extract long-term global correlation within and between the input frames. Subsequently, we can capture the temporal context of the input frames and the attentional features within a frame. Tests on benchmark datasets of four WCE videos showed 95.1% sensitivity and 83.4% specificity. These results may significantly advance automated lesion detection techniques for WCE images.

Neutrophils to Lymphocytes Ratio and Platelets to Lymphocytes Ratio as Predictors of Small Intestinal Lesions Detected by Video Capsule Endoscopy

Abstract Background Capsule endoscopy (CE), also known as wireless capsule endoscopy or video capsule endoscopy, is a gastrointestinal study that uses a pill camera to take images of the intestinal lumen. Objective: Patients and Methods The study was conducted on 93 subjects from the Gastroenterology and Hepatology Department of Ain Shams University Hospital. They were divided into three different groups as following: Crohn’s disease group, arteriovenous malformation group and small intestinal tumor group. All patients and controls were subjected to routine laboratory tests and NLR and PLR was estimated. Results There was statistically significant difference between control group and AVM group and Crohn's group as regard N/L ratio, while this difference is less significant in tumor group patient. Receiver operating characteristic curve (ROC) shows that the best cut off point of N/L ratio to detect AVM group was found >1.97 with sensitivity of 67.74, specificity of 72.04% and total accuracy of 73%, and the best cut off point of P/L ratio to detect AVM group was found >158.02 with sensitivity of 61.29%, specificity of 96.77%, and total accuracy of 78%. Receiver operating characteristic curve (ROC) shows that the best cut off point of N/L ratio to detect Tumor group was found >1.97 with sensitivity of 45.16%, specificity of 72.04% and total accuracy of 59%, and Receiver operating characteristic curve (ROC) shows that the best cut off point of P/L ratio to detect Tumor group was found >145.83 with sensitivity of 32.26%, specificity of 91.40% and total accuracy of 62%. Receiver operating characteristic curve (ROC) shows that the best cut off point of N/L ratio to detect Crohn's group was found >2.44 with sensitivity of 100.0%, specificity of 84.95 and total accuracy of 96%, and Receiver operating characteristic curve (ROC) shows that the best cut off point of P/L ratio to detect Crohn's group was found >138.35 with sensitivity of 80.65%, specificity of 89.25% and total accuracy of 92%. Conclusion N/L ratio and P/L ratio can be used as non- invasive and cheap predictors for severity of crohn's disease, arteriovenous malformations and stage of malignancy in small intestinal tract.

De-redundancy in wireless capsule endoscopy video sequences using correspondence matching and motion analysis

De-redundancy in wireless capsule endoscopy video sequences using correspondence matching and motion analysis

Combining deep features and hand-crafted features for abnormality detection in WCE images

In this paper, a computer-aided method is proposed for abnormality detection in Wireless Capsule Endoscopy (WCE) video frames. Common abnormalities in WCE images include ulcers, bleeding, Angiodysplasia, Lymphoid Hyperplasia, and polyp. In this paper, deep features and Hand-crafted features are combined to detect these abnormalities in WCE images. There are not sufficient images available to train deep structures, therefore the ResNet50 pre-trained model is used to extract deep features. Hand-crafted features are associated with color, shape, and texture. We used a novel idea to reveal unexpected color changes in the background due to existing lesions as a color feature set. Histogram of gradient (HOG) and local binary pattern (LBP) were used respectively for shape and texture features. They are extracted from the region of interest (ROI), i.e. suspicious region. The expectation Maximization (EM) algorithm is used to extract more distinct areas in the background as ROI. The expectation Maximization (EM) algorithm is configured in a way that can extract areas with a distinct texture and color as ROI. The EM algorithm is also initialized with a new fast method which leads to an increase in the accuracy of the method. A large number of features are created by the method, so the minimum redundancy maximum relevance approach is used to select a subset of more effective features. These selected features are then fed to a Support Vector Machine for classification. The results show that the proposed approach can detect mentioned abnormalities in WCE frames with the accuracy of 97.82%

Machine Vision based Unsupervised Summarization of Wireless Capsule Endoscopy Video

Wireless capsule endoscopy is a medical procedure that uses a small, pill-sized capsule with a tiny camera inside to examine the inside of the digestive system. The capsule contains a camera that takes pictures as it moves through the digestive tract. The images are transmitted to a small recording device that the patient wears on a belt or shoulder strap. The recording device stores the images, which are later downloaded and examined by a doctor. The pill-sized camera records a video for eight to twelve hours. The purpose of this work is to investigate and put into practise a technique for summarising the capsule endoscopy footage. The method clusters the video frames in order to accomplish this by looking for local maxima in the neighbouring distance graph that is generated from the colour histograms of each video frame. The method then determines the centroid frame for each cluster and produces a key-frame for each cluster. Visualising the conclusion of this investigation is its second objective. This is accomplished by placing each cluster on a timeline according to the size and calibre of the keyframe. The summary of the capsule endoscopy video is generated as a sequence of keyframes. These keyframes are representative frames of the video. The work also proposes a technique to assess the quality and the quantity of the keyframes extracted.

Machine learning techniques to detect bleeding frame and area in wireless capsule endoscopy video

Wireless Capsule Endoscopy (WCE) allows direct visual inspecting of the full digestive system of the patient without invasion and pain, at the price of a long examination by physicians of a large number of photographs. This research presents a new approach to color extraction to differentiate bleeding frames from normal ones and locate more bleeding areas. We have a dual-system suggestion. We use entire color information on the WCE pictures and the pixel-represented clustering approach to get the clustered centers that characterize WCE pictures as words. Then we evaluate the status of a WCE framework using the nearby SVM and K methods (KNN). The classification performance is 95.75% accurate for the AUC 0.9771% and validates the exciting performance for bleeding classification provided by the suggested approach. Second, we present a two-step approach for extracting saliency maps to emphasize bleeding locations with a distinct color channel mixer to build a first-stage salience map. The second stage salience map was taken with optical contrast.We locate bleeding spots following a suitable fusion approach and threshold. Quantitative and qualitative studies demonstrate that our approaches can correctly distinguish bleeding sites from neighborhoods.

FIAS3: Frame Importance-Assisted Sparse Subset Selection to Summarize Wireless Capsule Endoscopy Videos

Wireless capsule endoscopy (WCE) is a recently developed tool that allows for the painless and non-invasive examination of the entire gastrointestinal (GI) tract. The microcamera captures a large number of redundant frames for each WCE examination such that a video summarization technique is needed to assist in diagnosis. However, prevalent methods of summarizing WCE videos focus only on the representativeness of the frames owing to a lack of high-level information on their importance. This paper develops a Frame Importance-Assisted Sparse Subset Selection model, called FIAS3, to integrate the high-level frame importance from networks into a sparse subset selection model. The FIAS3 is optimized under three constraints: 1) a frame importance matrix to help pay more attention to important frames, 2) a sparsity constraint to make video summaries more compact, and 3) a similarity-inhibiting constraint to reduce redundancy. The results of experiments on a public dataset demonstrated that our FIAS3 outperforms other methods of summarizing WCE videos. Specifically, its coverage and video reconstruction error were 92% and 0.143, respectively, at a 90% compression ratio, recording respective at least 16.9% and 0.031 improvements over other methods. The results of generalization experiments showed that FIAS3 also achieves competitive results on private datasets.

Wireless Capsule Endoscopy Video Summarization using Transfer Learning and Random Forests

Wireless Capsule Endoscopy Video Summarization using Transfer Learning and Random Forests

Video Super-Resolution for Wireless Capsule Endoscopy Imaging Sensor

As a preferred diagnostic imaging sensor for small bowel diseases, wireless capsule endoscopy (WCE) suffers from poor video quality, which can be mitigated via video super-resolution (VSR) techniques. However, the promise of applying VSR into WCE applications has been offset by two significant challenges. Firstly, high-resolution (HR) counterparts of real endoscopic frames are unavailable for supervised learning. Secondly, utilizing temporal correlations to enhance recovery performance becomes difficult due to the poor video quality. To tackle these two challenges, we develop a novel training dataset and an effective alignment network for VSR on practical WCE applications. Specifically, clean natural videos and synthetic endoscopic videos are collected as the ground truth of the training dataset. Then, a set of complex degradation models is designed to generate their low-resolution (LR) counterparts for supervised learning. Although real HR endoscopic videos remain unknown during training, this complicated dataset enables the trained network to have a strong generalization capacity so that it can make reasonable inferences on real endoscopic videos. To utilize temporal correlations from WCE videos, a block-based temporal attention alignment network (BTAAN) is proposed to build accurate and robust correspondences among consecutive frames under poor video quality. This network exploits the global information from neighboring frames to capture large-scale motions, and suppresses the effect of unreliable points by a temporal attention mechanism. Moreover, its block-wise characteristic reduces the computational complexity significantly, which makes it hardware-friendly. Extensive experiments demonstrate the superiority of the proposed method over other state-of-the-art approaches on practical WCE applications.

Deep chroma prediction of Wyner–Ziv frames in distributed video coding of wireless capsule endoscopy video

Compression of captured video frames is crucial for saving the power in wireless capsule endoscopy (WCE). A low complexity encoder is desired to limit the power consumption required for compressing the WCE video. Distributed video coding (DVC) technique is best suitable for designing a low complexity encoder. In this technique, frames captured in RGB colour space are converted into YCbCr colour space. Both Y and CbCr representing luma and chroma components of the Wyner–Ziv (WZ) frames are processed and encoded in existing DVC techniques proposed for WCE video compression. In the WCE video, consecutive frames exhibit more similarity in texture and colour properties. The proposed work uses these properties to present a method for processing and encoding only the luma component of a WZ frame. The chroma components of the WZ frame are predicted by an encoder–decoder based deep chroma prediction model at the decoder by matching luma and texture information of the keyframe and WZ frame. The proposed method reduces the computations required for encoding and transmitting of WZ chroma component. The results show that the proposed DVC with a deep chroma prediction model performs better when compared to motion JPEG and existing DVC systems for WCE at the reduced encoder complexity.

Capsule endoscopy in gastrointestinal disease: Evaluation, diagnosis, and treatment.

Capsule endoscopy, also known as wireless capsule endoscopy or video capsule endoscopy, is a noninvasive procedure that uses a swallowed capsule-shaped miniature camera for direct visual and diagnostic evaluation of gastrointestinal (GI) disease. Although originally intended as a tool to examine the small intestine, which is mostly beyond the reach of conventional endoscopy, capsule endoscopy is now also being used to examine the entire length of the GI tract.

Sequential Models for Endoluminal Image Classification.

Wireless Capsule Endoscopy (WCE) is a procedure to examine the human digestive system for potential mucosal polyps, tumours, or bleedings using an encapsulated camera. This work focuses on polyp detection within WCE videos through Machine Learning. When using Machine Learning in the medical field, scarce and unbalanced datasets often make it hard to receive a satisfying performance. We claim that using Sequential Models in order to take the temporal nature of the data into account improves the performance of previous approaches. Thus, we present a bidirectional Long Short-Term Memory Network (BLSTM), a sequential network that is particularly designed for temporal data. We find the BLSTM Network outperforms non-sequential architectures and other previous models, receiving a final Area under the Curve of . Experiments show that our method of extracting spatial and temporal features yields better performance and could be a possible method to decrease the time needed by physicians to analyse the video material.

RAt-CapsNet: A Deep Learning Network Utilizing Attention and Regional Information for Abnormality Detection in Wireless Capsule Endoscopy.

Background: The emergence of wireless capsule endoscopy (WCE) has presented a viable non-invasive mean of identifying gastrointestinal diseases in the field of clinical gastroenterology. However, to overcome its extended time of manual inspection, a computer aided automatic detection system is getting vast popularity. In this case, major challenges are low resolution and lack of regional context in images extracted from WCE videos. Methods: For tackling these challenges, in this paper a convolution neural network (CNN) based architecture, namely RAt-CapsNet, is proposed that reliably employs regional information and attention mechanism to classify abnormalities from WCE video data. The proposed RAt-CapsNet consists of two major pipelines: Compression Pipeline and Regional Correlative Pipeline. In the compression pipeline, an encoder module is designed using a Volumetric Attention Mechanism which provides 3D enhancement to feature maps using spatial domain condensation as well as channel-wise filtering for preserving relevant structural information of images. On the other hand, the regional correlative pipeline consists of Pyramid Feature Extractor which operates on image driven feature vectors to generalize and propagate local relationships of pixels from WCE abnormalities with respect to the normal healthy surrounding. The feature vectors generated by the pipelines are then accumulated to formulate a classification standpoint. Results: Promising computational accuracy of mean 98.51% in binary class and over 95.65% in multi-class are obtained through extensive experimentation on a highly unbalanced public dataset with over 47 thousand labelled. Conclusion: This outcome in turn supports the efficacy of the proposed methodology as a noteworthy WCE abnormality detection as well as diagnostic system.

A deep CNN model for anomaly detection and localization in wireless capsule endoscopy images

Wireless capsule endoscopy (WCE) is one of the most efficient methods for the examination of gastrointestinal tracts. Computer-aided intelligent diagnostic tools alleviate the challenges faced during manual inspection of long WCE videos. Several approaches have been proposed in the literature for the automatic detection and localization of anomalies in WCE images. Some of them focus on specific anomalies such as bleeding, polyp, lesion, etc. However, relatively fewer generic methods have been proposed to detect all those common anomalies simultaneously. In this paper, a deep convolutional neural network (CNN) based model ‘WCENet’ is proposed for anomaly detection and localization in WCE images. The model works in two phases. In the first phase, a simple and efficient attention-based CNN classifies an image into one of the four categories: polyp, vascular, inflammatory, or normal. If the image is classified in one of the abnormal categories, it is processed in the second phase for the anomaly localization. Fusion of Grad-CAM++ and a custom SegNet is used for anomalous region segmentation in the abnormal image. WCENet classifier attains accuracy and area under receiver operating characteristic of 98% and 99%. The WCENet segmentation model obtains a frequency weighted intersection over union of 81%, and an average dice score of 56% on the KID dataset. WCENet outperforms nine different state-of-the-art conventional machine learning and deep learning models on the KID dataset. The proposed model demonstrates potential for clinical applications.

An intelligent compression system for wireless capsule endoscopy images

In this paper, we propose an intelligent compression system that addresses the problems of energy limitations of wireless video capsule endoscopy. The principle is to include a classification feedback loop, based on deep learning, to determine the importance of the images being transmitted. This classification is used with a simple prediction-based compression algorithm to allow an intelligent management of the limited energy of the capsule. For this, the capsule starts by transmitting a subsampled version of each image with a small rate. The images will be decoded and classified, automatically, to detect any possible lesions. Following the classification result, the images considered as important, for diagnosis, will be enhanced with additional content, whereas the less important ones will be recorded with low quality. In this way, large amounts of bits will be saved, without affecting the diagnosis. The saved energy can be used to extend the life of the capsule or to increase the resolution and frame rate of some WCE images. The results of classification show an accuracy of more than 99%, which allowed us to code losslessly almost all the important images of our test sequences. Our results also show that many additional images can be transmitted and their number depend on the used subsampling and the number of important images.