Wireless Capsule Endoscopy Video Research Articles

Feature Pyramid Network Based Spatial Attention and Cross‐Level Semantic Similarity for Diseases Segmentation From Capsule Endoscopy Images

ABSTRACTAs an emerging technology that uses a pill‐sized camera to visualize images of the digestive tract. Wireless capsule endoscopy (WCE) presents several advantages, since it is far less invasive, does not need sedation and has less possible complications compared to standard endoscopy. Hence, it might be exploited as alternative to the standard procedure. WCE is used to diagnosis a variety of gastro‐intestinal diseases such as polyps, ulcers, crohns disease, and hemorrhages. Nevertheless, WCE videos produced after a test may consist of thousands of frames per patient that must be viewed by medical specialists, besides, the capsule free mobility and technological limits cause production of a low quality images. Hence, development of an automatic tool based on artificial intelligence might be very helpful. Moreover, most state‐of‐the‐art works aim at images classification (normal/abnormal) while ignoring diseases segmentation. Therefore, in this work a novel method based on Feature Pyramid Network model is presented. This approach aims at diseases segmentation from WCE images. In this model, modules to optimize and combine features were employed. Specifically, semantic and spatial features were mutually compensated by spatial attention and cross‐level global feature fusion modules. The proposed method testing F1‐score and mean intersection over union are 94.149% and 89.414%, respectively, in the MICCAI 2017 dataset. In the KID Atlas dataset, the method achieved a testing F1‐score and mean intersection over union of 94.557% and 90.416%, respectively. Through the performance analysis, the mean intersection over union in the MICCAI 2017 dataset is 20.414%, 18.484%, 11.444%, 8.794% more than existing approaches. Moreover, the proposed scheme surpassed the methods used for comparison by 29.986% and 9.416% in terms of mean intersection over union in KID Atlas dataset. These results indicate that the proposed approach is promising in diseases segmentation from WCE images.

A Deep Learning Application of Capsule Endoscopic Gastric Structure Recognition Based on a Transformer Model.

Gastric structure recognition systems have become increasingly necessary for the accurate diagnosis of gastric lesions in capsule endoscopy. Deep learning, especially using transformer models, has shown great potential in the recognition of gastrointestinal (GI) images according to self-attention. This study aims to establish an identification model of capsule endoscopy gastric structures to improve the clinical applicability of deep learning to endoscopic image recognition. A total of 3343 wireless capsule endoscopy videos collected at Nanfang Hospital between 2011 and 2021 were used for unsupervised pretraining, while 2433 were for training and 118 were for validation. Fifteen upper GI structures were selected for quantifying the examination quality. We also conducted a comparison of the classification performance between the artificial intelligence model and endoscopists by the accuracy, sensitivity, specificity, and positive and negative predictive values. The transformer-based AI model reached a relatively high level of diagnostic accuracy in gastric structure recognition. Regarding the performance of identifying 15 upper GI structures, the AI model achieved a macroaverage accuracy of 99.6% (95% CI: 99.5-99.7), a macroaverage sensitivity of 96.4% (95% CI: 95.3-97.5), and a macroaverage specificity of 99.8% (95% CI: 99.7-99.9) and achieved a high level of interobserver agreement with endoscopists. The transformer-based AI model can accurately evaluate the gastric structure information of capsule endoscopy with the same performance as that of endoscopists, which will provide tremendous help for doctors in making a diagnosis from a large number of images and improve the efficiency of examination.

Deep Learning-Based Real-Time Organ Localization and Transit Time Estimation in Wireless Capsule Endoscopy.

Wireless capsule endoscopy (WCE) has significantly advanced the diagnosis of gastrointestinal (GI) diseases by allowing for the non-invasive visualization of the entire small intestine. However, machine learning-based methods for organ classification in WCE often rely on color information, leading to decreased performance when obstacles such as food debris are present. This study proposes a novel model that integrates convolutional neural networks (CNNs) and long short-term memory (LSTM) networks to analyze multiple frames and incorporate temporal information, ensuring that it performs well even when visual information is limited. We collected data from 126 patients using PillCam™ SB3 (Medtronic, Minneapolis, MN, USA), which comprised 2,395,932 images. Our deep learning model was trained to identify organs (stomach, small intestine, and colon) using data from 44 training and 10 validation cases. We applied calibration using a Gaussian filter to enhance the accuracy of detecting organ boundaries. Additionally, we estimated the transit time of the capsule in the gastric and small intestine regions using a combination of a convolutional neural network (CNN) and a long short-term memory (LSTM) designed to be aware of the sequence information of continuous videos. Finally, we evaluated the model's performance using WCE videos from 72 patients. Our model demonstrated high performance in organ classification, achieving an accuracy, sensitivity, and specificity of over 95% for each organ (stomach, small intestine, and colon), with an overall accuracy and F1-score of 97.1%. The Matthews Correlation Coefficient (MCC) and Geometric Mean (G-mean) were used to evaluate the model's performance on imbalanced datasets, achieving MCC values of 0.93 for the stomach, 0.91 for the small intestine, and 0.94 for the colon, and G-mean values of 0.96 for the stomach, 0.95 for the small intestine, and 0.97 for the colon. Regarding the estimation of gastric and small intestine transit times, the mean time differences between the model predictions and ground truth were 4.3 ± 9.7 min for the stomach and 24.7 ± 33.8 min for the small intestine. Notably, the model's predictions for gastric transit times were within 15 min of the ground truth for 95.8% of the test dataset (69 out of 72 cases). The proposed model shows overall superior performance compared to a model using only CNN. The combination of CNN and LSTM proves to be both accurate and clinically effective for organ classification and transit time estimation in WCE. Our model's ability to integrate temporal information allows it to maintain high performance even in challenging conditions where color information alone is insufficient. Including MCC and G-mean metrics further validates the robustness of our approach in handling imbalanced datasets. These findings suggest that the proposed method can significantly improve the diagnostic accuracy and efficiency of WCE, making it a valuable tool in clinical practice for diagnosing and managing GI diseases.

P293 Safety and diagnostic yield of wireless video capsule endoscopy in early onset inflammatory bowel disease (EOIBD) in children under the age of 6 years in a tertiary paediatric IBD centre

Abstract Background Wireless video capsule endoscopy (VCE) is a well-established diagnostic tool used in the evaluation of small bowel disease in paediatric IBD. Patients usually swallow the capsule. Young children are rarely able to swallow the capsule and these therefore need to be placed with the help of an OGD and an ACORN device. 2 capsule systems were used: PillCam and MiroCam. The aim of this study was to look at the safety of capsule insertion in children less than 6 years of age and to also look at the diagnostic yield. Methods We conducted a retrospective review of our VCE data base over a 7 year period from April 2016 to August 2023. Results We conducted a retrospective review of our VCE data base over a 7 year period from April 2016 to August 2023. 22 patients (14 male) and 27 capsule episodes were included into the study with a an age range at point of capsule insertion between 2 years and 5 months and 5 years and 8 months (median 5 years). All capsules were inserted with an ACORN device. All patients had a histologically proven diagnosis of EOIBD; during this period 4 patients had 2 VCEs and 1 patient 3 VCEs placed. All patient had genetic tests performed, only 1/24 patients had a monogenic mutation confirmed. 15/27 (56%) of all capsules were normal, 8/27 (30%) showed small bowel ulcerations only, 2/27 (7%) showed both small bowel and colonic ulcerations and 5/27 (19%) showed colitis only. We were able to insert the capsule in all 27 capsule episodes without any complications. All studies were complete with the capsule entering the colon and no capsule got stuck. Conclusion Wireless video capsule endoscopy in early onset inflammatory bowel disease in children under the age of 6 years is safe with a diagnostic yield of around 50%. This modality should be used in the diagnostic pathway of EOIBD.

Evaluation of Deep Learning Models in Search by Example using Capsule Endoscopy Images

Wireless capsule endoscopy (WCE) has revolutionized the field of gastrointestinal examinations, being MedtronicTM WCE one of the most used in clinics. In those WCE videos, medical experts use RAPID READERTM tool to annotate findings in videos. However, the frame annotations are not available in an open format and, when exported, they have different resolutions and some annotated artefacts that make difficult their localization in the original videos. This difficult the use of WCE medical experts’ annotations in the research of new computed-aid diagnostic (CAD) methods. In this paper, we propose a methodology to compare image similarities and evaluate it in a private MedtronicTM WCE SB3 video dataset to automatically identify the annotated frames in the videos. We used state-of-the-art pre-trained convolutional neural network (CNN) models, including MobileNet, InceptionResNetv2, ResNet50v2, VGG19, VGG16, ResNet101v2, ResNet152v2, and DenseNet121, as frame features extractors and compared them with the Euclidean distance. We evaluated the methodology performance on a private dataset consisting of 100 WCE videos, totalling 905 frames. The experimental results showed promising performance. The MobileNet model achieved an accuracy of 94% for identifying the first match, while the top 5, top 10, and top 20 matches were identified with accuracies of 94%, 94%, and 98%, respectively. The VGG16 and ResNet50v2 models also demonstrated strong performance, achieving accuracies ranging from 88% to 93% for various match positions. These results highlight the effectiveness of our proposed methodology in localizing target frames and even identifying similar frames very use useful for training data-driven models in CAD research. The code utilized in this experiment is available on the Github†.

Evaluation of Deep Learning Models in Search by Example using Capsule Endoscopy Images

Wireless capsule endoscopy (WCE) has revolutionized the field of gastrointestinal examinations, being MedtronicTM WCE one of the most used in clinics. In those WCE videos, medical experts use RAPID READERTM tool to annotate findings in videos. However, the frame annotations are not available in an open format and, when exported, they have different resolutions and some annotated artefacts that make difficult their localization in the original videos. This difficult the use of WCE medical experts’ annotations in the research of new computed-aid diagnostic (CAD) methods. In this paper, we propose a methodology to compare image similarities and evaluate it in a private MedtronicTM WCE SB3 video dataset to automatically identify the annotated frames in the videos. We used state-of-the-art pre-trained convolutional neural network (CNN) models, including MobileNet, InceptionResNetv2, ResNet50v2, VGG19, VGG16, ResNet101v2, ResNet152v2, and DenseNet121, as frame features extractors and compared them with the Euclidean distance. We evaluated the methodology performance on a private dataset consisting of 100 WCE videos, totalling 905 frames. The experimental results showed promising performance. The MobileNet model achieved an accuracy of 94% for identifying the first match, while the top 5, top 10, and top 20 matches were identified with accuracies of 94%, 94%, and 98%, respectively. The VGG16 and ResNet50v2 models also demonstrated strong performance, achieving accuracies ranging from 88% to 93% for various match positions. These results highlight the effectiveness of our proposed methodology in localizing target frames and even identifying similar frames very use useful for training data-driven models in CAD research. The code utilized in this experiment is available on the Github†.

Deep convolutional neural networks for filtering out normal frames in reviewing wireless capsule endoscopy videos

Wireless capsule endoscopy (WCE) has emerged as a valuable non-invasive technique for visualizing the entire gastrointestinal (GI) tract. However, manual evaluation of WCE videos is a time-consuming and costly process. In this study, we present a novel diagnostic assistant system that employs deep convolutional neural networks (DCNNs) to accelerate the evaluation process. Our primary objective is to achieve a high negative predictive value (NPV), which is essential for the efficient identification of normal frames. Six distinct DCNN models were developed and implemented with this objective in mind. The models were trained on a limited dataset encompassing common GI pathologies that reflect real clinical scenarios. Each DCNN architecture comprises a convolutional part derived from renowned pre-trained networks and a custom-designed classifier block optimized for high NPV and classification accuracy. Following a comprehensive assessment utilizing the 5-fold cross-validation approach, the VG_BFCG model was identified as the most effective, exhibiting an average test accuracy of 0.946 and an NPV of 0.983. Moreover, in the event of encountering novel pathologies not present in the training data, our models exhibited robustness in NPV, which is of great importance for practical applications. For example, the DN_BFCG model demonstrated consistent performance, with an NPV exceeding 0.99 across a range of new pathologies. This validates the reliability of our models in clinical settings. Our findings suggest that our developed DCNN architectures have the potential to enhance the efficiency and accuracy of WCE video analysis, which could transform the landscape of gastroenterological diagnostics.

CoInNet: A Convolution-Involution Network With a Novel Statistical Attention for Automatic Polyp Segmentation.

Polyps are very common abnormalities in human gastrointestinal regions. Their early diagnosis may help in reducing the risk of colorectal cancer. Vision-based computer-aided diagnostic systems automatically identify polyp regions to assist surgeons in their removal. Due to their varying shape, color, size, texture, and unclear boundaries, polyp segmentation in images is a challenging problem. Existing deep learning segmentation models mostly rely on convolutional neural networks that have certain limitations in learning the diversity in visual patterns at different spatial locations. Further, they fail to capture inter-feature dependencies. Vision transformer models have also been deployed for polyp segmentation due to their powerful global feature extraction capabilities. But they too are supplemented by convolution layers for learning contextual local information. In the present paper, a polyp segmentation model CoInNet is proposed with a novel feature extraction mechanism that leverages the strengths of convolution and involution operations and learns to highlight polyp regions in images by considering the relationship between different feature maps through a statistical feature attention unit. To further aid the network in learning polyp boundaries, an anomaly boundary approximation module is introduced that uses recursively fed feature fusion to refine segmentation results. It is indeed remarkable that even tiny-sized polyps with only 0.01% of an image area can be precisely segmented by CoInNet. It is crucial for clinical applications, as small polyps can be easily overlooked even in the manual examination due to the voluminous size of wireless capsule endoscopy videos. CoInNet outperforms thirteen state-of-the-art methods on five benchmark polyp segmentation datasets.

Feature Extraction for Polyp Detection in Wireless Capsule Endoscopy Video Frames

Wireless capsule endoscopy (WCE) is a technology for filming the gastrointestinal (GI) tract to find abnormalities such as tumors, polyps, and bleeding. This paper proposes a new method based on hand-crafted features to detect polyps in WCE frames. A polyp has a convex surface containing pixel values with a specified Gaussian distribution. If a polyp exists in the WCE image, edges will be seen at the border of the occupied area. Since WCE images often suffer from low illumination, a histogram equalization (HE) technique can be used to enhance the image. In this paper, we initially find probable polyp edges via thresholding. Then, we use the edges to find the region of interest (ROI). Then, the mean, standard deviation (STD), and division of mean by STD from the ROI are computed as features to discriminate between polyp and nonpolyp using a support vector machine (SVM). The evaluation results on the Kvasir-Capsule dataset show 99% accuracy for the proposed method in polyp detection. Furthermore, the proposed method runs at a real-time speed of ∼0.031 seconds detection for each image.

Federated Deep Learning for Wireless Capsule Endoscopy Analysis: Enabling Collaboration Across Multiple Data Centers for Robust Learning of Diverse Pathologies

Wireless capsule endoscopy (WCE) is a revolutionary diagnostic method for small bowel pathology. The manual perusal of the resulting lengthy and redundant videos is cumbersome. Automated analysis of WCE video frames is an intricate data modeling task because of the diverse representations of anomalies caused by inappropriate capture conditions. Deep neural networks require training to learn diverse pathological manifestations utilizing heterogeneous data collected from multiple institutions. However, the accessibility of WCE data poses privacy concerns for multiple centers. The efficient learning of heterogeneous data distributed over multiple institutions in a privacy-preserving fashion has become a challenge hampering the adoption of AI-based diagnoses in clinical practice. Prior studies have contrived extensive data augmentation and the generation of synthetic images from the same center. However, models trained at one center are at risk of a lack of generalization for a global deployment. Federated learning (FL) is a novel paradigm in which models learn from distributed data and share knowledge without accessing the data themselves. This study proposes an FL framework for multiple anomaly classifications of WCE frames, elaborating on the potential of collaborative learning from multiple data centers on the edge. Our empirical results prove that the proposed decentralized approach can learn the generalized features of WCE frames. Validating heterogeneous test sets revealed a 10–12% improvement in performance for decentralized models based on FL compared to the best-case performance of centralized models, demonstrating the potential of the federated framework to support multiple anomaly classification of WCE frames while preserving data privacy across various clinical setups.

Video Analysis of Small Bowel Capsule Endoscopy Using a Transformer Network.

Although wireless capsule endoscopy (WCE) detects small bowel diseases effectively, it has some limitations. For example, the reading process can be time consuming due to the numerous images generated per case and the lesion detection accuracy may rely on the operators' skills and experiences. Hence, many researchers have recently developed deep-learning-based methods to address these limitations. However, they tend to select only a portion of the images from a given WCE video and analyze each image individually. In this study, we note that more information can be extracted from the unused frames and the temporal relations of sequential frames. Specifically, to increase the accuracy of lesion detection without depending on experts' frame selection skills, we suggest using whole video frames as the input to the deep learning system. Thus, we propose a new Transformer-architecture-based neural encoder that takes the entire video as the input, exploiting the power of the Transformer architecture to extract long-term global correlation within and between the input frames. Subsequently, we can capture the temporal context of the input frames and the attentional features within a frame. Tests on benchmark datasets of four WCE videos showed 95.1% sensitivity and 83.4% specificity. These results may significantly advance automated lesion detection techniques for WCE images.

Neutrophils to Lymphocytes Ratio and Platelets to Lymphocytes Ratio as Predictors of Small Intestinal Lesions Detected by Video Capsule Endoscopy

Abstract Background Capsule endoscopy (CE), also known as wireless capsule endoscopy or video capsule endoscopy, is a gastrointestinal study that uses a pill camera to take images of the intestinal lumen. Objective: Patients and Methods The study was conducted on 93 subjects from the Gastroenterology and Hepatology Department of Ain Shams University Hospital. They were divided into three different groups as following: Crohn’s disease group, arteriovenous malformation group and small intestinal tumor group. All patients and controls were subjected to routine laboratory tests and NLR and PLR was estimated. Results There was statistically significant difference between control group and AVM group and Crohn's group as regard N/L ratio, while this difference is less significant in tumor group patient. Receiver operating characteristic curve (ROC) shows that the best cut off point of N/L ratio to detect AVM group was found >1.97 with sensitivity of 67.74, specificity of 72.04% and total accuracy of 73%, and the best cut off point of P/L ratio to detect AVM group was found >158.02 with sensitivity of 61.29%, specificity of 96.77%, and total accuracy of 78%. Receiver operating characteristic curve (ROC) shows that the best cut off point of N/L ratio to detect Tumor group was found >1.97 with sensitivity of 45.16%, specificity of 72.04% and total accuracy of 59%, and Receiver operating characteristic curve (ROC) shows that the best cut off point of P/L ratio to detect Tumor group was found >145.83 with sensitivity of 32.26%, specificity of 91.40% and total accuracy of 62%. Receiver operating characteristic curve (ROC) shows that the best cut off point of N/L ratio to detect Crohn's group was found >2.44 with sensitivity of 100.0%, specificity of 84.95 and total accuracy of 96%, and Receiver operating characteristic curve (ROC) shows that the best cut off point of P/L ratio to detect Crohn's group was found >138.35 with sensitivity of 80.65%, specificity of 89.25% and total accuracy of 92%. Conclusion N/L ratio and P/L ratio can be used as non- invasive and cheap predictors for severity of crohn's disease, arteriovenous malformations and stage of malignancy in small intestinal tract.

De-redundancy in wireless capsule endoscopy video sequences using correspondence matching and motion analysis

De-redundancy in wireless capsule endoscopy video sequences using correspondence matching and motion analysis

Combining deep features and hand-crafted features for abnormality detection in WCE images

In this paper, a computer-aided method is proposed for abnormality detection in Wireless Capsule Endoscopy (WCE) video frames. Common abnormalities in WCE images include ulcers, bleeding, Angiodysplasia, Lymphoid Hyperplasia, and polyp. In this paper, deep features and Hand-crafted features are combined to detect these abnormalities in WCE images. There are not sufficient images available to train deep structures, therefore the ResNet50 pre-trained model is used to extract deep features. Hand-crafted features are associated with color, shape, and texture. We used a novel idea to reveal unexpected color changes in the background due to existing lesions as a color feature set. Histogram of gradient (HOG) and local binary pattern (LBP) were used respectively for shape and texture features. They are extracted from the region of interest (ROI), i.e. suspicious region. The expectation Maximization (EM) algorithm is used to extract more distinct areas in the background as ROI. The expectation Maximization (EM) algorithm is configured in a way that can extract areas with a distinct texture and color as ROI. The EM algorithm is also initialized with a new fast method which leads to an increase in the accuracy of the method. A large number of features are created by the method, so the minimum redundancy maximum relevance approach is used to select a subset of more effective features. These selected features are then fed to a Support Vector Machine for classification. The results show that the proposed approach can detect mentioned abnormalities in WCE frames with the accuracy of 97.82%

Machine Vision based Unsupervised Summarization of Wireless Capsule Endoscopy Video

Wireless capsule endoscopy is a medical procedure that uses a small, pill-sized capsule with a tiny camera inside to examine the inside of the digestive system. The capsule contains a camera that takes pictures as it moves through the digestive tract. The images are transmitted to a small recording device that the patient wears on a belt or shoulder strap. The recording device stores the images, which are later downloaded and examined by a doctor. The pill-sized camera records a video for eight to twelve hours. The purpose of this work is to investigate and put into practise a technique for summarising the capsule endoscopy footage. The method clusters the video frames in order to accomplish this by looking for local maxima in the neighbouring distance graph that is generated from the colour histograms of each video frame. The method then determines the centroid frame for each cluster and produces a key-frame for each cluster. Visualising the conclusion of this investigation is its second objective. This is accomplished by placing each cluster on a timeline according to the size and calibre of the keyframe. The summary of the capsule endoscopy video is generated as a sequence of keyframes. These keyframes are representative frames of the video. The work also proposes a technique to assess the quality and the quantity of the keyframes extracted.

Machine learning techniques to detect bleeding frame and area in wireless capsule endoscopy video

Wireless Capsule Endoscopy (WCE) allows direct visual inspecting of the full digestive system of the patient without invasion and pain, at the price of a long examination by physicians of a large number of photographs. This research presents a new approach to color extraction to differentiate bleeding frames from normal ones and locate more bleeding areas. We have a dual-system suggestion. We use entire color information on the WCE pictures and the pixel-represented clustering approach to get the clustered centers that characterize WCE pictures as words. Then we evaluate the status of a WCE framework using the nearby SVM and K methods (KNN). The classification performance is 95.75% accurate for the AUC 0.9771% and validates the exciting performance for bleeding classification provided by the suggested approach. Second, we present a two-step approach for extracting saliency maps to emphasize bleeding locations with a distinct color channel mixer to build a first-stage salience map. The second stage salience map was taken with optical contrast.We locate bleeding spots following a suitable fusion approach and threshold. Quantitative and qualitative studies demonstrate that our approaches can correctly distinguish bleeding sites from neighborhoods.

FIAS3: Frame Importance-Assisted Sparse Subset Selection to Summarize Wireless Capsule Endoscopy Videos

Wireless capsule endoscopy (WCE) is a recently developed tool that allows for the painless and non-invasive examination of the entire gastrointestinal (GI) tract. The microcamera captures a large number of redundant frames for each WCE examination such that a video summarization technique is needed to assist in diagnosis. However, prevalent methods of summarizing WCE videos focus only on the representativeness of the frames owing to a lack of high-level information on their importance. This paper develops a Frame Importance-Assisted Sparse Subset Selection model, called FIAS3, to integrate the high-level frame importance from networks into a sparse subset selection model. The FIAS3 is optimized under three constraints: 1) a frame importance matrix to help pay more attention to important frames, 2) a sparsity constraint to make video summaries more compact, and 3) a similarity-inhibiting constraint to reduce redundancy. The results of experiments on a public dataset demonstrated that our FIAS3 outperforms other methods of summarizing WCE videos. Specifically, its coverage and video reconstruction error were 92% and 0.143, respectively, at a 90% compression ratio, recording respective at least 16.9% and 0.031 improvements over other methods. The results of generalization experiments showed that FIAS3 also achieves competitive results on private datasets.

Wireless Capsule Endoscopy Video Summarization using Transfer Learning and Random Forests

Wireless Capsule Endoscopy Video Summarization using Transfer Learning and Random Forests

Video Super-Resolution for Wireless Capsule Endoscopy Imaging Sensor

As a preferred diagnostic imaging sensor for small bowel diseases, wireless capsule endoscopy (WCE) suffers from poor video quality, which can be mitigated via video super-resolution (VSR) techniques. However, the promise of applying VSR into WCE applications has been offset by two significant challenges. Firstly, high-resolution (HR) counterparts of real endoscopic frames are unavailable for supervised learning. Secondly, utilizing temporal correlations to enhance recovery performance becomes difficult due to the poor video quality. To tackle these two challenges, we develop a novel training dataset and an effective alignment network for VSR on practical WCE applications. Specifically, clean natural videos and synthetic endoscopic videos are collected as the ground truth of the training dataset. Then, a set of complex degradation models is designed to generate their low-resolution (LR) counterparts for supervised learning. Although real HR endoscopic videos remain unknown during training, this complicated dataset enables the trained network to have a strong generalization capacity so that it can make reasonable inferences on real endoscopic videos. To utilize temporal correlations from WCE videos, a block-based temporal attention alignment network (BTAAN) is proposed to build accurate and robust correspondences among consecutive frames under poor video quality. This network exploits the global information from neighboring frames to capture large-scale motions, and suppresses the effect of unreliable points by a temporal attention mechanism. Moreover, its block-wise characteristic reduces the computational complexity significantly, which makes it hardware-friendly. Extensive experiments demonstrate the superiority of the proposed method over other state-of-the-art approaches on practical WCE applications.

Deep chroma prediction of Wyner–Ziv frames in distributed video coding of wireless capsule endoscopy video

Compression of captured video frames is crucial for saving the power in wireless capsule endoscopy (WCE). A low complexity encoder is desired to limit the power consumption required for compressing the WCE video. Distributed video coding (DVC) technique is best suitable for designing a low complexity encoder. In this technique, frames captured in RGB colour space are converted into YCbCr colour space. Both Y and CbCr representing luma and chroma components of the Wyner–Ziv (WZ) frames are processed and encoded in existing DVC techniques proposed for WCE video compression. In the WCE video, consecutive frames exhibit more similarity in texture and colour properties. The proposed work uses these properties to present a method for processing and encoding only the luma component of a WZ frame. The chroma components of the WZ frame are predicted by an encoder–decoder based deep chroma prediction model at the decoder by matching luma and texture information of the keyframe and WZ frame. The proposed method reduces the computations required for encoding and transmitting of WZ chroma component. The results show that the proposed DVC with a deep chroma prediction model performs better when compared to motion JPEG and existing DVC systems for WCE at the reduced encoder complexity.