pCLE Images Research Articles

A novel contact optimization algorithm for endomicroscopic surface scanning.

Probe-based confocal laser endomicroscopy (pCLE) offers real-time, cell-level imaging and holds promise for early cancer diagnosis. However, a large area surface scanning for image acquisition is needed to overcome the limitation of field-of-view. Obtaining high-quality images during scanning requires maintaining a stable contact distance between the tissue and probe. This work presents a novel contact optimization algorithm to acquire high-quality pCLE images. The contact optimization algorithm, based on swarm intelligence of whale optimization algorithm, is designed to optimize the probe position, according to the quality of the image acquired by probe. An accurate image quality assessment of total co-occurrence entropy is introduced to evaluate the pCLE image quality. The algorithm aims to maintain a consistent probe-tissue contact, resulting in high-quality images acquisition. Scanning experiments on sponge, ex vivo swine skin tissue and stomach tissue demonstrate the effectiveness of the contact optimization algorithm. Scanning results of the sponge with three different trajectories (spiral trajectory, circle trajectory, and raster trajectory) reveal high-quality mosaics with clear details in every part of the image and no blurred sections. The contact optimization algorithm successfully identifies the optimal distance between probe and tissue, improving the quality of pCLE images. Experimental results confirm the high potential of this method in endomicroscopic surface scanning.

Clustering-Guided Twin Contrastive Learning for Endomicroscopy Image Classification.

Learning better representations is essential in medical image analysis for computer-aided diagnosis. However, learning discriminative semantic features is a major challenge due to the lack of large-scale well-annotated datasets. Thus, how can we learn a well-structured categorizable embedding space in limited-scale and unlabeled datasets? In this paper, we proposed a novel clustering-guided twin-contrastive learning framework (CTCL) that learns the discriminative representations of probe-based confocal laser endomicroscopy (pCLE) images for gastrointestinal (GI) tumor classification. Compared with traditional contrastive learning, in which only two randomly augmented views of the same instance are considered, the proposed CTCL aligns more semantically related and class-consistent samples by clustering, which improved intra-class tightness and inter-class variability to produce more informative representations. Furthermore, based on the inherent properties of CLE (geometric invariance and intrinsic noise), we proposed to regard CLE images with any angle rotation and CLE images with different noises as the same instance, respectively, for increased variability and diversity of samples. By optimizing CTCL in an end-to-end expectation-maximization framework, comprehensive experimental results demonstrated that CTCL-based visual representations achieved competitive performance on each downstream task as well as more robustness and transferability compared with existing state-of-the-art SSL and supervised methods. Notably, CTCL achieved 75.60%/78.45% and 64.12%/77.37% top-1 accuracy on the linear evaluation protocol and few-shot classification downstream tasks, respectively, which outperformed the previous best results by 1.27%/1.63% and 0.5%/3%, respectively. The proposed method holds great potential to assist pathologists in achieving an automated, fast, and high-precision diagnosis of GI tumors and accurately determining different stages of tumor development based on CLE images.

Self-Supervised Joint Learning for pCLE Image Denoising.

Probe-based confocal laser endoscopy (pCLE) has emerged as a powerful tool for disease diagnosis, yet it faces challenges such as the formation of hexagonal patterns in images due to the inherent characteristics of fiber bundles. Recent advancements in deep learning offer promise in image denoising, but the acquisition of clean-noisy image pairs for training networks across all potential scenarios can be prohibitively costly. Few studies have explored training denoising networks on such pairs. Here, we propose an innovative self-supervised denoising method. Our approach integrates noise prediction networks, image quality assessment networks, and denoising networks in a collaborative, jointly trained manner. Compared to prior self-supervised denoising methods, our approach yields superior results on pCLE images and fluorescence microscopy images. In summary, our novel self-supervised denoising technique enhances image quality in pCLE diagnosis by leveraging the synergy of noise prediction, image quality assessment, and denoising networks, surpassing previous methods on both pCLE and fluorescence microscopy images.

FF-ViT: probe orientation regression for robot-assisted endomicroscopy tissue scanning

PurposeProbe-based confocal laser endomicroscopy (pCLE) enables visualization of cellular tissue morphology during surgical procedures. To capture high-quality pCLE images during tissue scanning, it is important to maintain close contact between the probe and the tissue, while also keeping the probe perpendicular to the tissue surface. Existing robotic pCLE tissue scanning systems, which rely on macroscopic vision, struggle to accurately place the probe at the optimal position on the tissue surface. As a result, the need arises for regression of longitudinal distance and orientation via endomicroscopic vision.MethodThis paper introduces a novel method for automatically regressing the orientation between a pCLE probe and the tissue surface during robotic scanning, utilizing the fast Fourier vision transformer (FF-ViT) to extract local frequency representations and use them for probe orientation regression. Additionally, the FF-ViT incorporates a blur mapping attention (BMA) module to refine latent representations, which is combined with the pyramid angle regressor (PAR) to precisely estimate probe orientation.ResultA first of its kind dataset for pCLE probe-tissue orientation (pCLE-PTO) has been created. The performance evaluation demonstrates that our proposed network surpasses other top regression networks in accuracy, stability, and generalizability, while maintaining low computational complexity (1.8G FLOPs) and high inference speed (90 fps).ConclusionThe performance evaluation study verifies the clinical value of the proposed framework and its potential to be integrated into surgical robotic platforms for intraoperative tissue scanning.

High-Resolution Image Processing of Probe-Based Confocal Laser Endomicroscopy Based on Multistage Neural Networks and Cross-Channel Attention Module

Probe-based confocal laser endomicroscopy (pCLE) is a subcellular in vivo imaging technique that generates diagnostic images revealing malignant structural modifications in epithelial tissues. In the clinical diagnosis of probe confocal laser endomicroscopy (pCLE), the image background generally has the problems of dynamic blur or information loss, which is not conducive to achieving high-resolution and clear pCLE imaging. In recent years, deep learning technology has achieved remarkable results in image deblurring. For the task of recovering high-resolution pCLE images, the current methods still suffer from the following drawbacks: it is difficult to choose a strategy to make CNN converge at a deeper level and mainstream methods cannot handle the complex balance between spatial details and high-level feature information well when reconstructing clear images. In order to solve the problem, we propose a new cross-channel attention, multistage, high-resolution pCLE image deblurring structure. This methodology improves the supervised attention mechanism, enhances the ability of feature extraction and fusion capabilities, and improves the quality of image deblurring by adding cross-channel attention module (CAM) into the multistage neural networks’ architecture. The experimental results show that the average peak signal-to-noise ratio (PSNR) of the proposed model on the dataset is as high as 29.643 dB, and the structural similarity (SSIM) reaches 0.855. This method is superior to the prior algorithms in the visualization of recovered images, and the edge and texture details of the restored pCLE images are clearer.

Machine Learning-Based Prediction of Responsiveness to Neoadjuvant Chemoradiotheapy in Locally Advanced Rectal Cancer Patients from Endomicroscopy.

The protocol for treating locally advanced rectal cancer consists of the application of chemoradiotherapy (neoCRT) followed by surgical intervention. One issue for clinical oncologists is predicting the efficacy of neoCRT in order to adjust the dosage and avoid treatment toxicity in cases when surgery should be conducted promptly. Biomarkers may be used for this purpose along with in vivo cell-level images of the colorectal mucosa obtained by probe-based confocal laser endomicroscopy (pCLE) during colonoscopy. The aim of this article is to report our experience with Motiro, a computational framework that we developed for machine learning (ML) based analysis of pCLE videos for predicting neoCRT response in locally advanced rectal cancer patients. pCLE videos were collected from 47 patients who were diagnosed with locally advanced rectal cancer (T3/T4, or N+). The patients received neoCRT. Response to treatment by all patients was assessed by endoscopy along with biopsy and magnetic resonance imaging (MRI). Thirty-seven patients were classified as non-responsive to neoCRT because they presented a visible macroscopic neoplastic lesion, as confirmed by pCLE examination. Ten remaining patients were considered responsive to neoCRT because they presented lesions as a scar or small ulcer with negative biopsy, at post-treatment follow-up. Motiro was used for batch mode analysis of pCLE videos. It automatically characterized the tumoral region and its surroundings. That enabled classifying a patient as responsive or non-responsive to neoCRT based on pre-neoCRT pCLE videos. Motiro classified patients as responsive or non-responsive to neoCRT with an accuracy of ~ 0.62 when using images of the tumor. When using images of regions surrounding the tumor, it reached an accuracy of ~ 0.70. Feature analysis showed that spatial heterogeneity in fluorescence distribution within regions surrounding the tumor was the main contributor to predicting response to neoCRT. We developed a computational framework to predict response to neoCRT by locally advanced rectal cancer patients based on pCLE images acquired pre-neoCRT. We demonstrate that the analysis of the mucosa of the region surrounding the tumor provides stronger predictive power.

Optical Biopsies Using Probe-Based Confocal Laser Endomicroscopy for Autoimmune Pulmonary Alveolar Proteinosis

Introduction: Increasing numbers of cases of mild asymptomatic pulmonary alveolar proteinosis (PAP) are being reported with the recent increase in chest computed tomography (CT). Bronchoscopic diagnosis of mild PAP is challenging because of the patchy distribution of lesions, which makes it difficult to obtain sufficient biopsy samples. Additionally, the pathological findings of mild PAP, particularly those that differ from severe PAP, have not been fully elucidated. This study aimed to clarify the pathological findings of mild PAP and the usefulness of optical biopsy using probe-based confocal laser endomicroscopy (pCLE). Methods: We performed bronchoscopic optical biopsy using pCLE and tissue biopsy in 5 consecutive patients with PAP (three with mild PAP and two with severe PAP). We compared the pCLE images of mild PAP with those of severe PAP by integrating clinical findings, tissue pathology, and chest CT images. Results: pCLE images of PAP showed giant cells with strong fluorescence, amorphous substances, and thin alveolar walls. Images of affected lesions in mild PAP were equivalent to those obtained in arbitrary lung lesions in severe cases. All 3 patients with mild PAP spontaneously improved or remained stable after ≥3 years of follow-up. Serum autoantibodies to granulocyte-macrophage colony-stimulating factor were detected in all 5 cases. Conclusion: Optical biopsy using pCLE can yield specific diagnostic findings, even in patients with mild PAP. pCLE images of affected areas in mild and severe PAP showed similar findings, indicating that the dysfunction level of pathogenic alveolar macrophages in affected areas is similar between both disease intensities.

Probe-based confocal laser endomicroscopy for evaluating high-altitude pulmonary edema in canines

BackgroundReal-time assessment of high-altitude pulmonary edema (HAPE) remains a challenge. Probe-based confocal laser microscopy (pCLE) allows a real-time in vivo visualization of the alveoli. This study aimed to develop a new non-invasive method for analyzing microscopic images in a canine model of HAPE using pCLE. Materials and methodsThis was a prospective, controlled animal study in adult male beagle dogs randomized to control and HAPE groups. The HAPE group was exposed to a high altitude of 6000 m for 48 h. The blood gas levels, lung morphological changes, infectious factors, and lung wet-to-dry ratio were analyzed in different groups. The pCLE images were described based on the volume air index (VAI), which applies an integral over specific signal intensities. ResultsThe lung wet-to-dry weight ratio and injury scores in the HAPE group were significantly increased compared with those of the control group. The levels of infectious factors interleukin-1 beta, tumor necrosis factor-alpha, and interleukin-6 were significantly increased in the HAPE group compared with those in the control group. VAI was significantly decreased in the HAPE group. ConclusionpCLE is a potential adjudicative bronchoscopic imaging technique for assessing HAPE. VAI may be acquired from quantitative parameters in the analysis of images.

A novel OCT servoing control method for endomicroscopy surface scanning

Probe-based confocal laser endomicroscopy (pCLE) enables cell-level visualization for in-vivo tissue of the organ surface in real time. However, an image sequence is needed to generate a large-scale mosaic because the field of view (FOV) of pCLE is restricted. Image mosaicking solves the small FOV issue but only for high-quality pCLE images. The key to acquiring high-quality images is maintaining a constant contact distance even if the surface of the tissue is significantly rough. In this work, optical coherence tomography (OCT) is used to perform surface scanning by maintaining constant contact between the probe tip and the tissue. We proposed a distance detection method to find the position of the tissue surface in OCT images and used a position-based OCT servoing control scheme to guide the surface scanning. Experimental results show that the proposed control scheme with OCT guidance can obtain a large and clear circular mosaic (the diameter is 1 mm). The distance between the probe tip and the tissue is stabilized at the imaging position with distance variance less than 40 μm. The area of mosaics for a spiral trajectory with the proposed control scheme is 8.61 mm2. The proposed control scheme has a high potential to collect a clear pCLE image sequence by maintaining a constant distance between the probe tip and the tissue.

Boosting few-shot confocal endomicroscopy image recognition with feature-level MixSiam.

As an emerging early diagnostic technology for gastrointestinal diseases, confocal laser endomicroscopy lacks large-scale perfect annotated data, leading to a major challenge in learning discriminative semantic features. So, how should we learn representations without labels or a few labels? In this paper, we proposed a feature-level MixSiam method based on the traditional Siamese network that learns the discriminative features of probe-based confocal laser endomicroscopy (pCLE) images for gastrointestinal (GI) tumor classification. The proposed method is divided into two stages: self-supervised learning (SSL) and few-shot learning (FS). First, in the self-supervised learning stage, the novel feature-level-based feature mixing approach introduced more task-relevant information via regularization, facilitating the traditional Siamese structure can adapt to the large intra-class variance of the pCLE dataset. Then, in the few-shot learning stage, we adopted the pre-trained model obtained through self-supervised learning as the base learner in the few-shot learning pipeline, enabling the feature extractor to learn richer and more transferable visual representations for rapid generalization to other pCLE classification tasks when labeled data are limited. On two disjoint pCLE gastrointestinal image datasets, the proposed method is evaluated. With the linear evaluation protocol, feature-level MixSiam outperforms the baseline by 6% (Top-1) and the supervised model by 2% (Top1), which demonstrates the effectiveness of the proposed feature-level-based feature mixing method. Furthermore, the proposed method outperforms the previous baseline method for the few-shot classification task, which can help improve the classification of pCLE images lacking large-scale annotated data for different stages of tumor development.

Bronchoscopic Probe-Based Confocal Laser Endomicroscopy to Diagnose Diffuse Parenchymal Lung Diseases.

Diagnosis of diffuse parenchymal lung disease (DPLD) is based on clinical evaluation, radiological imaging and histology. However, additional techniques are warranted to improve diagnosis. Probe based confocal laser endomicroscopy (pCLE) allows real time in vivo visualisation of the alveolar compartment during bronchoscopy based on autofluorescence of elastic fibres. We used pCLE (Cellvizio®, Mauna Kea Technology. Inc, Paris, France) to characterise alveolar patterns in patients with different types of DPLD. In this pilot study we included 42 therapy naive patients (13 female, age 72.6 +/- 2.3 years), who underwent bronchoscopy for workup of DPLD. pCLE images were obtained during rigid bronchoscopy in affected lung segments according to HR-CT scan, followed by cryobiopsies in the identical area. Diagnoses were made by a multidisciplinary panel. The description of pCLE patterns was based on the degree of distortion of the hexagonal alveolar pattern, the density of alveolar structures, the presence of consolidations or loaded alveolar macrophages (AM). The assessment was performed by 2 investigators blinded for the final diagnosis. The normal lung showed a typical alveolar loop pattern. In amiodarone lung disease loaded AM were predominant. COP showed characteristic focal consolidations. IPF was characterized by significant distortion and destruction, NSIP showed significant increase in density, and chronic HP presented with consolidations, mild distortion and density. pCLE shows potential as an adjunctive bronchoscopic imaging technique in the differential diagnosis of DPLD. Structured and quantitative analysis of the images is required.

Intensity-based nonrigid endomicroscopic image mosaicking incorporating texture relevance for compensation of tissue deformation

Image mosaicking has emerged as a universal technique to broaden the field-of-view of the probe-based confocal laser endomicroscopy (pCLE) imaging system. However, due to the influence of probe-tissue contact forces and optical components on imaging quality, existing mosaicking methods remain insufficient to deal with practical challenges. In this paper, we present the texture encoded sum of conditional variance (TESCV) as a novel similarity metric, and effectively incorporate it into a sequential mosaicking scheme to simultaneously correct rigid probe shift and nonrigid tissue deformation. TESCV combines both intensity dependency and texture relevance to quantify the differences between pCLE image frames, where a discriminative binary descriptor named fully cross-detected local derivative pattern (FCLDP) is designed to extract more detailed structural textures. Furthermore, we also analytically derive the closed-form gradient of TESCV with respect to the transformation variables. Experiments on the circular dataset highlighted the advantage of the TESCV metric in improving mosaicking performance compared with the other four recently published metrics. The comparison with the other four state-of-the-art mosaicking methods on the spiral and manual datasets indicated that the proposed TESCV-based method not only worked stably at different contact forces, but was also suitable for both low- and high-resolution imaging systems. With more accurate and delicate mosaics, the proposed method holds promises to meet clinical demands for intraoperative optical biopsy.

Diagnostic accuracy of in vivo early tumor imaging from probe-based confocal laser endomicroscopy versus histologic examination in head and neck squamous cell carcinoma

Probe-based confocal laser endomicroscopy (pCLE) is a noninvasive and real-time imaging technique allowing acquisition of in situ images of the tissue microarchitecture during oral surgery. We aimed to assess the diagnostic performance of pCLE combined with patent blue V (PB) in improving the management of early oral cavity, oro/hypopharyngeal, and laryngeal cancer by imaging squamous cell carcinoma in vivo. The prospective study enrolled 44 patients with early head and neck lesions. All patients underwent white-light inspection or panendoscopy depending on the lesion's location, followed by pCLE imaging of the tumor core and its margins after topical application of PB. Each zone imaged by pCLE was interpreted at distance of the exam by three pathologists blinded to final histology. Most imaged zones could be presented to pathologists; the final sensitivity and specificity of pCLE imaging in head and neck cancers was 73.2-75% and 30-57.4%, respectively. During imaging, head and neck surgeons encountered some challenges that required resolving, such as accessing lesions with the flexible optical probe, achieving sufficiently precise imaging on the targeted tissues, and heterogeneous tissue staining by fluorescent dye. Final sensitivity scores were reasonable but final specificity scores were low. pCLE zones used to calculate specificity were acquired in areas of tumor margins, and the poor quality of the images acquired in these areas explains the final low specificity scores. Practical adjustments and technical training are needed to analyze head and neck lesions in various anatomical sites in real-time by pCLE.

Intranasal inoculation with Bordetella pertussis confers protection without inducing classical whooping cough in baboons.

The resurgence of whooping cough in many countries highlights the crucial need for a better understanding of the pathogenesis of respiratory infection by Bordetella pertussis. Exposure of baboons to B. pertussis by the intranasal and intra-tracheal routes is a recently described preclinical model that reproduces both B. pertussis infection of humans and whooping cough disease. Here, we tested both intranasal and intranasal+intra-tracheal exposure routes and assessed their impact on disease development and immunity. Young baboons were intranasally exposed to the B1917 clinical isolate, representative of circulating strains in Europe, or its green-fluorescent protein expressing derivative. Animals were followed for pertussis symptoms and bacterial colonization and by in vivo probe-based confocal laser endomicroscopy (pCLE) imaging. Sero-conversion and protection against subsequent infection were then evaluated. Seroconversion and bacterial colonization of both the nasopharynx and trachea was observed in baboons exposed to B. pertussis by the intranasal route only, and also in those animals challenged by both the intranasal and intra-tracheal routes together. However, baboons exposed solely by the intranasal route developed only mild clinical symptoms, with no paroxysmal cough. These animals were protected against re-infection by B. pertussis. Intranasal exposure of baboons to B. pertussis does not induce disease but elicits immune mechanisms that protect them from subsequent exposure to the bacteria. These findings suggest that the intranasal route of inoculation in this non-human primate model could be used in the pre-clinical evaluation of nasal candidate vaccines against pertussis.

Probe-based Confocal Laser Endomicroscopy for In Vivo Assessment of Histological Healing in Ulcerative Colitis: Development and Validation of the ENHANCE Index.

Histological healing may represent the ultimate therapeutic goal in ulcerative colitis [UC], but it requires biopsies. Our aim was to develop a non-invasive index able to assess histological disease activity in ulcerative colitis, using probe-based confocal laser endomicroscopy [pCLE]. One hundred patients with quiescent UC were prospectively included in five French centres. After fluorescein intravenous injection, during colonoscopy, the colorectal mucosa was analysed by white light imaging and pCLE, and then biopsied in different locations. Five endoscopists performed central reading of pCLE images blinded to clinical, endoscopic, and histological data. One expert pathologist performed a central histological reading [Nancy index: gold standard]. Univariate and multivariate analyses were performed to identify the endomicroscopic items associated with the presence of histologically active disease. Over 1000 pCLE videos sequences performed in 100 UC patients in endoscopic remission [Mayo 0 and 1] were evaluated. We observed that vessel diameter >20 µm, dilated crypt lumen, fluorescein leakage, and irregular crypt architecture were statistically associated with histologically proven inflammation according to the Nancy index. Hence, we built a pCLE index of mucosal inflammation with overall accuracy of 79.6% and overall sensitivity and specificity of, respectively, 57.8% and 82.8%. Negative predictive value, especially when a pCLE index ≤1 was observed, was high [93.1%]. Using a robust methodology, large vessel diameter, dilated crypt lumen, fluorescein leakage,and irregular crypt architecture are reliable endomicroscopic items defining the ENHANCE index for real-time assessment of histological disease activity in UC.

Reduced Intravenous Fluorescein Dose for Upper and Lower Gastrointestinal Tract Probe-Based Confocal Laser Endomicroscopy.

Background/AimsProbe-based confocal laser endomicroscopy (pCLE) requires the administration of intravenous (IV) fluorescein. This study aimed to determine the optimal dose of IV fluorescein for both upper and lower gastrointestinal (GI) tract pCLE. MethodsPatients 20 to 79 years old with gastric high-grade dysplasia (HGD) or colorectal neoplasms (CRNs) were enrolled in the study. The dose de-escalation method was employed with five levels. The primary endpoint of the study was the determination of the optimal dose of IV fluorescein for pCLE of the GI tract. The reduced dose was determined based on off-line reviews by three endoscopists. An insufficient dose of fluorescein was defined as the dose of fluorescein with which the pCLE images were not deemed to be visible. If all three endoscopists determined that the tissue structure was visible, the doses were de-escalated. ResultsA total of 12 patients with gastric HGD and 12 patients with CRNs were enrolled in the study. Doses were de-escalated to 0.5 mg/kg of fluorescein for both non-neoplastic duodenal and colorectal mucosa. All gastric HGD or CRNs were visible with pCLE with IV fluorescein at 0.5 mg/kg. ConclusionsIn the present study, pCLE with IV fluorescein 0.5 mg/kg was adequate to visualize the magnified structure of both the upper and lower GI tract.

Konfokální laserová endomikroskopie v diagnostice onemocnění jícnu: pilotní studie

Probebased confocal laser endomicroscopy (pCLE) is a novel diagnostic technique for endoscopy which enables a microscopic view at a cellular resolution in realtime. Endoscopic detection of early neoplasia in the distal esophagus is difficult and often these lesions can be missed. The aim of the pilot study was to obtain characteristic pCLE figures in esophageal diseases for following studies, and to evaluate the possible future role of pCLE in the diagnostics of dysplastic Barretts esophagus (BE) or early esophageal adenocarcinoma (EAC).A review of the current literature was performed and previously published pCLE images and classifications of esophageal diseases were searched and studied first. In phase two of the pilot study patients with esophageal diseases such as reflux esophagitis, BE and EAC were enrolled and scheduled for upper endoscopy with pCLE. A healthy cohort was also included.From January 2019 to July 2019, a total of 14 patients were enrolled in this prospective pilot study: 3 patients with reflux esophagitis, 4 with BE, 3 with EAC and 4 persons were included in the healthy cohort. The endoscopy with pCLE was performed and characteristic pCLE figures were obtained. The correct diagnoses based on realtime pCLE were evaluated by an endoscopist in 11 of the 14 cases (78.6 %).It was possible to obtain typical pCLE images of esophageal diseases during a standard capassisted endoscopic procedure. pCLE seems to be a feasible new technique in BE surveillance and early neoplastic lesion detection. However, more studies and data on larger number of patients are needed.

Near-Infrared Polymer Dots in the Portal-Hepatic Circulation Achieve Localization of Hepatic Carcinoma In Vivo.

The present study aims to use polymer dots to explore whether they can visualize tumor lesions in a diethylnitrosamine (DENA)-induced hepatocellular carcinoma (HCC) model. The HCC rat model was set up, and serum liver function indexes and AFP were tested on days 0, 30, 60, and 90 of the modeling process. After characterization of the polymer dots, they were injected into the rats and mice. The liver, spleen, and kidney of rats and the gallbladder of mice were extracted to verify the metabolic pathways of the polymer dots and their capability of fluorescent localization of HCC and gallbladder by fluorescence imaging. Strong fluorescent emission from the liver appeared immediately and 15 min after the polymer dots were injected through the main portal veins and tail veins of the model rats, respectively. A satisfactory fluorescent imaging effect lasted up to 45 min. Polymer dots circulate through the bloodstream within intrahepatic vessels rather than intracellular areas and can be clearly visualized by using both the pCLE and IVIS spectrum imaging systems. Contrast imaging of HCC lesions without fluorescent emissions was due to the lack of normal portal-hepatic veins within the tumor areas. Fluorescent imaging of the gallbladder could also be detected at 15 min after the polymer dots were injected through the tail veins of mice. The polymer dots had satisfactory fluorescent localization capability for targeted intrahepatic vessels and HCC lesions in vivo and showed potential practical value in hepato-biliary surgery.

Learning from irregularly sampled data for endomicroscopy super-resolution: a comparative study of sparse and dense approaches

PurposeProbe-based confocal laser endomicroscopy (pCLE) enables performing an optical biopsy via a probe. pCLE probes consist of multiple optical fibres arranged in a bundle, which taken together generate signals in an irregularly sampled pattern. Current pCLE reconstruction is based on interpolating irregular signals onto an over-sampled Cartesian grid, using a naive linear interpolation. It was shown that convolutional neural networks (CNNs) could improve pCLE image quality. Yet classical CNNs may be suboptimal in regard to irregular data.MethodsWe compare pCLE reconstruction and super-resolution (SR) methods taking irregularly sampled or reconstructed pCLE images as input. We also propose to embed a Nadaraya–Watson (NW) kernel regression into the CNN framework as a novel trainable CNN layer. We design deep learning architectures allowing for reconstructing high-quality pCLE images directly from the irregularly sampled input data. We created synthetic sparse pCLE images to evaluate our methodology.ResultsThe results were validated through an image quality assessment based on a combination of the following metrics: peak signal-to-noise ratio and the structural similarity index. Our analysis indicates that both dense and sparse CNNs outperform the reconstruction method currently used in the clinic.ConclusionThe main contributions of our study are a comparison of sparse and dense approach in pCLE image reconstruction. We also implement trainable generalised NW kernel regression as a novel sparse approach. We also generated synthetic data for training pCLE SR.

Deep Graph-Based Multimodal Feature Embedding for Endomicroscopy Image Retrieval.

Representation learning is a critical task for medical image analysis in computer-aided diagnosis. However, it is challenging to learn discriminative features due to the limited size of the data set and the lack of labels. In this article, we propose a deep graph-based multimodal feature embedding (DGMFE) framework for medical image retrieval with application to breast tissue classification by learning discriminative features of probe-based confocal laser endomicroscopy (pCLE). We first build a multimodality graph model based on the visual similarity between pCLE data and reference histology images. The latent similar pCLE-histology pairs are extracted by walking with the cyclic path on the graph while the dissimilar pairs are extracted based on the geodesic distance. Given the similar and dissimilar pairs, the latent feature space is discovered by reconstructing the similarity between pCLE and histology images via deep Siamese neural networks. The proposed method is evaluated on a clinical database with 700 pCLE mosaics. The accuracy of image retrieval demonstrates that DGMFE can outperform previous works on feature learning. Especially, the top-1 accuracy in an eight-class retrieval task is 0.739, thus demonstrating a 10% improvement compared to the state-of-the-art method.