Coherent Fiber Bundle Research Articles

Side-Polished Coherent Fiber Bundle Assemblies: A Pathway to Large Imaging Arrays

Side-Polished Coherent Fiber Bundle Assemblies: A Pathway to Large Imaging Arrays

Curved fiber compound eye camera inspired by the Strepsiptera vision.

The Strepsiptera vision possesses intriguing features of a large field of view (FOV) and relatively high resolution compared to normal compound eyes. However, it presents a significant challenge of the mismatch between the curved compound eyelet lens array and the planar image sensor to image in a large FOV for artificial compound eyes (ACE). We propose what we believe to be a novel curved fiber compound eye camera (CFCEC) here, which employs coherent fiber bundles as the optical relay system to transmit sub-images curvilinearly. A total of 106 eyelets are arranged based on a scheme similar to the Goldberg polyhedron, with the advantages of uniform interval and minor edge blindness. Then, a prototype of the CFCEC is fabricated and assembled. A series of experiments are conducted to assess the FOV, contrast, resolution, and overlap rate of FOV of the prototype. The results prove that the CFCEC has a total FOV of up to 160°×160° and a total overlap rate of FOV of approximately 65%, demonstrating the promising potential of the CFCEC in various applications, such as panoramic surveillance, 3D detection, and motion tracking.

OS03.7.A AUTOFLUORESCENCE BASED RECOGNITION OF BRAIN TUMORS WITH A CONVOLUTIONAL NEURAL NETWORK

Abstract BACKGROUND The delineation of malignant brain tumors from surrounding tissue is crucial for maximal safe resection. Existing tools for intraoperative tumor visualization lack the possibility of real-time tissue classification. We therefore built a coherent fiber bundle (CFB) endoscope for autofluorescence imaging and trained a convolutional neural network (CNN) to distinguish tumor tissue (T) and non-tumor tissue (nT) in an ex vivo proof of concept study. MATERIAL AND METHODS The T-cohort included frozen tissue sections of 35 glioblastoma and 30 brain metastasis patients (10 breast cancer, 10 melanoma, 10 lung cancer). The nT-cohort comprised frozen tissue sections of hippocampi and temporal lobes from 23 patients undergoing surgery for epilepsy treatment. For image acquisition, a laser with excitation wavelength of 473 nm was coupled into a CFB and tissue autofluorescence between 500-550 nm was collected using a bandpass filter. The acquisition time of the camera was set to 700 ms. A CNN (VGG-19) was trained to differentiate the labels T from nT wherein the ground truth was defined by histopathological diagnosis. Training, validation and test sets were randomly generated across all patients in 66.6% / 16.6% / 16.6% proportion. Set generation was repeated 5 times and reported results for the independent test sets are mean values of all iterations. RESULTS In total, 3259 CFB images were acquired (T:2408, nT:851), with usually 40 CFB images per sample. Removal of CFB background was not improving classification accuracy and thus, was not performed. The classification accuracy of the CNN with a threshold probability of 0.5 for T-label assignment was 0.986 (sensitivity: 0.988, specificity: 0.982) for analysis of images and 0.98 (sensitivity: 0.971, specificity: 1) when calculating the result for each patient. To simulate clinical application where correct identification of healthy tissue is essential, we created a specificity-tuned CNN with adjusted threshold probability of 0.9 for T-label assignment. This led to a decreased accuracy of 0.945 but increased specificity on the image evaluation approach to 1 (sensitivity: 0.92). CONCLUSION Autofluorescence imaging through a CFB is capable of ex vivo recognition of brain tumor tissue with a CNN. The simplicity of the setup and minimal need for data pre-processing allows for intraoperative implementation and potentially enables real-time in vivo brain tumor delineation. To realize intraoperative transition, analysis of ex vivo bulk tissue will be upcoming to capture the signal complexity caused by tissue depth.

Contrastive Adversarial Learning for Endomicroscopy Imaging Super-Resolution.

Endomicroscopy is an emerging imaging modality for real-time optical biopsy. One limitation of existing endomicroscopy based on coherent fibre bundles is that the image resolution is intrinsically limited by the number of fibres that can be practically integrated within the small imaging probe. To improve the image resolution, Super-Resolution (SR) techniques combined with image priors can enhance the clinical utility of endomicroscopy whereas existing SR algorithms suffer from the lack of explicit guidance from ground truth high-resolution (HR) images. In this paper, we propose an unsupervised SR pipeline to allow stable offline and kernel-generic learning. Our method takes advantage of both internal statistics and external cross-modality priors. To improve the joint learning process, we present a Sharpness-aware Contrastive Generative Adversarial Network (SCGAN) with two dedicated modules, a sharpness-aware generator and a contrastive-learning discriminator. In the generator, an auxiliary task of sharpness discrimination is formulated to facilitate internal learning by considering the rankings of training instances in various sharpness levels. In the discriminator, we design a contrastive-learning module to mitigate the ill-posed nature of SR tasks via constraints from both positive and negative images. Experiments on multiple datasets demonstrate that SCGAN reduces the performance gap between previous unsupervised approaches and the upper bounds defined in supervised settings by more than 50%, delivering a new state-of-the-art performance score for endomicroscopy super-resolution. Further application on a realistic Voronoi-based pCLE downsampling kernel proves that SCGAN attains PSNR of 35.851dB, improving 5.23dB compared with the traditional Delaunay interpolation.

Real time full-color imaging in a Meta-optical fiber endoscope

Endoscopes are an important component for the development of minimally invasive surgeries. Their size is one of the most critical aspects, because smaller and less rigid endoscopes enable higher agility, facilitate larger accessibility, and induce less stress on the surrounding tissue. In all existing endoscopes, the size of the optics poses a major limitation in miniaturization of the imaging system. Not only is making small optics difficult, but their performance also degrades with downscaling. Meta-optics have recently emerged as a promising candidate to drastically miniaturize optics while achieving similar functionalities with significantly reduced size. Herein, we report an inverse-designed meta-optic, which combined with a coherent fiber bundle enables a 33% reduction in the rigid tip length over traditional gradient-index (GRIN) lenses. We use the meta-optic fiber endoscope (MOFIE) to demonstrate real-time video capture in full visible color, the spatial resolution of which is primarily limited by the fiber itself. Our work shows the potential of meta-optics for integration and miniaturization of biomedical devices towards minimally invasive surgery.

The fast camera (Fastcam) imaging diagnostic systems on the DIII-D tokamak

Two camera systems are installed on the DIII-D tokamak at the toroidal positions of 90° (90° system) and 225° (225° system), respectively. The cameras have two types of relay optics, namely, a coherent optical fiber bundle and a periscope system. The periscope system provides absolute intensity calibration stability while sacrificing resolution (10 lp/mm), while the fiber system provides high resolution (16 lp/mm) while sacrificing calibration stability. The periscope is available only for the 90° system. The optics of the 225° system were designed for view stability, repeatability, and easy maintenance. The cameras are located inside optimized neutron, x ray and magnetic shielding in order to reduce electronics damage, reboots, and magnetic and neutron interference, increasing the overall system reliability. An automated filter wheel, providing remote filter change, allows for remote wavelength selection. A software suite automates camera acquisition and data storage, allowing for remote operation and reduced operator involvement. System metadata is used to streamline the data analysis workflow, particularly for intensity calibration. The spatial calibration uses multiple observable wall features, resulting in a reconstruction accuracy ≤2cm.

Fourier holographic endoscopy for imaging continuously moving objects.

Coherent fiber bundles are widely used for endoscopy, but conventional approaches require distal optics to form an object image and acquire pixelated information owing to the geometry of the fiber cores. Recently, holographic recording of a reflection matrix enables a bare fiber bundle to perform pixelation-free microscopic imaging as well as allows a flexible mode operation, because the random core-to-core phase retardations due to any fiber bending and twisting could be removed in situ from the recorded matrix. Despite its flexibility, the method is not suitable for a moving object because the fiber probe should remain stationary during the matrix recording to avoid the alteration of the phase retardations. Here, we acquire a reflection matrix of a Fourier holographic endoscope equipped with a fiber bundle and explore the effect of fiber bending on the recorded matrix. By removing the motion effect, we develop a method that can resolve the perturbation of the reflection matrix caused by a continuously moving fiber bundle. Thus, we demonstrate high-resolution endoscopic imaging through a fiber bundle, even when the fiber probe changes its shape along with the moving objects. The proposed method can be used for minimally invasive monitoring of behaving animals.

Preparation and imaging properties of coherent chalcogenide glass fiber bundles with large planar array for far-infrared transmission (invited)

Preparation and imaging properties of coherent chalcogenide glass fiber bundles with large planar array for far-infrared transmission (invited)

Selective plane illumination optical endomicroscopy with polymer imaging fibers

Imaging fibers are used to perform real-time fluorescence endomicroscopy, in vivo, in situ, with the goal of increasing diagnostic information for a plethora of organ systems and diseases. Widefield fiber endomicroscopy systems are simple, cost-effective, and come with fast image acquisition times. However, alternative approaches such as scanning systems produce higher contrast images with intrinsic optical sectioning, improving the visibility of histological features, albeit at the expense of simplicity, cost, and acquisition rate. We developed a selective plane illumination microscopy endoscopic fiber platform, consisting of an ultrafast laser fabricated end-cap, integrated with a polymer coherent fiber bundle, and an epifluorescence microscope. Polymer fibers are known to fluoresce when pumped with blue light, enhancing the background and noise in images. Our end-cap design circumvents this challenge. We demonstrate a reduction of out-of-focus features, along with improved contrast of in-focus features, in images of a tissue phantom. Moreover, we demonstrate the utility of our platform for endomicroscopy using a whole, ex vivo human lung model.

Diffuser-based fiber endoscopy for single-shot 3D fluorescence imaging

Minimally invasive endoscopy using coherent fiber bundles shows great potential for numerous applications in biomedical imaging. With a diffuser on the distal side of the fiber bundle and computational image recovery, single-shot 3D imaging is possible by encoding the image volume into 2D speckle patterns. In comparison to equivalent lens systems, a higher space-bandwidth product can be achieved. However, decoding the image with iterative algorithms is time-consuming. Thus, we propose utilizing a neural network for fast 2D and 3D image reconstruction at video rate. In this work, single-shot 3D fluorescence imaging with an ultra-thin endoscope is demonstrated, enabling applications like calcium imaging for in vivo brain diagnostics at cellular resolution.

GRINtrode: a neural implant for simultaneous two-photon imaging and extracellular electrophysiology in freely moving animals.

In vivo imaging and electrophysiology are powerful tools to explore neuronal function that each offer unique complementary information with advantages and limitations. Capturing both data types from the same neural population in the freely moving animal would allow researchers to take advantage of the capabilities of both modalities and further understand how they relate to each other. Here, we present a head-mounted neural implant suitable for in vivo two-photon imaging of neuronal activity with simultaneous extracellular electrical recording in head-fixed or fiber-coupled freely moving animals. A gradient refractive index (GRIN) lens-based head-mounted neural implant with extracellular electrical recording provided by tetrodes on the periphery of the GRIN lens was chronically implanted. The design of the neural implant allows for recording from head-fixed animals, as well as freely moving animals by coupling the imaging system to a coherent imaging fiber bundle. We demonstrate simultaneous two-photon imaging of GCaMP and extracellular electrophysiology of neural activity in awake head-fixed and freely moving mice. Using the collected information, we perform correlation analysis to reveal positive correlation between optical and local field potential recordings. Simultaneously recording neural activity using both optical and electrical methods provides complementary information from each modality. Designs that can provide such bi-modal recording in freely moving animals allow for the investigation of neural activity underlying a broader range of behavioral paradigms.

Calibration of coherent optical fiber bundle with LCD screen for deformation measurements

Owning to its flexibility, high temperature resistance, and freedom from electromagnetic interference, coherent optical fiber bundle (COFB) is widely used in special imaging applications. However, limited by the manufacturing process, the fiber arrangement of COFB is prone to misalignment, resulting in imaging distortions that are not easily corrected. This limits the application of COFB in the field of optical measurement, making it difficult to meet the requirements of high-precision deformation measurement. To effectively calibrate the misalignment distortion and recover the original image information, this study proposes an easy-to-implement and highly accurate misalignment distortion calibration method. The LCD screen was used as the calibration reference to display a pre-designed regular arrangement pattern. The mapping relationship between the fiber output and the screen display information was confirmed using a correlation algorithm. The proposed method requires only 36 images to achieve misalignment calibration for 450 × 450 fibers. The image reconstruction results show that the proposed method can not only achieve global misalignment calibration, but also successfully eliminate honeycomb artifacts, thus enhancing the accuracy and practicality of COFB for deformation measurement. In addition, a COFB-based digital image correlation (DIC) system was established and a set of typical DIC experiments were implemented. The experimental results show that the proposed calibration method can significantly improve the deformation measurement accuracy of the COFB-based DIC system, and the accuracy of the two-dimensional (2D) displacement field measurement result is improved from 0.50 pixels to 0.03 pixels.

Learned end-to-end high-resolution lensless fiber imaging towards real-time cancer diagnosis

Recent advances in label-free histology promise a new era for real-time diagnosis in neurosurgery. Deep learning using autofluorescence is promising for tumor classification without histochemical staining process. The high image resolution and minimally invasive diagnostics with negligible tissue damage is of great importance. The state of the art is raster scanning endoscopes, but the distal lens optics limits the size. Lensless fiber bundle endoscopy offers both small diameters of a few 100 microns and the suitability as single-use probes, which is beneficial in sterilization. The problem is the inherent honeycomb artifacts of coherent fiber bundles (CFB). For the first time, we demonstrate an end-to-end lensless fiber imaging with exploiting the near-field. The framework includes resolution enhancement and classification networks that use single-shot CFB images to provide both high-resolution imaging and tumor diagnosis. The well-trained resolution enhancement network not only recovers high-resolution features beyond the physical limitations of CFB, but also helps improving tumor recognition rate. Especially for glioblastoma, the resolution enhancement network helps increasing the classification accuracy from 90.8 to 95.6%. The novel technique enables histological real-time imaging with lensless fiber endoscopy and is promising for a quick and minimally invasive intraoperative treatment and cancer diagnosis in neurosurgery.

Upgrade and application of the gas puff imaging system in EAST

The gas puff imaging (GPI) system on EAST was developed in 2012 and upgraded in 2021. A new relay optical system, consisting of a front reflecting prism, a series of lenses and a filter, is developed for the GPI diagnostic. At the end of the relay optical system, the rays are focused on a thin image surface, which is captured by the sensor of a high-speed camera. In contrast with the previous optical system of GPI in which a coherent glass fiber bundle is used to transmit the image from the end of a telescope inside the vacuum vessel to the outside, the new relay optical system has much lower light loss, i.e., the emission intensity on the image plane of the new GPI is at least 15 times higher than the previous one. In consequence, the temporal resolution of GPI diagnostic on EAST can be raised significantly. The analysis of the optical design denotes that the imaging quality is high enough to ensure a spatial resolution of 2 mm on the objective plane. In the spring experimental campaign of 2021, the upgraded GPI system was commissioned in EAST. Clear poloidal and radial propagations of the edge fluctuations are measured directly by GPI with a high sampling rate of 530 kHz. The poloidal and radial velocities of the edge fluctuations are derived by the time-delay cross-correlation method, with the radial velocity propagating outward, and the poloidal velocity propagating in the ion-diamagnetic drift direction in the SOL and in the electron-diamagnetic drift direction inside the LCFS.

Miniature structured illumination microscope for in vivo 3D imaging of brain structures with optical sectioning.

We present a high-resolution miniature, light-weight fluorescence microscope with electrowetting lens and onboard CMOS for high resolution volumetric imaging and structured illumination for rejection of out-of-focus and scattered light. The miniature microscope (SIMscope3D) delivers structured light using a coherent fiber bundle to obtain optical sectioning with an axial resolution of 18 µm. Volumetric imaging of eGFP labeled cells in fixed mouse brain tissue at depths up to 260 µm is demonstrated. The functionality of SIMscope3D to provide background free 3D imaging is shown by recording time series of microglia dynamics in awake mice at depths up to 120 µm in the brain.

Hybrid confocal Raman endomicroscopy for morpho-chemical tissue characterization

Confocal laser endomicroscopy (CLE) offers imaging of tissue microarchitecture and has emerged as a promising tool for in vivo clinical diagnosis of cancer across many organs. CLE, however, can show high inter-observer dependency and does not provide information about tissue molecular composition. In contrast, Raman spectroscopy is a label-free optical technique that provides detailed biomolecular compositional information but offers limited or no morphological information. Here we present a novel hybrid fiber-optic confocal Raman endomicroscopy system for morpho-chemical tissue imaging and analysis. The developed confocal endomicroscopy system is based on a novel detection scheme for rejecting Raman silica fiber interference permitting simultaneous CLE imaging and Raman spectral acquisition of tissues through a coherent fiber bundle. We show that this technique enables real-time microscopic visualization of tissue architecture as well as simultaneous pointwise label-free biomolecular characterization and fingerprinting of tissue paving the way for multimodal diagnostics at endoscopy.

Single-shot 3D endoscopic imaging exploiting a diffuser and neural networks

Lens-based endoscopes offer high lateral resolution, but suffer from rigid imaging properties, such as a fixed focal plane. We present a miniaturized 0.5 mm diameter endoscope in which the objective lens is replaced by an optical diffuser. The intensity information of the object space is scattered and passed to a camera via a coherent fibre bundle. The image is reconstructed by a neural network. The field of view and resolution depend on the object distance. 3D-single-shot imaging up to video rate can be enabled. The approach shows great potential for applications like robust 3D fluorescence imaging.

Calibration of Coherent Optical Fiber Bundle with Lcd Screen for Deformation Measurements

Calibration of Coherent Optical Fiber Bundle with Lcd Screen for Deformation Measurements

Development of an ultracompact endoscopic three-dimensional scanner with flexible imaging fiber optics

We developed an ultracompact three-dimensional (3D) endoscopic scanner that utilizes an electromagnetic-based stereo imaging method (EBSIM) for shape measurements. The method was used to investigate the applicability of imaging fiber optics to featureless surface profiles in confined spaces. The EBSIM is combined with a coherent flexible fiber bundle for structured light pattern projection to ensure that the scanner head can achieve a final diameter of merely 3.4 mm and a baseline length of 1.4 mm. Compared with the existing methods, the scanner is designed to have a minimum focal distance of 2 mm to address the intraoperative evaluation problem. System registration and complex calibration are unnecessary before measurements, and the scanner can be operated from any position and orientation. The scanner is capable of acquiring 3D videos at 30 frames per second. The experimental results demonstrate the feasibility of using the EBSIM for 3D surface profile measurements of featureless cylindrical surfaces. A prototype scanner is presented in extensor, and the average error of a series of known depth distances is found to be equal to 0.15 mm. The implementation shows that the 3D scanner can be potentially applied for endoscopic inspections and intraoperative evaluations in minimally invasive surgeries.

Depixelation of coherent fiber bundle imaging by fiber-core-targeted scanning.

A novel fast proximal scanning method, to the best of our knowledge, termed fiber-core-targeted scanning (FCTS), is proposed for illuminating individual fiber cores sequentially to remove the pixelation effect in fiber bundle (FB) imaging. FCTS is based on a galvanometer scanning system. Through a dynamic control of the scan trajectory and speed using the prior knowledge of fiber core positions, FCTS experimentally verifies a precise sequential delivery of laser pulses into fiber cores at a maximal speed of 45,000 cores per second. By applying FCTS on a FB-based photoacoustic forward-imaging probe, the results demonstrate that FCTS eliminates the pixelation effect and improves the imaging quality.