Imaging Fiber Bundle Research Articles

Diffuser-based fiber endoscopy for single-shot 3D fluorescence imaging

Minimally invasive endoscopy using coherent fiber bundles shows great potential for numerous applications in biomedical imaging. With a diffuser on the distal side of the fiber bundle and computational image recovery, single-shot 3D imaging is possible by encoding the image volume into 2D speckle patterns. In comparison to equivalent lens systems, a higher space-bandwidth product can be achieved. However, decoding the image with iterative algorithms is time-consuming. Thus, we propose utilizing a neural network for fast 2D and 3D image reconstruction at video rate. In this work, single-shot 3D fluorescence imaging with an ultra-thin endoscope is demonstrated, enabling applications like calcium imaging for in vivo brain diagnostics at cellular resolution.

Honeycomb Artifact Removal Using Convolutional Neural Network for Fiber Bundle Imaging

We present a new deep learning framework for removing honeycomb artifacts yielded by optical path blocking of cladding layers in fiber bundle imaging. The proposed framework, HAR-CNN, provides an end-to-end mapping from a raw fiber bundle image to an artifact-free image via a convolution neural network (CNN). The synthesis of honeycomb patterns on ordinary images allows conveniently learning and validating the network without the enormous ground truth collection by extra hardware setups. As a result, HAR-CNN shows significant performance improvement in honeycomb pattern removal and also detailed preservation for the 1961 USAF chart sample, compared with other conventional methods. Finally, HAR-CNN is GPU-accelerated for real-time processing and enhanced image mosaicking performance.

Model-based deep learning for fiber bundle infrared image restoration

As the representative of flexibility in optical imaging media, in recent years, fiber bundles have emerged as a promising architecture in the development of compact visual systems. Dedicated to tackling the problems of universal honeycomb artifacts and low signal-to-noise ratio (SNR) imaging in fiber bundles, the iterative super-resolution reconstruction network based on a physical model is proposed. Under the constraint of solving the two subproblems of data fidelity and prior regularization term alternately, the network can efficiently “regenerate” the lost spatial resolution with deep learning. By building and calibrating a dual-path imaging system, the real-world dataset where paired low-resolution (LR) - high-resolution (HR) images on the same scene can be generated simultaneously. Numerical results on both the United States Air Force (USAF) resolution target and complex target objects demonstrate that the algorithm can restore high-contrast images without pixilated noise. On the basis of super-resolution reconstruction, compound eye image composition based on fiber bundle is also embedded in this paper for the actual imaging requirements. The proposed work is the first to apply a physical model-based deep learning network to fiber bundle imaging in the infrared band, effectively promoting the engineering application of thermal radiation detection.

GRINtrode: a neural implant for simultaneous two-photon imaging and extracellular electrophysiology in freely moving animals.

In vivo imaging and electrophysiology are powerful tools to explore neuronal function that each offer unique complementary information with advantages and limitations. Capturing both data types from the same neural population in the freely moving animal would allow researchers to take advantage of the capabilities of both modalities and further understand how they relate to each other. Here, we present a head-mounted neural implant suitable for in vivo two-photon imaging of neuronal activity with simultaneous extracellular electrical recording in head-fixed or fiber-coupled freely moving animals. A gradient refractive index (GRIN) lens-based head-mounted neural implant with extracellular electrical recording provided by tetrodes on the periphery of the GRIN lens was chronically implanted. The design of the neural implant allows for recording from head-fixed animals, as well as freely moving animals by coupling the imaging system to a coherent imaging fiber bundle. We demonstrate simultaneous two-photon imaging of GCaMP and extracellular electrophysiology of neural activity in awake head-fixed and freely moving mice. Using the collected information, we perform correlation analysis to reveal positive correlation between optical and local field potential recordings. Simultaneously recording neural activity using both optical and electrical methods provides complementary information from each modality. Designs that can provide such bi-modal recording in freely moving animals allow for the investigation of neural activity underlying a broader range of behavioral paradigms.

Endoscopic displacement measurement based on fiber optic bundles.

In-line monitoring and routine inspection are essential for using and maintaining complex equipment. The simultaneous implementation of visual positioning and displacement measurement allows the accurate acquisition of characteristics, including object dimensions and mechanical vibrations, while rapidly locking the target position. However, the internal structure of equipment is frequently obscured, making direct visual inspection challenging; therefore, flexible and bendable fiber optic-based endoscopes are extremely valuable in harsh conditions. This study enables all-fiber visual displacement measurement using a single-mode fiber and an imaging fiber bundle. Based on optical triangulation and spot centers extraction method from fiber bundle images, 0.07 mm precision at a measurement distance of 40.12 mm is achieved vertically for rough objects. We demonstrate its surface reconstruction and vibration measurement functions. Factors that affect measurement accuracy, such as light source and object roughness, are also discussed.

Research on Parameter Design and Simulation of Infrared Optical Fiber Imaging System Based on Error Theory

As the microlens array is attached to the infrared optical fiber imaging bundle, forming a rigid assemble is a new method to optimize the imaging quality of the infrared optical fiber imaging system. The alignment accuracy of the assembly is essential to the overall imaging quality of the system. To ensure that the assembly can still produce quality images under a certain alignment error, the imaging system model of assembly was established. Then introduced the concept of error margin and error margin coefficient to analyze the alignment error types and corresponding error effects of the assembly. Based on these concepts, the parameter design of the optical fiber imaging system was started. The limiting alignment errors were calculated at the error margin factor ξ = 1.25: transverse δx = 9 μm, longitudinal dz = 50 μm, and deflection angle γ = 1.5°. Lastly, a ZEMAX simulation study was carried out. The simulation results showed that: under a certain alignment error, the light did not leak but be completely coupled into the core of the fiber, which confirmed the accuracy of error analysis and parameter design. And the assembly was realized in certain high-quality imaging under an alignment error. So the error analysis and parameters were designed correctly, and high imaging quality transfer of the system with a certain alignment error was achieved for the assembly.

Research on the Coupled Modulation Transfer Function of the Discrete Sampling System with Hexagonal Fiber-Optic Imaging Bundles

In this study, we developed a numerical model of the coupled modulation transfer function (coupled-MTF) based on the discrete sampling system from the perspective of optical system imaging quality evaluation for coupled two-dimensional discrete sampling characteristics of the hexagonally aligned fiber-optic imaging bundles and CCD image elements. The results show that when the spatial frequency of the input target signal deviates from the Nyquist frequency by 1%, an increase in the number of fibers leads to a faster convergence of the oscillation of the coupled-MTF, and the coupled-MTF converges to a stable value when the number of fibers reaches 1000 × 1000. The deviation of the spatial frequency of the input target signal from the Nyquist frequency is within 1%, and the oscillatory convergence of the coupled-MTF accelerates with increasing deviation. The coupled-MTF oscillates with the deviation period of the wave peak of the input target signal from the initial position of the fiber center, and the theoretical oscillation spatial period is twice the fiber diameter. This study produces important guidelines for the selection of the number of fibers, input spatial frequency, and initial position deviation of the hexagonally arranged fiber imaging bundles.

Magnetically-sensitive nanodiamond thin-films on glass fibers

By assembling 140 nm-sized fluorescent nanodiamonds (FNDs) in a thin-film on (3-aminopropyl) triethoxysilane functionalized glass surface, we prepare magnetically-sensitive FND-fiber probes for endoscopy. The obtained FND layers show good uniformity over large surfaces and are characterized using confocal, fluorescence, and atomic force microscopes. Further, FNDs are assembled on single large-core multimode optical fibers and imaging fiber bundles end face to detect optically detectable magnetic resonance (ODMR) signals. The ODMR signals are recorded through the fiber’s far end in magnetic fields between 0 to 2.5 mT. A multi-channel sensor is demonstrated with the capability of parallel-in-time mapping and instantaneous readout from individual pixel and enabling magnetic mapping at high spatial resolution. Results of this study are promising for early stage detection in bio-diagnostic applications.

Lensless Fiber Imaging With Long Working Distance Based on Active Depth Measurement

Lensless fiber endoscopes allow noncontact imaging in narrow spaces without lenses or scanning devices at the distal tip. In this work, an active depth measurement module based on a Fabry–Perot interferometer is integrated onto the imaging fiber bundles. An accurate depth-dependent point spread function can be constructed for <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$a$ </tex-math></inline-formula> <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">priori</i> deconvolution in image deblurring. We have demonstrated that the working distance of a lensless fiber endoscope can be increased from the end face to several tens of micrometers, or even up to more than double for certain spatial frequencies. The contrast of the reconstructed image is also significantly improved. Furthermore, the proposed device can be implemented rapidly without precalibration and is resilient to fiber bending; thus, it can be used for real-time monitoring applications.

Sensitivity improved Cerenkov luminescence endoscopy using optimal system parameters.

The challenges of clinical translation of optical imaging, including the limited availability of clinically used imaging probes and the restricted penetration depth of light propagation in tissues can be avoided using Cerenkov luminescence endoscopy (CLE). However, the clinical applications of CLE are limited due to the low signal level of Cerenkov luminescence and the large transmission loss caused by the endoscope, which results in a relatively low detection sensitivity of current CLE. The aim of this study was to enhance the detection sensitivity of the CLE system and thus improve the system for clinical application in the detection of gastrointestinal diseases. Four optical fiber endoscopes were customized with different system parameters, including monofilament (MF) diameter of imaging fiber bundles, fiber material, probe coating, etc. The endoscopes were connected to the detector via a specifically designed straight connection device to form the CLE system. The β-2-[18F]-Fluoro-2-deoxy-D-glucose (18F-FDG) solution and the radionuclide of Gallium-68 (68Ga) were used to evaluate the performance of the CLE system. The images of the 18F-FDG solution acquired by the CLE were used to optimize imaging parameters of the system. By using the endoscope with optimized parameters, including the MF diameter of imaging fiber bundles, fiber materials, etc., the resolution and sensitivity of the assembled CLE system were measured by imaging the radionuclide of 68Ga. The results of 18F-FDG experiments showed that larger MF diameter led to higher collection efficiency. The fiber material and probe coating with high transmission ratios in the range of 400-900 nm also increased signal collection and transmission efficiency. The results of 68Ga evaluations showed that a minimum radioactive activity of radionuclides as low as 0.03 µCi was detected in vitro within 5 minutes, while that of 0.68 µCi can be detected within 1 minute. In vivo experiments also demonstrated that the developed CLE system achieved a high sensitivity at a submicrocurie level; that is, 0.44 µCi within 5 minutes, and 0.83 µCi within 1 minute. The weaker in vivo sensitivity was due to the attenuation of the signal by the mouse tissue skin and the autofluorescence interference produced by biological tissues. By optimizing the structural parameters of fiber endoscope and imaging parameters for data acquisition, we developed a CLE system with a sensitivity at submicrocurie level. These results support the possibility that this technology can clinically detect early tumors within 1 minute.

Block-Based Compressed Sensing for Fast Optic Fiber Bundle Imaging with High Spatial Resolution

Block-Based Compressed Sensing for Fast Optic Fiber Bundle Imaging with High Spatial Resolution

Fast visible camera diagnostic for dual shattered pellet injections at KSTAR

Direct imaging from fast visible cameras can provide intuitive information for the detailed dynamics of a fusion plasma. Images with different toroidal positions help our understanding of the 3D structure of the plasma phenomena. Fast-Visible camera (FVC) system was installed in KSTAR to investigate the assimilation process during plasma disruptions induced by Shattered Pellet Injections (SPIs). Since these events are highly non-axisymmetric and good toroidal coverage and high spatial and temporal resolution is required, the field of view of each camera is oriented towards the injection tube of each of the two SPIs, which are located toroidally opposite to each other. To investigate fragment penetration and ablation using images from a FVC, all components of the system should be able to support a fast frame rate due to the speed of shattered fragments of several hundreds of meter per second. The KSTAR FVC system typically uses a complementary metal-oxide-semiconductor (CMOS) with 10,000 frame per second at a resolution of 832 × 600 pixel typically for SPI experiments and its maximum frame rate is up to 600,000 frames per second with 64 × 8 pixels resolution. A hybrid optical system that consists of 2.04 m long relay optics mounted in the viewport cassette and a 4.5 m long imaging fiber bundle secures sufficient distance to reduce the harmful effects on the sensitive fast camera from radiation, neutron impact, and strong magnetic field. This paper reports the installation of the FVC system on KSTAR is introduced and semi three-dimensional observations of disruption mitigation processes during SPIs.

Resolution improvement in real-time and video mosaicing for fiber bundle imaging

A fiber bundle allows easy access to a wide range of human tissue. It has smaller diameters, which limits its effective field of view (FOV), and consists of a large number of cores surrounded by a cladding layer, which reduces its spatial resolution. In this paper, we develop an algorithm that processes successively captured raw fiber bundle images in an online fashion. Our algorithm tackles the tasks of super-resolution (SR) and video mosaicing jointly. The natural movement of the fiber tip in successive frames produces offsets that are random in the pixel domain to apply multi-frame SR imaging. Meanwhile, the associated FOV can be extended by mosaicing reconstructed SR images with obtained shifted information. Our approach has low computational complexity that allows for processing in real-time. The performance of resolution improvement in real-time and video mosaicing is demonstrated on the resolution target and biological samples.

Depixelation of coherent fiber bundle imaging by fiber-core-targeted scanning.

A novel fast proximal scanning method, to the best of our knowledge, termed fiber-core-targeted scanning (FCTS), is proposed for illuminating individual fiber cores sequentially to remove the pixelation effect in fiber bundle (FB) imaging. FCTS is based on a galvanometer scanning system. Through a dynamic control of the scan trajectory and speed using the prior knowledge of fiber core positions, FCTS experimentally verifies a precise sequential delivery of laser pulses into fiber cores at a maximal speed of 45,000 cores per second. By applying FCTS on a FB-based photoacoustic forward-imaging probe, the results demonstrate that FCTS eliminates the pixelation effect and improves the imaging quality.

Radiofrequency thermocoagulation of trigeminal nerve assisted by nerve bundle extraction and image fusion based on hamilton-jacobi equation

Background and Objective: Radiofrequency thermocoagulation is an effective method for treating classic trigeminal neuralgia. However, the accurate positioning of thermocoagulation is difficult. The purpose of this study was to design an optimal strategy for performing adjuvant surgery.Methods: A total of 60 patients with trigeminal neuralgia were divided into two groups. One group received conventional computed tomography (CT) guided treatment (CT group). In the other group, neural fiber bundles were firstly extracted based on the Hamilton-Jacobi equation. Then, the MRI, CT, and fiber bundle images were fused to visualize the relationship among semilunar ganglion, trigeminal nerve, and puncture needle (fusion group).Results: Trigeminal fiber bundles were extracted quickly by the contour tracking method, and different types of image fusion were realized for radiofrequency surgery navigation. In the fusion group, 13.3% of patients could not reach semilunar ganglion, and 76.9% of the remaining cases reached the ideal damage area. In the CT group, the preoperative design shows that 26.7% of patients may have puncture difficulty, and 54.5% of remaining cases reached the ideal damage area.Conclusion: The technique of neural bundle extraction and image fusion based on the Hamilton-Jacobi equation can be used to plan the personalized puncture path targeting the semilunar ganglion.

Visible wide-angle view imaging system for the first plasma on the HL-2M tokamak.

The wide-angle view imaging system, in terms of a tangential view diagnostic with field of view (FOV) of 56.8° and a downward-looking diagnostic from the top of the machine with FOV of 94.7°, has been newly constructed for the first plasma of the HL-2M tokamak achieved in December 2020. Its mission in this stage is to monitor the plasma evolution during its startup, sustainment, and disruption in the visible spectral range as well as the plasma-wall interaction. For the latter ultrawide view diagnostic, nearly three-quarters of the divertor region and half the area of the inner wall are in the view range. Both the diagnostics are characterized by a similar optical structure, i.e., the light emission from the plasma is collected by a front-end lens and transferred through an imaging fiber bundle to the camera. This optical structure is suitable for application in the complex tokamak environment mainly because the fiber bundle is flexible. Photos of glow discharges are acquired prior to the first plasma for testing the FOVs in the vacuum vessel. The spatial resolution is ∼4mm for the tangential view diagnostic and ∼10mm for the downward-looking diagnostic. The temporal resolutions, ranging from 90 to 360 Hz by changing the region of interest or binning acquisition mode of the color camera, are applied to record the plasma evolutions and/or dust creation events during the first plasma campaign.

High Contrast Imaging through Infrared Chalcogenide Imaging Fiber Bundle Illuminated by Quantum Cascade Laser

High Contrast Imaging through Infrared Chalcogenide Imaging Fiber Bundle Illuminated by Quantum Cascade Laser

Core crosstalk in ordered imaging fiber bundles.

Coherent fiber bundles are used widely for imaging. Commonly, disordered arrays of randomly sized fiber cores avoid proximity between like-cores, which would otherwise result in increased core crosstalk and a negative impact on imaging. Recently, stack-and-draw fiber manufacture techniques have been used to produce fibers with a controlled core layout to minimize core crosstalk. However, one must take manufacturing considerations into account during stack-and-draw fiber design in order to avoid impractical or unachievable fabrication. This comes with a set of practical compromises, such as using only a small number of different core sizes. Through characterization of core crosstalk patterns, this Letter aims to aid the understanding of crosstalk limitations imposed by such compromises in the core layout made for ease of fabrication.

Inline holographic microscopy through fiber imaging bundles.

Fiber imaging bundles are widely used as thin, passive image conduits for miniaturized and endoscopic microscopy, particularly for confocal fluorescence imaging. Holographic microscopy through fiber bundles is more challenging; phase conjugation approaches are complex and require extensive calibration. This paper describes how simple inline holographic microscopy can be performed through an imaging bundle using a partially coherent illumination source from a multimode fiber. The sample is imaged in transmission, with the intensity hologram sampled by the bundle and transmitted to a remote camera. The hologram can then be numerically refocused for volumetric imaging, achieving a resolution of approximately 6 µm over a depth range of 1 mm. The scheme does not require any complex prior calibration and hence is insensitive to bending.

Compact Dual-Channel (Hyperspectral and Video) Endoscopy

Combining reflected endoscopic image with spectral data has recently attracted much attention. We use optical fiber bundle probe, galvo-scanning module and imaging spectrometer module to realize a dual-channel endoscope capable of simultaneous acquisition of real-time video data and high throughput hyperspectral data. The frame per second of the video channel is 30, and the wavelength range in the hyperspectral channel is 400 - 750 nm, with 3 nm spectral resolution at 547 nm. To achieve fast hyperspectral imaging, we extract the region of interest (ROI) area from the video channel and utilize it as guidance to drive the galvo-scanning module to obtain the hyperspectral data in the ROI. In this way, the hyperspectral imaging speed for a selected ROI area can be reduced to less than 1 second, making it possible to achieve rapid detection. Utilizing this system, we acquired the hyperspectral image of fingerprints, dorsum of hand and skin melanin nevus, all of which demonstrate the compact dual-channel endoscopy has broad implications for research and therapeutics.