High compression deep learning based single-pixel hyperspectral macroscopic fluorescence lifetime imaging in vivo.

M Ochoa,P Yan,R Yao,X Intes,M Barroso,A Rudkouskaya

doi:10.1364/boe.396771

Abstract

Single pixel imaging frameworks facilitate the acquisition of high-dimensional optical data in biological applications with photon starved conditions. However, they are still limited to slow acquisition times and low pixel resolution. Herein, we propose a convolutional neural network for fluorescence lifetime imaging with compressed sensing at high compression (NetFLICS-CR), which enables in vivo applications at enhanced resolution, acquisition and processing speeds, without the need for experimental training datasets. NetFLICS-CR produces intensity and lifetime reconstructions at 128 × 128 pixel resolution over 16 spectral channels while using only up to 1% of the required measurements, therefore reducing acquisition times from ∼2.5 hours at 50% compression to ∼3 minutes at 99% compression. Its potential is demonstrated in silico, in vitro and for mice in vivo through the monitoring of receptor-ligand interactions in liver and bladder and further imaging of intracellular delivery of the clinical drug Trastuzumab to HER2-positive breast tumor xenografts. The data acquisition time and resolution improvement through NetFLICS-CR, facilitate the translation of single pixel macroscopic flurorescence lifetime imaging (SP-MFLI) for in vivo monitoring of lifetime properties and drug uptake.

Highlights

Acquiring dense high-dimensional optical data in biological applications remains a challenge due to the very low levels of light typically encountered
This architecture is still dependent on inputting a large set of spatially coded measurements to form the images. To overcome this obstacle and accelerate bed-side translation of single-pixel imaging, we propose NetFLICS-compression ratios (CRs), a novel deep learning framework that reduces the acquisition times for Hyperspectral Macroscopic Fluorescence Lifetime Imaging (HMFLI) by more than one order of magnitude
Single pixel imaging retrieves fluorescence time domain (TD) information of the sample plane x through inverse solving m(t) = Px(t) (Eq 1) where P represents the sensitivity matrix of patterns used for the acquisition of m(t), composed of an 1D measurement per pattern P over time t, so that the number of 1D measurements needed for reconstruction equals the number of pixels at the desired resolution

Summary

Introduction

Acquiring dense high-dimensional optical data in biological applications remains a challenge due to the very low levels of light typically encountered. With NetFLICS19, a Convolutional Neural Network (CNN), single pixel raw fluorescence data can be reconstructed into intensity and lifetime images in a single workflow and without the need of parameter optimization.

Results

Conclusion