Deep learning-based protocols to enhance infrared imaging systems

Abstract

Infrared (IR) spectroscopic imaging provides both morphologic and chemical detail; however, obtaining this extensive spectral-spatial information requires the ability to rapidly record high-quality data. Discrete frequency infrared (DFIR) imaging using a point scanning microscope strikes a balance between data quality and acquisition speed that, in principle, can further be aided by computational methods. Here, we report a deep learning-based framework to complement the process of data acquisition and information extraction. First, we introduce a convolutional neural network (CNN) to leverage both spatial and spectral information for segmenting data into informative sub-classes, which we call the IR-SEG network. We show that this framework increases accuracy by using approximately half of the features used in the typical pixel-wise classification of IR data. Second, we present a generative adversarial network (GAN)-based approach to reconstruct full data sets with low loss in the information from incomplete spatial and spectral data recording. Termed IR-REC, this approach is shown to speed up data acquisition by up to 20-fold for typical biomedical samples. In addition to enhancing the speed and quality of data, we also propose a method to utilize complementary morphologic detail to estimate the spatial details of a single band IR image beyond the diffraction limit. Finally, we discuss potential pitfalls and new opportunities that can be addressed by developing these methods further. Together, these deep learning techniques provide new capabilities for IR imaging to extract better quality information faster.