Mutual Transfer Learning of Reconstructing Images Through a Multimode Fiber or a Scattering Medium

Xuetian Lai,Ziyang Chen,Xiaoyan Wu,Guodong Liu,Qiongyao Li,Jixiong Pu

doi:10.1109/access.2021.3077560

Abstract

In recent years, convolutional neural network (CNN) has been successfully applied to reconstruct image from the speckle, which is generated as an object passes through a scattering medium or a multimode fiber (MMF). To reconstruct image from the speckle, the CNN must be trained with a large number of object-speckle pairs (training dataset), and the trained CNN is capable of reconstructing image from dataset (test dataset), which is taken in the same condition as the training dataset. However, in some cases, data type and the scattering medium may vary with the situation. In this case, the CNN has to be re-trained using a large number of new data taken from the new scattering media for reconstructing image. In this paper, we develop a CNN called as Mobiledense-net (MDN) to realize the mutual transfer learning . Specifically, the MDN is first pre-trained with a large number of object-speckle pairs taken from MMF or scattering slab, then tuned with quite small number of object-speckle pairs from scattering slab or MMF. It is shown that in this case the MDN can reconstruct image from the speckle with quite good quality, in which the speckle is taken from MMF or the scattering slab. We also show that using a more complex dataset for pre-training, the amount of data for pre-training can be largely reduced and reconstruction quality can be further improved. Using transfer learning , the reconstruction quality is quite good, being up to 99%. The results in this paper provide a more generalized method for studying the imaging through scattering imaging or MMF by using CNN.

Highlights

In recent years, there have been increasing interests in the field of computational imaging with deep learning
The experimental results are presented as follows: 1) First of all, we prove the transfer learning from multimode fiber (MMF) to scattering slabs for reconstructing images as the objects pass through three different scattering slabs
In this paper, we designed a convolutional neural network (CNN) structure to realize the mutual transfer learning between MMF and scattering slabs

Summary

Introduction

There have been increasing interests in the field of computational imaging with deep learning. With a deeper understanding and increased processing speed, the deep neural networks are found to be used in the field of computational imaging. The deep neural networks can solve many remaining problems in fields of scattering imaging, computational ghost imaging, Fourier laminar microscopic imaging, phase retrieval and hologram data compression etc. Various approaches have been proposed for imaging through scattering media or MMF, such as wavefront shaping, speckle correlation and digital phase conjugation etc. [12,13,14,15,16] These approaches are found to be capable of reconstructing images from speckles. These approaches require very complicated and precise experimental control and VOLUME XX, 2017

Methods

Results

Discussion

Conclusion