Improving correlation based super-resolution microscopy images through image fusion by self-supervised deep learning

Abstract

Super-resolution imaging is a powerful tool in modern biological research, allowing for the optical observation of subcellular structures with great detail. In this paper, we present a deep learning approach for image fusion of intensity and super-resolution optical fluctuation imaging (SOFI) microscopy images. We construct a network that can successfully combine the advantages of these two imaging methods, producing a fused image with a resolution comparable to that of SOFI and an SNR comparable to that of the intensity image. We also demonstrate the effectiveness of our approach experimentally, specifically on cell samples where microtubules were stained with ATTO647N and imaged using a confocal microscope with a single photon fiber bundle camera, allowing for the simultaneous acquisition of an image scanning microscopy (ISM) image and a SOFISM (ISM and SOFI) image. Our network is designed as a self-supervised network and shows the ability to train on a single pair of images and to generalize to other image pairs without the need for additional training. Our approach offers a flexible and efficient way to combine the strengths of correlation based imaging techniques along with traditional intensity based microscopy, and can be readily applied to other fluctuation based imaging modalities.