Three-dimensional residual channel attention networks denoise and sharpen fluorescence microscopy image volumes.

Jiji Chen,Srineil Nizambad,Christian A Combs,Hari Shroff,Jiamin Liu,Yicong Wu,Arpita Upadhyaya,Ivan Rey-Suarez,Hideki Sasaki,Hung-Yu Chang,Xuesong Li,Shih-Jong J Lee,Hoyin Lai,Alexander Zhovmer,Yijun Su,Min Guo,Chi Chou Huang,Luciano A G Lucas

doi:10.1038/s41592-021-01155-x

Abstract

We demonstrate residual channel attention networks (RCAN) for the restoration and enhancement of volumetric time-lapse (four-dimensional) fluorescence microscopy data. First we modify RCAN to handle image volumes, showing that our network enables denoising competitive with three other state-of-the-art neural networks. We use RCAN to restore noisy four-dimensional super-resolution data, enabling image capture of over tens of thousands of images (thousands of volumes) without apparent photobleaching. Second, using simulations we show that RCAN enables resolution enhancement equivalent to, or better than, other networks. Third, we exploit RCAN for denoising and resolution improvement in confocal microscopy, enabling ~2.5-fold lateral resolution enhancement using stimulated emission depletion microscopy ground truth. Fourth, we develop methods to improve spatial resolution in structured illumination microscopy using expansion microscopy data as ground truth, achieving improvements of ~1.9-fold laterally and ~3.6-fold axially. Finally, we characterize the limits of denoising and resolution enhancement, suggesting practical benchmarks for evaluation and further enhancement of network performance.

Highlights

All fluorescence microscopes suffer drawbacks and tradeoffs because they partition a finite signal budget in space and time
We modified the original residual channel attention networks (RCAN) architecture to handle image volumes rather than images, improving network efficiency so that our modified 3D RCAN model fits within graphics processing unit (GPU) memory (Fig. 1a, Methods, Supplementary Note 1)
To investigate RCAN denoising performance on fluorescence data, we began by acquiring matched pairs of low- and high- SNR Instant structured illumination microscopy (iSIM) volumes of fixed U2OS cells transfected with mEmeraldTomm20 (Methods, Supplementary Table 1, 2), labeling the outer mitochondrial membrane (Fig. 1b)

Summary

Introduction

All fluorescence microscopes suffer drawbacks and tradeoffs because they partition a finite signal budget in space and time These limitations manifest when comparing different microscope types (e.g., three-dimensional structured illumination microscopy (SIM) offers better spatial resolution than high numerical aperture light sheet microscopy, but worse photobleaching); different implementations of the same microscope type (e.g., traditional implementations of SIM offer better spatial resolution than instant SIM (iSIM), but worse depth penetration and lower speed4); and within the same microscope (longer exposures and bigger pixels increase signal-to-noise ratio (SNR) at the expense of speed and resolution). We modify RCAN for 3D applications, showing that it matches or exceeds the performance of previous networks in denoising fluorescence microscopy data. We apply this capability for super-resolution imaging over thousands of image volumes (tens of thousands of images). We demonstrate 4-5 fold volumetric resolution improvement in multiple fixed- and live-cell samples when using stimulated emission depletion (STED)- and expansion18- microscopy ground truth to train RCAN models

Results

Conflicts of Interest

Methods

Code availability