Abstract
We demonstrate residual channel attention networks (RCAN) for the restoration and enhancement of volumetric time-lapse (four-dimensional) fluorescence microscopy data. First we modify RCAN to handle image volumes, showing that our network enables denoising competitive with three other state-of-the-art neural networks. We use RCAN to restore noisy four-dimensional super-resolution data, enabling image capture of over tens of thousands of images (thousands of volumes) without apparent photobleaching. Second, using simulations we show that RCAN enables resolution enhancement equivalent to, or better than, other networks. Third, we exploit RCAN for denoising and resolution improvement in confocal microscopy, enabling ~2.5-fold lateral resolution enhancement using stimulated emission depletion microscopy ground truth. Fourth, we develop methods to improve spatial resolution in structured illumination microscopy using expansion microscopy data as ground truth, achieving improvements of ~1.9-fold laterally and ~3.6-fold axially. Finally, we characterize the limits of denoising and resolution enhancement, suggesting practical benchmarks for evaluation and further enhancement of network performance.
Highlights
All fluorescence microscopes suffer drawbacks and tradeoffs because they partition a finite signal budget in space and time
We modified the original residual channel attention networks (RCAN) architecture to handle image volumes rather than images, improving network efficiency so that our modified 3D RCAN model fits within graphics processing unit (GPU) memory (Fig. 1a, Methods, Supplementary Note 1)
To investigate RCAN denoising performance on fluorescence data, we began by acquiring matched pairs of low- and high- SNR Instant structured illumination microscopy (iSIM) volumes of fixed U2OS cells transfected with mEmeraldTomm20 (Methods, Supplementary Table 1, 2), labeling the outer mitochondrial membrane (Fig. 1b)
Summary
All fluorescence microscopes suffer drawbacks and tradeoffs because they partition a finite signal budget in space and time These limitations manifest when comparing different microscope types (e.g., three-dimensional structured illumination microscopy (SIM) offers better spatial resolution than high numerical aperture light sheet microscopy, but worse photobleaching); different implementations of the same microscope type (e.g., traditional implementations of SIM offer better spatial resolution than instant SIM (iSIM), but worse depth penetration and lower speed4); and within the same microscope (longer exposures and bigger pixels increase signal-to-noise ratio (SNR) at the expense of speed and resolution). We modify RCAN for 3D applications, showing that it matches or exceeds the performance of previous networks in denoising fluorescence microscopy data. We apply this capability for super-resolution imaging over thousands of image volumes (tens of thousands of images). We demonstrate 4-5 fold volumetric resolution improvement in multiple fixed- and live-cell samples when using stimulated emission depletion (STED)- and expansion18- microscopy ground truth to train RCAN models
Sign-in/Register to view highlights & summary Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
