Multi-resolution convolutional neural networks for inverse problems

Feng Wang,Johannes Müller,Trond R. Henninen,Rolf Erni,Christoph T. Koch,Alberto Eljarrat

doi:10.1038/s41598-020-62484-z

Abstract

Inverse problems in image processing, phase imaging, and computer vision often share the same structure of mapping input image(s) to output image(s) but are usually solved by different application-specific algorithms. Deep convolutional neural networks have shown great potential for highly variable tasks across many image-based domains, but sometimes can be challenging to train due to their internal non-linearity. We propose a novel, fast-converging neural network architecture capable of solving generic image(s)-to-image(s) inverse problems relevant to a diverse set of domains. We show this approach is useful in recovering wavefronts from direct intensity measurements, imaging objects from diffusely reflected images, and denoising scanning transmission electron microscopy images, just by using different training datasets. These successful applications demonstrate the proposed network to be an ideal candidate solving general inverse problems falling into the category of image(s)-to-image(s) translation.

Highlights

Inverse problems in image processing, phase imaging, and computer vision often share the same structure of mapping input image(s) to output image(s) but are usually solved by different applicationspecific algorithms
Recovering parameters that describe the physical state of a system from measurement requires solving an inverse problem, which often can be a problem that cannot be solved deterministically and non-iteratively
Aware of the fact that a deep convolutional neural network (DCNN) often first quickly recovers the dominant low-frequency components, and afterward the high-frequency ones in a rather slow manner[19,20], and inspired by the idea of multi-grid methods[21], we propose a novel multi-resolution deep convolutional neural network

Summary

Introduction

Inverse problems in image processing, phase imaging, and computer vision often share the same structure of mapping input image(s) to output image(s) but are usually solved by different applicationspecific algorithms. Fast-converging neural network architecture capable of solving generic image(s)-to-image(s) inverse problems relevant to a diverse set of domains We show this approach is useful in recovering wavefronts from direct intensity measurements, imaging objects from diffusely reflected images, and denoising scanning transmission electron microscopy images, just by using different training datasets. Further modification by attaching different coarse inputs to the layers in the encoder has been tested, but no apparent improvements in convergence is observed Another possible variation is mutating the architecture progressively by starting to fit only the components of low frequencies at the initial phases, inserting new layers to match the increasingly high-frequency features during the training, as has been demonstrated in some of the recent applications[24,25]. This generalization capability is demonstrated by solving three different inverse problems

Methods

Findings

Conclusion