Abstract

In recent years the use of convolutional layers to encode an inductive bias (translational equivariance) in neural networks has proven to be a very fruitful idea. The successes of this approach have motivated a line of research into incorporating other symmetries into deep learning methods, in the form of group equivariant convolutional neural networks. Much of this work has been focused on roto-translational symmetry of R d , but other examples are the scaling symmetry of R d and rotational symmetry of the sphere. In this work, we demonstrate that group equivariant convolutional operations can naturally be incorporated into learned reconstruction methods for inverse problems that are motivated by the variational regularisation approach. Indeed, if the regularisation functional is invariant under a group symmetry, the corresponding proximal operator will satisfy an equivariance property with respect to the same group symmetry. As a result of this observation, we design learned iterative methods in which the proximal operators are modelled as group equivariant convolutional neural networks. We use roto-translationally equivariant operations in the proposed methodology and apply it to the problems of low-dose computerised tomography reconstruction and subsampled magnetic resonance imaging reconstruction. The proposed methodology is demonstrated to improve the reconstruction quality of a learned reconstruction method with a little extra computational cost at training time but without any extra cost at test time.

Highlights

  • Deep learning has recently had a large impact on a wide variety of fields; research laboratories have published state-of-the-art results applying deep learning to sundry tasks such as playing Go [1], predicting protein structures [2] and generating natural language [3]

  • We investigate the use of equivariant neural networks within the framework of learned iterative reconstruction methods [5, 22], which constitute some of the most prototypical deep learning solutions to inverse problems

  • We have shown that equivariant neural networks can be naturally incorporated into learnable reconstruction methods for inverse problems

Read more

Summary

Introduction

Deep learning has recently had a large impact on a wide variety of fields; research laboratories have published state-of-the-art results applying deep learning to sundry tasks such as playing Go [1], predicting protein structures [2] and generating natural language [3]. Training data tends to be considerably less abundant in medical and scientific imaging than in the computer vision and image analysis tasks that are typical of the deep learning revolution, such as ImageNet classification [21] This suggests that the lower sample complexity of equivariant neural networks (as compared to ordinary CNNs) may be harnessed in this setting with scarce data to learn better reconstruction methods. We show that invariance of a functional to a group symmetry implies that its proximal operator satisfies an equivariance property with respect to that group This insight can be combined with the unrolled iterative method approach: it makes sense for a regularisation functional to be invariant to roto-translations if there is no prior knowledge on the orientation and position of structures in the images, in which case the corresponding proximal operators are roto-translationally equivariant. This outperformance is manifested in two main ways: the equivariant method is better able to take advantage of small training sets than the ordinary one, and its performance is more robust to transformations that leave images in orientations not seen during training

Notation and background on groups and representations
Learnable equivariant maps
Equivariant linear operators
Equivariant nonlinearities
Reconstruction methods motivated by variational regularisation
Equivariance in splitting methods
Learned proximal gradient descent
Experiments
Datasets
Experimental setup
CT experiment: varying the size of the training set
MRI experiment: varying the size of the training set
Findings
Conclusions and discussion

Talk to us

Join us for a 30 min session where you can share your feedback and ask us any queries you have

Schedule a call

Disclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.