Abstract
In this paper we propose a new method to jointly design a sensor and its neural-network based processing. Using a differential ray tracing (DRT) model, we simulate the sensor point-spread function (PSF) and its partial derivative with respect to any of the sensor lens parameters. The proposed ray tracing model makes no thin lens nor paraxial approximation, and is valid for any field of view and point source position. Using the gradient backpropagation framework for neural network optimization, any of the lens parameter can then be jointly optimized along with the neural network parameters. We validate our method for image restoration applications using three proves of concept of focus setting optimization of a given sensor. We provide here interpretations of the joint optical and processing optimization results obtained with the proposed method in these simple cases. Our method paves the way to end-to-end design of a neural network and lens using the complete set of optical parameters within the full sensor field of view.
Highlights
The increasing interest in the field of computational imaging has naturally led to the question of the joint design of sensor and processing
Few papers consider the whole set of lens optical parameters, which implies to interact with an optical design software [2, 8,9,10]
As in state of the art papers [11,12,13,14,15,16,17,18,19,20,21], the sensor is modeled with a convolutional layer within the neural network framework, we propose to use an optical model based on differential ray tracing (DRT), relying on the complete set of real lens parameters
Summary
The increasing interest in the field of computational imaging has naturally led to the question of the joint design of sensor and processing. As in state of the art papers [11,12,13,14,15,16,17,18,19,20,21], the sensor is modeled with a convolutional layer within the neural network framework, we propose to use an optical model based on differential ray tracing (DRT), relying on the complete set of real lens parameters. In contrast with the literature, this model does not rely on the thin lens approximation nor paraxial rays It can be used for the joint optimization of any set of real lens and the neural network parameters, for any field of view.
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a call
