Cross-Layer Noise Analysis in Smart Digital Pixel Sensors With Integrated Deep Neural Network

Abstract

Digital pixel based image sensors with embedded deep neural network (DNN) allow many mission critical surveillance applications. However, image noise caused by variations and non-idealities in the sensor aggravates the quality of image and further degrades the performance of a DNN. We propose a digital pixel-DNN cross-layer simulation methodology for accurate training and evaluation of a DNN under image noise induced from sensors. In particular, this paper focuses on the image noise derived from device mismatches in digital pixel circuits with 3D integrated and pixel-parallel readout circuits, and studies the effect of the resulting image noise on the accuracy of a DNN. The simulation results show that the device mismatch in the digital pixel creates distinct noise structure on output image and should be accurately considered while training a DNN. We also present design space explorations using our cross-layer simulation.