A 0.5-V Real-Time Computational CMOS Image Sensor With Programmable Kernel for Feature Extraction

Abstract

As the growing demand on artificial intelligence (AI) Internet-of-Things (IoT) devices, smart vision sensors with energy-efficient computing capability are required. This article presents a low-power and low-voltage dual mode 0.5-V computational CMOS image sensor (C <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> IS) with array-parallel computing capability for feature extraction using convolution. In the feature extraction mode, by applying the pulsewidth modulation (PWM) pixel and switch-current integration (SCI) circuit, the in-sensor eight-directional matrix-parallel multiply-accumulate (MAC) operation is realized. Furthermore, the analog-domain convolution-on-readout (COR) operation, the programmable 3×3 kernel with ±3-bit weights, and the tunable-resolution column-parallel analog-to-digital converter (ADC) (1-8 bit) are implemented to achieve the real-time feature extraction without using additional memory and sacrificing frame rate. In the image capturing mode, the sensor provides the linear-response 8-bit raw image data. The C <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> IS prototype has been fabricated in the TSMC 0.18-μm standard process technology and verified to demonstrate the raw and feature images at 480 frames/s with a power consumption of 77/ 117 μW and the resultant FoM of 9.8/14.8 pJ/pixel/frame, respectively. The prototype sensor is used as a real-time edge feature detection frond-end camera and accompanied with a simplified convolutional neural network (CNN) architecture to demonstrate the hand gesture recognition. The prototype system achieves more than 95% validation accuracy.