Temporal Frame Filtering for Autonomous Driving Using 3D-Stacked Global Shutter CIS With IWO Buffer Memory and Near-Pixel Compute

Abstract

With the advancement of deep learning to solve autonomous driving problems, the computation and memory requirements have been growing rapidly. Near-pixel compute-based CMOS image sensors (CIS) have been investigated as a potential candidate to perform the initial computations of workloads close to the pixel and reduce data movement. In this work, we design a near-pixel compute CIS capable of implementing a temporal frame filtering network, which rejects redundant image frames targeting autonomous driving applications. To improve performance and avoid image distortion, 3D-stacked global shutter CIS is proposed. This architecture integrates photodiodes with memory and compute units using Cu-Cu hybrid bonding. We propose to use back-end-of-line (BEOL) compatible Tungsten-doped Indium Oxide Transistors (IWO FETs) based embedded DRAM as buffer memory to achieve refresh-free storage and high bandwidth connections between various components. Near-pixel compute circuit is optimized by including sparsity-aware adder tree and using NOR gates as data buffers. The two-tier system comprises photodiodes on tier-1 in 40 nm node, and near-pixel compute and buffer memory on tier-2 in 22 nm node. We perform simulations in Cadence, obtaining an energy efficiency of 65 TOPS/W and a compute density of 1.04 TOPS/mm <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$^2$</tex-math> </inline-formula> for 8 <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\times$</tex-math> </inline-formula> 8b MAC, with a total latency of 1.15 ms/frame.