A 915–1220 TOPS/W Hybrid In-Memory Computing based Image Restoration and Region Proposal Integrated Circuit for Neuromorphic Vision Sensors in 65nm CMOS

Abstract

The bio-inspired asynchronous event-based neuromorphic vision sensors (NVS) are introducing a paradigm shift in visual information sensing and processing [1]. The feature of event-driven operation makes it ideal for low-power operation in the Internet-of-Things scenario such as traffic monitoring. However, the inherent noise in the sensor causes redundant wake-up operation and reduces tracking performance [2]. Energy efficient in-memory computing (IMC) based denoise operation allows blank-frame detection to gain 2X energy savings. Further energy savings can be obtained by exploiting spatial redundancy-objects usually occupy a small part ~5% of the frame in traffic monitoring [3]. Hence, region proposal (RP) is required to detect the region of interests (ROIs) in a valid frame along with their bounding box location coordinates, as shown in Fig. 1. For binary images, the conventional connected component labeling (CCL) algorithm [4] can propose ROIs by raster scanning the whole frame, but leads to longer search time and higher computing energy due to von Neumann operation. The promising IMC approach [3] has high energy efficiency, but has limited accuracy due to a simple algorithm constrained by in-memory operations as well as object fragmentation due to smooth surfaces (e.g. car windows) that do not generate events. In this work, we present a hybrid memory bit cell-collocated SRAM and DRAM (CRAM) consisting of 11 transistors for IMC-based image restoration (IR) and RP. The proposed CRAM supports image storage in SRAM and DRAM modes, denoise and region filling in diffusion mode and RP algorithm in projection mode.