A 1mJ/frame unified media application processor with a 179.7pJ mixed-mode feature extraction engine for embedded 3D-media contents processing

Abstract

A unified media application processor (UMAP) with a low-power mixed-mode feature extraction engine (FEE) is presented for 2D/3D image analysis/synthesis applications on handheld devices. UMAP supports not only graphics and vision for augmented reality (AR) but also 3D reconstruction and 3D display for 3D-view AR based on heterogeneous many-core platform. A frame-level 3-stage pipelined architecture enables real-time (50fps in VGA) performance in 3D-view AR, while a mixed-mode FEE dynamically saves active power by reconfiguring operation modes between analog and digital processing. Especially for low power operation in media processing, four pairs of analog current contention logics (CCL) are implemented in FEE. The implemented CCL does not require digital-to-analog or analog-to-digital converters (DAC/ADC) in interfacing digital and analog domains. It includes a diode-connected sensing stabilizer which reduces minimum sensing current. Therefore, average power consumed in CCL is reduced by 44.9%. In the implemented UMAP, the proposed FEE replaces the parallel processing core cluster in the analog processing mode, as a result, 96.5% of cluster power and 99.1% of target detection time are saved. The dynamic mode transition between analog and digital processing based on run-time tracking of region-of-interest (ROI) reduces system energy dissipation by up to 84.2% compared to the state-of-the-art embedded media processors.