A 1 mJ/Frame Unified Media Application Processor With Dynamic Analog-Digital Mode Reconfiguration for Embedded 3D-Media Contents Processing

Abstract

In this paper, a unified media application processor (UMAP) is presented for 2D/3D image analysis/synthesis applications on handheld devices. UMAP integrates parallel and sequential processing layers which consist of heterogeneous functional IPs for general media contents processing on today's application processors (AP). Based on the heterogeneous many-core platform, UMAP supports not only graphics and vision processing for real-time augmented reality (AR) but also disparity estimation and 3D display synthesis for 3D-view AR acceleration. A new concept of 3D-view AR which synthesizes 3D display contents from two vertically aligned stereo images and a self-constructed disparity map is introduced to achieve true realism for next generation mobile devices. For low-cost 3D-view AR processing, a homography-based disparity estimation (HDE) algorithm is proposed to construct a disparity map between two stereo images with small implementation overhead. For real-time and energy-efficient system organization, workload-balanced 3-stage pipelined architecture and a mixed-mode feature extraction engine (FEE) are also implemented in UMAP. The 3-stage pipelined system which consists of graphics, vision, and display operation stages reduces per-frame execution latency, while dynamic analog/digital mode reconfiguration based on mixed-mode FEE reduces per-frame energy dissipation, so real-time energy-efficient 3D-view AR can be realized in UMAP. FEE performs high-speed corner detection for vision processing based on four pairs of analog current contention logics (CCLs). Especially, a diode-connected current sensing stabilizer (CSS) in each CCL reduces minimum sensing current for corner detection, so average power consumed in CCL is reduced by 44.9%. In 2D or 3D-view AR processing, FEE with four CCLs replaces the parallel processing core cluster which is the most power hungry IP in UMAP, so 96.7% of cluster power and 99.1% of target detection time are saved in real operation. Based on the 3-stage pipelined architecture with the dynamic mode reconfiguration technique, the entire UMAP achieves up to 64.4% of energy reduction compared to the previous state-of-the-art media processors in full operation.