Mozart: A Mobile ToF System for Sensing in the Dark Through Phase Manipulation

Abstract

Sensing in low-light and dark environments has a wide range of applications. However, existing sensing technologies suffer several major challenges, such as excessive noise and low resolution. In this work, we propose Mozart - a new mobile sensing system that leverages off-the-shelf Time-of-Flight (ToF) depth cameras to generate high-resolution and rich-in-texture maps for applications in dark scenarios. The design of Mozart is based on our key observation that the phase components of ToF measurements can be manipulated to expose texture information. Through in-depth analysis of the physical reflection model, we show that the textures can be exposed and enhanced using highly compute-efficient phase manipulation functions. Moreover, by exploiting the physics texture models, we propose an autoencoderbased unsupervised learning approach that can automatically learn efficient representations from phase components to generate high-resolution maps. We implemented Mozart on several Android smartphone models and an edge testbed with standalone ToF camera platforms for various applications in the dark. The results show that Mozart can work in real time and delivers significant improvement over existing sensing technologies. The demo video at https:// www.youtube.com/watch?v=L_sxyTZxIdU shows that Mozart offers a low-cost, highperformance sensing technology for nextgeneration applications in the dark.