464 3D head visualization for mapping and tracking dermatological conditions

Abstract

Tracking or quantifying changes in hair and skin conditions over time is a task usually left up to the clinician: the affected areas (or images thereof) are either compared to the photos taken during past visits or a standard reference scale is used to compare the condition to a limited set of images or drawings to determine the severity level of the current state. While automated skin detection algorithms are gaining traction for binary diagnostic tasks or to segment areas of interest at a particular time-point, tracking changes automatically remains challenging due to the lack of easily accessible and affordable systems that can be used to quantitatively compare the changes in a standardized way. Focusing on tracking conditions that occur on the head, we present a new method to recover a clinically relevant model of a person’s head, defined as the complete 3D head surface in the absence of hair volume, starting only with a video taken from a single hand-held camera. Using techniques from computer vision, more specifically structure-from-motion and multi-view stereo, we determine first the shape of each person's head and then the alignment of the fitted 3D head for all video frames to recover texture mapping information, irrespective of the person's pose. This alignment is then used to map and visualize hair or skin information, for example disease quantifications, onto the head model for tracking changes in the condition over time. We demonstrate that our approach recovers a consistent geometry for varying head shapes, from videos taken by different people, with different smartphones, and in a variety of uncontrolled environments such as outdoors, living rooms, and hallways, and hence, can be applied to the clinical setting as well. Furthermore, we show how, once the head geometry for the person has been recovered, it can also be used in augmented reality compatible smartphones to guide image capture and map dermatological quantifications for integration in the clinic.