Guided Learning of Human Sensor Models with Low-Level Grounding

Abstract

Sensor data often lacks intuitive interpretability in its raw form, unlike language or image data. Furthermore, standard end-to-end training leaves little control over local representation learning. We postulate that guided local representation learning could be used to tackle both issues. In this paper we introduce a novel framework for sensor models which uses low-level grounding for guided learning of human sensor models. Our framework is amenable to different model architecture. We demonstrate our method on two different human activity datasets, one containing labels of low-level actions used in performing high-level activities, and one without any low-level labeling. We provide comprehensive analysis of our framework’s performance across many low-level action subsets and demonstrate how it can be easily adapted to data with no low-level labeling. Our results demonstrate that low-level grounding can be used to improve both the interpretability and performance of sensor models.