A multi-modal sensor analysis framework for in-car mobile applications

Abstract

In-car applications developed using mobile sensors generally employ single-modal sensor analysis, therefore obtained results remain limited. Taking advantage of the advanced sensing and computing capabilities of smart phones, this study proposes a framework for the development of signal and image processing algorithms and applications utilizing multi-modal sensor analysis on in-car mobile sensors. An abstraction for synchronized and fast access to sensor readings as well as a plugin based multi-modal analysis interface for signal and image processing applications are provided within this framework. An in-car mobile application is also developed as part of this study, which detects the defects on the road, such as the potholes, speed bumps, and automatically extracts the video section and image of the corresponding road segment. More than 80% successful bump detection and image extraction is obtained in the performed tests and the advantages of multi-modal approach are exhibited.