Distributed processing of big mobility data as spatio-temporal data streams

Abstract

Recent rapid development of wireless communication, mobile computing, global navigation satellite systems (GNSS), and spatially enabled sensors are leading to an exponential growth of available mobility data produced continuously at high speed. Due to these advancements, a new class of monitoring applications has come to the focus, including real-time intelligent transportation systems, traffic monitoring and mobile objects tracking. These new information flow processing (IFP) application domains need to process huge volume of mobility data arriving in the form of continuous data streams from mobile objects. IFP applications are pushing traditional database technologies beyond their limits due to their massively increasing data volumes and demands for real-time processing. Mobility data, i.e. real-time, transient, time-varying sequences of spatio-temporal data items, generated by embedded positioning sensors demonstrates at least two Big Data core features: volume and velocity. Existing distributed data stream management systems (DSMS), real-time computing systems (RTCS) and their processing models are dominantly based on relational paradigm and continuous operator model. Thus, they have rudimentary spatio-temporal capabilities, provide expensive fault recovery requiring either hot replication or long recovery times, and do not handle faults and slow nodes. The framework proposed in this paper is a cornerstone towards efficient real-time managing and monitoring of mobile objects through distributed spatio-temporal streams processing on large clusters. A prototype implementation is rooted in a new stream processing model that overcomes the challenges of current distributed stream processing models and enable seamless integration with batch and interactive processing like MapReduce.