CUPID: An efficient spatio-temporal data engine

Abstract

In the IoT era, abundant spatio-temporal data is generated from various devices thanks to the prevalence of positioning techniques. Due to a lack of effective systems to manipulate the data, advanced scalable and efficient data management systems are necessary to support more and more urban applications.We propose Cupid, which is powered by scholars from Chongqing University and Guangzhou Urban Planning and Design Survey Research Institute, an efficient spatio-temporal Data engine. It extends JUST by introducing many functionalities to make it more applicable to deployment, and can efficiently manage large-scale spatio-temporal data. In Cupid, Apache HBase is utilized as the storage, GeoMesa serves as the spatio-temporal data indexing tool, and Apache Spark acts as the execution engine. We introduce many optimizations to ensure usability and reduce computational overhead. To make Cupid easy to use, we design and implement a SQL-like query language. Furthermore, to deploy the system as a PaaS while speeding up the execution, we leverage Apache Livy, a service that enables easy interaction with a Spark cluster over REST interfaces, together with gRPC to effectively collect results. Extensive experiments illustrate that Cupid outperforms other state-of-the-art spatio-temporal big data management systems in terms of efficiency and scalability.