StreamDB: A Unified Data Management System for Service-Based Cloud Application

Abstract

Current data management systems are mainly divided into two categories: Database Management System (DBMS) and Data Stream Management System (DSMS). The increasing use of streaming analysis in modern service-based cloud applications has created an arms race among DBMS vendors to offer ever more sophisticated in-database streaming support, which requires handling the volume, variety, velocity and variability of fast data collections. Unfortunately, current solutions either only provide limited streaming analysis capacity and horizontal scalability (classic RDBMS) or trade off transaction processing for other properties (NoSQL DBMS), leading to the curse of no one size fits all for DBMS. In this paper, we argue that transaction processing is a relevant concept for DSMS. As a first step toward One Size Fits All Data Management System, we present StreamDB, which integrates transaction processing in DSMS as opposed to extending DBMS to support streams. First, we describe how StreamDB processes transactions in a streaming environment, then we compare our approach with traditional in-memory DBMS on typical transactional benchmarks. Our results show that StreamDB is advantageous in terms of throughput, scalability, and latency. Finally, we argue that the ideas present here provide insight on the development of next-generation data management systems and motivate further study of the challenges inherent in unifying DBMS and DSMS.