HCache: A Hash-based Hybrid Caching Model for Real-Time Streaming Data Analytics

Abstract

Up-to-date results in data stream analytics are difficult to obtain because the data are in rapid sequence and can be accessed only once. Due to the exactly-once delivery nature of streaming data, online processing is quite unstable. To guarantee comprehensive and accurate results, aggregating historical data is essential when processing streaming data. In this paper, we propose a hash-based hybrid cache model, namely, <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">HCache</i> , for fast data analytics covering real-time streaming data and historical data. The <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">HCache</i> model integrates the online cache and batch cache for hybrid online and batch processing and uses a hash structure to accelerate storage. When executing analytic tasks, the batch cache and online cache are accessed in parallel. Computed streaming data are stored in the online cache, which returns qualified results based on one-time visiting. The most recently visited historical data are stored in the batch cache, and they are also used to correct errors in the online cache. Efficient replacement strategies are used to keep the caches within a relatively stable size. To coordinate the online cache with the batch cache, an LRU-based selection strategy is designed to achieve comprehensive results. Experimental results show that the <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">HCache</i> model can quickly and efficiently execute analytic tasks with little additional overhead; moreover <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">HCache</i> is more stable and effective at data storage, access and query with less memory utilization than other models.