The stream data warehouse: Page replacement algorithms and quality of service metrics

Abstract

The stream data warehouse is an answer to the rapidly changing world of data analysis, which demands reliable and up-to-date results, obtained in a near real-time manner. Therefore it is a subject of recent research involving such areas as continuous updates and low-latency response, for example. In this paper, we study the stream adaptation of the OLAP cube and, in particular — its memory paging mechanism. It is driven by the page replacement algorithm, which manages the efficient data transfer and thus supplies users with constantly updatable data cubes. The following paper introduces an entirely novel approach to this topic. By perceiving the page replacement process as a multi-objective optimization problem, we propose three new algorithms that constantly analyze their varying environment and adapt to those changes by adjusting their behavior. Moreover, they consider user-provided constraints, which impose maximal values of specific parameters that cannot be exceeded. In addition to the page replacement algorithms, we propose two distinct quality of service metrics that measure the overall efficiency of data transfer inside the stream data warehouse. In order to verify and compare the new algorithms with their older counterparts, a series of experiments were conducted. Their results have confirmed that the proposed algorithms meet their requirements and visibly outperform the original solutions. The average wait time decreased between 25% and 66% (from 1.3x to 3.0x respectively, depending on the chosen algorithm), whereas the peak wait time decreased by approximately 99% (between 100x and 190x respectively).