A rewrite/merge approach for supporting real-time data warehousing via lightweight data integration

Abstract

This paper proposes and experimentally assesses a rewrite/merge approach for supporting real-time data warehousing via lightweight data integration. Real-time data warehouses are becoming more and more relevant actually, due to emerging research challenges such as Big Data and Cloud Computing. Our contribution fulfills limitations of actual data warehousing architectures, which are no suitable to perform classical operations (e.g., loading, aggregation, indexing, OLAP query answering, and so forth) under real-time constraints. The proposed approach is based on intelligent manipulation of SQL statements of input queries, which are decomposed in suitable sub-queries (the rewrite phase) that are finally submitted as (final) input queries to an ad hoc component responsible for the cooperative query answering via a parallel query processing inspired method (the merge phase). This method induces in a novel data warehousing framework where the static phase is separated by the dynamic phase, in order to achieve the real-time processing features. We complete our analytical contributions by means of an extensive experimental campaign where we stress the performance of our proposed real-time data warehousing framework against a popular data warehouse benchmark, and in comparison with traditional architectures, which finally confirms the benefits deriving from our proposal.