Analysis of Change Data Capture Method in Heterogeneous Data Sources to Support RTDW

Abstract

The need for rapid decision-making on the organization, causing the importance of developing a real time data warehouse (RTDW) system. In addition, what needs to be considered is how to keep the Extract, Transform, Load (ETL) process done, would not interfere the performance of the operational database being used. One of the methods that can be used is change data capture. Nevertheless, research that has been done so far only tests each method on one type of data source only. This leads to the quality of method that is claimed to be the best on one type of data source being tested, not necessarily running equally well on different types of data sources. Nowadays, the data sources that companies use for many operations are usually coming from different sources. The proposed research will identified which change data capture method is the best from three types of methods in three database application for each type of data source to be tested, that supports the RTDW system-making process. Then there will be a simulation of query execution test in a certain number and period, to test the methods and get the best quality comparison between various change data capture methods available. Finally, an integration scheme design will be created to build the data warehouse system needed to make strategic decisions with real time data and analysis of the results at the level of data structures to identify which variables influence the quality of change data capture method on certain types of data sources. From the results of research conducted proved that the best change data capture method is different for each type of data source tested in the same environment.