Automating ETL process with scripting technology

Abstract

Improving ETL process system data flows can provide better return on your Business investment. An enterprise level scheduling solution that is easy to use and handles heterogeneous environments may just do what you need. In the past, the focus was mainly on business process design, modeling. Recently, enterprises have realized that they can benefit tremendously from automating the ETL process with the objective of optimizing or improving them. In our research, we address the problem of warehousing, show how to automate the ETL process for pulling data from source to target environment. Typically, the source data consists of event streams that indicate changes in the business process state i.e progression of the process. The idea is to automate ETL process using scripting technologies. This paper is written to give scope in future for coming up with effective ETL tools that support command based automation or script based automation. We throw light on the need for usage of scripting technologies to automate ETL tools processing end to end process which reduce manual headache of running ETL process handling and also lead to development of ETL tools in future that support command based interface.