ETL Maturity Model for Data Warehouse Systems: A CMMI Compliant燜ramework

Abstract

The effectiveness of the Business Intelligence (BI) system mainly depends on the quality of knowledge it produces. The decision-making process is hindered, and the user’s trust is lost, if the knowledge offered is undesired or of poor quality. A Data Warehouse (DW) is a huge collection of data gathered from many sources and an important part of any BI solution to assist management in making better decisions. The Extract, Transform, and Load (ETL) process is the backbone of a DW system, and it is responsible for moving data from source systems into the DW system. The more mature the ETL process the more reliable the DW system. In this paper, we propose the ETL Maturity Model (EMM) that assists organizations in achieving a high-quality ETL system and thereby enhancing the quality of knowledge produced. The EMM is made up of five levels of maturity i.e., Chaotic, Acceptable, Stable, Efficient and Reliable. Each level of maturity contains Key Process Areas (KPAs) that have been endorsed by industry experts and include all critical features of a good ETL system. Quality Objectives (QOs) are defined procedures that, when implemented, resulted in a high-quality ETL process. Each KPA has its own set of QOs, the execution of which meets the requirements of that KPA. Multiple brainstorming sessions with relevant industry experts helped to enhance the model. EMM was deployed in two key projects utilizing multiple case studies to supplement the validation process and support our claim. This model can assist organizations in improving their current ETL process and transforming it into a more mature ETL system. This model can also provide high-quality information to assist users in making better decisions and gaining their trust.