4 - Data Warehousing and Online Analytical Processing

Abstract

This chapter presents an overview of data warehouse and online analytical processing (OLAP) technology. This overview is essential for understanding the overall data mining and knowledge discovery process. Data warehouses generalize and consolidate data in multidimensional space. The construction of data warehouses involves data cleaning, data integration, and data transformation, and can be viewed as an important preprocessing step for data mining. Moreover, data warehouses provide OLAP tools for the interactive analysis of multidimensional data of varied granularities, which facilitates effective data generalization and data mining. Many other data mining functions, such as association, classification, prediction, and clustering, can be integrated with OLAP operations to enhance interactive mining of knowledge at multiple levels of abstraction. Hence, the data warehouse has become an increasingly important platform for data analysis and OLAP and will provide an effective platform for data mining. Therefore, data warehousing and OLAP form an essential step in the knowledge discovery process. The data cube, a multidimensional data model for data warehouses and OLAP, as well as OLAP operations such as roll-up, drilldown, slicing, and dicing are focused on. Data warehouse design and usage are also discussed followed by a discussion of Multidimensional data mining, a powerful paradigm that integrates data warehouse and OLAP technology with that of data mining. An overview of data warehouse implementation examines general strategies for efficient data cube computation, OLAP data indexing, and OLAP query processing. Finally, data generalization by attribute-oriented induction is studied. This method uses concept hierarchies to generalize data to multiple levels of abstraction.