Incremental Refresh Research Articles

Incremental Load in a Data Warehousing Environment

Incremental load is an important factor for successful data warehousing. Lack of standardized incremental refresh methodologies can lead to poor analytical results, which can be unacceptable to an organization’s analytical community. Successful data warehouse implementation depends on consistent metadata as well as incremental data load techniques. If consistent load timestamps are maintained and efficient transformation algorithms are used, it is possible to refresh databases with complete accuracy and with little or no manual checking. This paper proposes an Extract-Transform-Load (ETL) metadata model that archives load observation timestamps and other useful load parameters. The author also recommends algorithms and techniques for incremental refreshes that enable table loading while ensuring data consistency, integrity, and improving load performance. In addition to significantly improving quality in incremental load techniques, these methods will save a substantial amount of data warehouse systems resources.

XML view materialization with deferred incremental refresh : the case of a restricted class of views

A view mechanism can provide a user with an appropriate portion of a database through data filtering and aggregation. Views are often materialized for query performance improvement, and in that case, their consistency needs to be maintained against updates of the underlying data. They can be either recomputed or incrementally refreshed by reflecting only the relevant updates. With the emergence of XML as the standard for data exchange on the Web, active research is under way on efficient storing and querying of XML documents with the DBMS. In this paper, we investigate the materialization of XML views and their incremental refresh for the case of a restricted class of views. The object-relational DBMS is employed to store XML documents and their materialized views, and the update log is used for deferred view refresh. Algorithms for checking a logged update's relevance to a view and for generating the optimized SQL statements to refresh the materialized view stored in an object-relational database through scanning of the update log are described. Experimental results show that our approach can be very effective in providing views of a large-scale XML warehouse on the Web.

XML 실체뷰 갱신 기법의 성능 평가

Materialized views have received much attention for query performance improvement. They need to be refreshed whenever their underlying data sources are updated. They could be recomputed from scratch or they could be incrementally refreshed by reflecting only those portions of updates that affect them. With emergence of XML as the standard for data exchange on the Web, active research is under way on effectively storing and retrieving XML documents. In this paper, we describe a performance study on the incremental refresh of XML materialized views for the case where XML documents are stored in a relational DBMS, and XML materialized views are maintained with incremental refresh. We describe implementation of a prototype XML storage system that supports XML materialized views and their incremental refresh, and report the performance results obtained with the implemented system through a detailed set of experiments on the incremental refresh of XML materialized views. The results show that the XML view maintenance with incremental refresh outperforms the ordinary view recomputation.

How to roll a join

Incremental refresh of a materialized join view is often less expensive than a full, non-incremental refresh. However, it is still a potentially costly atomic operation. This paper presents an algorithm that performs incremental view maintenance as a series of small, asynchronous steps. The size of each step can be controlled to limit contention between the refresh process and concurrent operations that access the materialized view or the underlying relations. The algorithm supports point-in-time refresh, which allows a materialized view to be refreshed to any time between the last refresh and the present.