Views In Data Warehouse Research Articles

Generating multidimensional schemata from relational aggregation queries

Designing a multidimensional model is a non-trivial task: the requirements collected from senior managers can be inaccurate and difficult to use, especially when they are not database practitioners. On the other hand the pressing need for data warehousing comes from the fact that managers submit analytical summary queries against operational relational databases. In this paper, we view these analytical queries as a more reliable basis for multidimensional schema design. We first study how to use such queries to automatically generate measures, dimensions and dimension hierarchies and their representation in a star schema. We then use the results studying the schema evolution problem: If a new query Q′ can’t be answered by the schema previously built, how to develop a new schema from the old one so that query Q′ can be answered in the new schema. Finally, we consider rewrite OLAP queries on conventional database (multidimensional data warehouse) using materialized views in data warehouse. In our method we analyze the attributes from the given relational queries and the dependency relationships among them to design a schema in which each dimension is a lattice. The schema can answer the queries that it was generated from and the number of dimensions is minimal. Furthermore, the schema can answer many more queries that are similar to the given ones.

Materialized Cube Selection Using Particle Swarm Optimization Algorithm

Storage of pre-computed views in data warehouse can essentially reduce query processing cost for decision support queries. The problem is to choose an optimal set of materialized views. Various frameworks such as lattice, MVPP and AND-OR graphs and algorithms like heuristic based, greedy, stochastic algorithm have been proposed in the literature for materialized view selection. Heuristic and greedy algorithms become slower in high dimensional search space while stochastic algorithms do not guarantee global optimal solution but reach to the optimum most solution in a fast and efficient way. In this paper we have implemented Particle Swarm Optimization (PSO) algorithm, one of the stochastic algorithm, on lattice framework to select an optimal set of views for materialization in data warehouse by minimizing query processing cost. We have compared our results with Genetic algorithm to prove the effectiveness of PSO algorithm over genetic algorithm.

Materialized View Selection Based on Adaptive Genetic Algorithm and Its Implementation with Apache Hive

AbstractFrequently accessed views in data warehouses are usually materialized in order to accelerate the speed of querying big data. However, the view materialization itself incurs huge costs. Moreover, some latest products of non-traditional data warehouse software, such as Apache Hive, still lack the support of ma- terialized views. In order to select the appropriate views to be materialized with the possible minimized cost, we propose a novel approach to the materialized view selection problem based on an adaptive ge- netic algorithm. We establish a cost model that integrates the query, maintenance and storage costs to evaluate the performance of approaches and measure the fitness of an individual in the genetic algorithm. In addition, we introduce the adjustable factors for crossover probability and mutation probability, allow- ing the genetic algorithm to run quickly and avoid premature convergence. We also conduct extensive experiments for its implementation with Apache Hive, which query and manage l...

An efficient method for maintaining data cubes incrementally

The data cube operator computes group-bys for all possible combinations of a set of dimension attributes. Since computing a data cube typically incurs a considerable cost, the data cube is often precomputed and stored as materialized views in data warehouses. A materialized data cube needs to be updated when the source relations are changed. The incremental maintenance of a data cube is to compute and propagate only its changes, rather than recompute the entire data cube from scratch. For n dimension attributes, the data cube consists of 2 n group-bys, each of which is called a cuboid. To incrementally maintain a data cube with 2 n cuboids, the conventional methods compute 2 n delta cuboids, each of which represents the change of a cuboid. In this paper, we propose an efficient incremental maintenance method that can maintain a data cube using only a subset of 2 n delta cuboids. We formulate an optimization problem to find the optimal subset of 2 n delta cuboids that minimizes the total maintenance cost, and propose a heuristic solution that allows us to maintain a data cube using only n ⌈ n / 2 ⌉ delta cuboids. As a result, the cost of maintaining a data cube is substantially reduced. Through various experiments, we show the performance advantages of the proposed method over the conventional methods. We also extend the proposed method to handle partially materialized cubes and dimension hierarchies.

A Hybrid Approach for Data Warehouse View Selection

Materialized view selection is one of the crucial decisions in designing a data warehouse for optimal efficiency. Static selection of views may materialize certain views that are not beneficial as the data and usage trends change over time. On the contrary, dynamic selection of views works better only for queries demanding a high degree of aggregation. These facts point to the need for a technique that combines the improved response time of the static approach and the automated tuning capability of the dynamic approach. In this article, we propose a hybrid approach for the selection of materialized views. The idea is to partition the collection of all views into a static and dynamic set such that views selected for materialization from the static set are persistent over multiple query (and maintenance) windows, whereas views selected from the dynamic set can be queried and/or replaced on the fly. Highly aggregated views are selected on the fly based on the query access patterns of users, whereas the more detailed static set of views plays a significant role in the efficient maintenance of the dynamic set of views and in answering certain detailed view queries. We prove that our proposed strategy satisfies the monotonicity requirements, which is essential in order for the greedy heuristic to deliver competitive solutions. Experimental results show that our approach outperforms Dynamat, a well-known dynamic view management system that is known to outperform optimal static view selection.

Multiagent Immediate Incremental View Maintenance for Data Warehouses

Data warehouse systems typically designate downtime for view maintenance, ranging from tens of minutes to hours depending on the system size. We develop a multiagent system that achieves immediate incremental view maintenance (IIVM) for continuous updating of data warehouse views. We describe an IIVM system that processes updates as transactions are executed at the underlying data sources to eliminate view maintenance downtime for the data warehouse-a crucial requirement for internet applications. The use of a multiagent framework provides considerable process speed improvement when compared with other IIVM systems. Since agents are used to delegate the data sources and warehouse views, it is easy to reorganize the components of the system. Through the use of cooperative agents, the data consistency of IIVM can be easily maintained. The test results from this research show that the proposed system increases the availability of the data warehouse while preserving a stringent requirement on data consistency.

삽입/삭제연산의 구분을 통한 실체뷰 갱신시간의 단축정책

데이터웨어하우스내에서 실체뷰의 개수가 많을수록 사용자의 질의요구를 실체뷰내에서 처리할 수 있는 확률이 높아지므로 신속한 응담이 가능하다. 그러나, 실체뷰가 많아지면 전체적인 실체뷰 갱신시간이 길어진다. 실체뷰 갱신작업 동안에는 데이터웨어하우스로의 접근이 통제되기 때문에 사용자의 질의요구를 처리할 수 없다. 따라서, 실체뷰의 갱신시간을 단축시킬 수 있는 알고리즘을 제안한다. 실험결과, 삽입/삭제 구분을 위한 오버헤드를 감안하더라도 제안한 알고리즘의 성능이 기존 기법에 비하여 우수함을 알 수 있었다. More views in data warehouse, can respond to the users more rapidly because the user's requests may be processed by utilizing only the materialized views with higher probabilities rather than accessing base relations. But, more views cause longer update duration of all the materialized views. During the update time, queries of users can not be processed because accesses to the datawarehouse are blocked. Thus, the update durations of the materialized views are necessary to be reduced. In this paper, we propose algorithm the update duration of the materialized views owing to managing by distinguishing insecton/deletion operations. Though there might overheads for distinguishing insertion/deletion, we concluded that the proposed algorithm could be excellent than others.

Finding an efficient rewriting of OLAP queries using materialized views in data warehouses

OLAP queries involve a lot of aggregations on a large amount of data in data warehouses. To process expensive OLAP queries efficiently, we propose a new method to rewrite a given OLAP query using various kinds of materialized views which already exist in data warehouses. We first define the normal forms of OLAP queries and materialized views based on the selection and aggregation granularities, which are derived from the lattice of dimension hierarchies. Conditions for usability of materialized views in rewriting a given query are specified by relationships between the components of their normal forms. We present a rewriting algorithm for OLAP queries that can effectively utilize materialized views having different selection granularities, selection regions, and aggregation granularities together. We also propose an algorithm to find a set of materialized views that results in a rewritten query which can be executed efficiently. We show the effectiveness and performance of the algorithm experimentally.

Detecting redundant materialized views in data warehouse evolution

A data warehouse (DW) can be abstractly seen as a set of materialized views defined over a set of remote data sources. A DW is intended to satisfy a set of queries. The views materialized in a DW relate to each other in a complex manner, through common subexpressions, in order to guarantee high query performance and low view maintenance cost. DWs are time varying. As time passes new materialized views are added in order to satisfy new queries, or for performance reasons, while old queries are dropped. The evolution of a DW can result in a redundant set of materialized views. In this paper, we address the problem of detecting redundant materialized views in a given DW view selection, that is, materialized views that can be removed from DW without negatively affecting the query evaluation or the view maintenance process. Using an AND/OR dag representation for multiple queries and views, we first formalize the process of propagating source relation changes to the materialized views by exploiting common subexpressions between views and by using other materialized views that are not affected by these changes. Then, we provide an algorithm for detecting materialized views that are not needed in the process of propagating source relation changes to the DW. We also show how trivially redundant views can be identified in this process. Finally, we use these results to provide a procedure for detecting materialized views that are redundant in a DW. Our approach considers a broad class of views that includes grouping/aggregation views and is not dependent on a specific cost model.

Efficient materialization and use of views in data warehouses

Given the complexity of many queries over a Data Warehouse (DW), it is interesting to precompute and store in the DW the answer sets of some demanding operations, so called materialized views. In this paper, we present an algorithm, including its experimental evaluation, which allows the materialization of several views simultaneously without losing sight of processing costs for queries using these materialized views.