Warehouse Applications Research Articles

A review on stochastic models and analysis of warehouse operations

This paper provides an overview of stochastic research in warehouse operations. We identify uncertainty sources of warehousing systems and systematically present typical warehouse operations from a stochastic system viewpoint. Stochastic modeling methods and analysis techniques in existing literature are summarized, along with current research limitations. Through a comparison between potential and existing stochastic warehouse applications, we identify potential new research applications. Furthermore, by comparing potential and existing solution methods, methodological directions relevant to practice and largely unexplored in warehouse literature are identified.

A unit test approach for database schema evolution

Context The constant changes in today’s business requirements demand continuous database revisions. Hence, database structures, not unlike software applications, deteriorate during their lifespan and thus require refactoring in order to achieve a longer life span. Although unit tests support changes to application programs and refactoring, there is currently a lack of testing strategies for database schema evolution. Objective This work examines the challenges for database schema evolution and explores the possibility of using various testing strategies to assist with schema evolution. Specifically, the work proposes a novel unit test approach for the application code that accesses databases with the objective of proactively evaluating the code against the altered database. Method The approach was validated through the implementation of a testing framework in conjunction with a sample application and a relatively simple database schema. Although the database schema in this study was simple, it was nevertheless able to demonstrate the advantages of the proposed approach. Results After changes in the database schema, the proposed approach found all SELECT statements as well as the majority of other statements requiring modifications in the application code. Due to its efficiency with SELECT statements, the proposed approach is expected to be more successful with database warehouse applications where SELECT statements are dominant. Conclusion The unit test approach that accesses databases has proven to be successful in evaluating the application code against the evolved database. In particular, the approach is simple and straightforward to implement, which makes it easily adoptable in practice.

THE ENHANCEMENT OF ODL STUDENT RECRUITING CAMPAIGN WITH DATA WAREHOUSE DEVELOPMENT AND DATA MINING TECHNIQUES

The purpose of this research was to develop an OLAP cube data warehouse, and, using data mining techniques, to support the university's public relations, admissions, and planning divisions in the efficient recruiting of students by surveying, through interviews; the opinions of management and operational personnel, and through documents; the attributes in application forms and annual reports. User requirements, source data and systems were all examined. The data warehouse and front-end applications developed are described below. 1. Student Data Warehouse—this repository was designed to store students' historical data and to facilitate analysis and reporting following the user requirements. Students' historical data including demographic data from 2001-2005 were extracted, loaded and transformed from source systems, then they were cleaned before uploading to the data warehouse using star schema. 2. OLAP Cub—this 122 multidimensional structure enables users to analyze the students' demographic data in many dimensions such as “Number of Registered Students in each year by Semester, Major, School, Gender, Occupation, Region, etc.” Predefined reports were created and published to an intranet and users were able to create ad-hoc reports through web browsers as well as XLAddin. 3. Data Mining—this technique finds hidden knowledge and patterns in ODL student data supporting decision making, using three algorithms: Naïve Bayes, Clustering and Association Rules. Occupation of students is the strongest factor influencing students' choices of Schools. Students' demographic data can be clustered into groups with similar or dissimilar characteristics such as “Single, Unemployed, Low Income (<3,000 Baht)” or “Married, Male, Studying Law, High Income”, and can generate rules from frequently occurring cases such as “Occupation=Teacher-Lecturer (private sector), Marital Status=Single > School=School of Educational Studies” or “Occupation=Police, Marital Status=Single -> School=School of Law”. The results from the study indicated that users were satisfied using information and applications from the data warehouse, OLAP cube and data mining techniques which enable the university to reduce costs and to reach the desired enrolment target effectively.

Semantics-Aware Advanced OLAP Visualization of Multidimensional Data Cubes

Efficiently supporting advanced OLAP visualization of multidimensional data cubes is a novel and challenging research topic, which results to be of interest for a large family of data warehouse applications relying on the management of spatio-temporal (e.g., mobile) data, scientific and statistical data, sensor network data, biological data, etc. On the other hand, the issue of visualizing multidimensional data domains has been quite neglected from the research community, since it does not belong to the well-founded conceptual-logical-physical design hierarchy inherited from relational database methodologies. Inspired from these considerations, in this article we propose an innovative advanced OLAP visualization technique that meaningfully combines (i) the so-called OLAP dimension flattening process, which allows us to extract two-dimensional OLAP views from multidimensional data cubes, and (ii) very efficient data compression techniques for such views, which allow us to generate “semantics-aware” compressed representations where data are grouped along OLAP hierarchies.

Making snapshot isolation serializable

Snapshot Isolation (SI) is a multiversion concurrency control algorithm, first described in Berenson et al. [1995]. SI is attractive because it provides an isolation level that avoids many of the common concurrency anomalies, and has been implemented by Oracle and Microsoft SQL Server (with certain minor variations). SI does not guarantee serializability in all cases, but the TPC-C benchmark application [TPC-C], for example, executes under SI without serialization anomalies. All major database system products are delivered with default nonserializable isolation levels, often ones that encounter serialization anomalies more commonly than SI, and we suspect that numerous isolation errors occur each day at many large sites because of this, leading to corrupt data sometimes noted in data warehouse applications. The classical justification for lower isolation levels is that applications can be run under such levels to improve efficiency when they can be shown not to result in serious errors, but little or no guidance has been offered to application programmers and DBAs by vendors as to how to avoid such errors. This article develops a theory that characterizes when nonserializable executions of applications can occur under SI. Near the end of the article, we apply this theory to demonstrate that the TPC-C benchmark application has no serialization anomalies under SI, and then discuss how this demonstration can be generalized to other applications. We also present a discussion on how to modify the program logic of applications that are nonserializable under SI so that serializability will be guaranteed.

Task-Technology Fit in Data Warehousing Environments: Analyzing the Factors that Affect Utilization

Enterprise data warehouses can be an expensive investment in both time and resources. The promise of data warehouses is improved decision making through user empowerment and enablement of sophisticated decision support tools. Have organizations been able to achieve this goal? The theory of tasktechnology fit defines a model that suggests that for a technology to be utilized it must meet the needs of a user. When the technology meets the users’ needs and provides features that support the fit of the requirements of the task, performance impacts will result. When an organization commits the time and resources necessary to develop an enterprise data warehouse, their expectation is that there will be a high tasktechnology fit and that performance impacts will result in an improved organization. This study extends prior task-technology fit research to provide an evaluation of task-technology fit in data warehouse applications. The focus of the study is to examine the factors that contribute to the task-technology fit specifically within data warehouse environments. Six organizations that have implemented a data warehouse were surveyed. The sample population consisted of users of large-scale data warehouses within the insurance industry.

SQL Extensions to Enable Decision Support within Database Systems

Modern organizations store massive amount of information within their data warehouses to carry out complex analysis and decision support, such as identifying interesting trends, making unusual patterns to stand out and verifying hypotheses. Besides traditional data warehousing applications, a new class of data-intensive systems is emerging: applications in which the data is modeled not as persistent relations but rather as transient data streams. In this article we approach decision support from the database viewpoint. We briefly mention SQL features – the standard query language of database systems – and review several proposed extensions to make it a decision support-oriented language. Special emphasis is given to EMF SQL, an approach that has proven very useful in complex querying over traditional data ware-housing applications. We argue that this extension can also be useful in the context of stream data. Finally, we review and discuss decision support challenges over data stream systems.

Dynamic query scheduling in parallel data warehouses

AbstractParallel processing is a key to high performance in very large data warehouse applications that execute complex analytical queries on huge amounts of data. Although parallel database systems (PDBSs) have been studied extensively in the past decades, the specifics of load balancing in parallel data warehouses have not been addressed in detail.In this study, we investigate how the load balancing potential of a Shared Disk (SD) architecture can be utilized for data warehouse applications. We propose an integrated scheduling strategy that simultaneously considers both processors and disks, regarding not only the total workload on each resource but also the distribution of load over time. We evaluate the performance of the new method in a comprehensive simulation study and compare it to several other approaches. The analysis incorporates skew aspects and considers typical data warehouse features such as star schemas with large fact tables and bitmap indices. Copyright © 2003 John Wiley & Sons, Ltd.

Understanding why marketing does not use the corporate data warehouse for CRM applications

This study uses a previously validated model to explore the experiences of one large regional US health care payer in its implementation of a corporate data warehouse to support the marketing function via customer relationship management (CRM) applications. The organisation invested significant time and other valuable resources on its data warehouse applications but perceived that the warehouse was underutilised by the marketing function. Current estimates indicate that as many as 50–70 per cent of CRM implementations have failed and the figures for data warehouse applications are not much better. Using a case study methodology and independent coders, the authors analysed transcripts from five focus group sessions that they conducted with 21 marketing, information systems and strategic managers at a large regional healthcare payer. Their findings suggest that the ability of marketing to use the system for its functional needs, various aspects of information quality and the internal training were factors that ranked highly in terms of significance to implementation success. In this case, the corporate data warehouse and associated interface often failed to meet users' expectations.

A fuzzy approach to build an intelligent data warehouse

Data warehousing is emerging as one of the hottest areas of growth in global business and the critical component and tool for business intelligence. In the data warehouse environment, the level of summarization plays an important role. Some Data Warehouse applications, directly account for the uncertainty and vagueness often associated with time-series and enterprise-wide decision making. In this paper, we enhance the concept of summarization using fuzzy technology, keeping in view the time-series and enterprise-wide data. This enhancement, which extends the aggregation process used in data warehouse, provides insightful information intelligently and allows greater facility for modeling human decision making.

Query optimization for selections using bitmaps

Bitmaps are popular indexes for data warehouse (DW) applications and most database management systems offer them today. This paper proposes query optimization strategies for selections using bitmaps. Both continuous and discrete selection criteria are considered. Query optimization strategies are categorized into static and dynamic. Static optimization strategies discussed are the optimal design of bitmaps, and algorithms based on tree and logical reduction. The dynamic optimization discussed is the approach of inclusion and exclusion for both bit-sliced indexes and encoded bitmap indexes.

Data integration and warehousing in Telecom Italia

We discuss the main methodological and technological issues arosen in the last years in the development of the enterprise integrated database of Telecom Italia and, subsequently in the management of the primary data store for Telecom Italia data warehouse applications. The two efforts, although driven by different needs and requirements can be regarded as a continous development of an integrated view of the enterprise data. We review the experience accumulated in the integration of over 50 internal databases, highlighting the benefits and drawbacks of this scenario for data warehousing and discuss the development of a large dedicated data store to support the analysis of data about customers and phone traffic.

Maintenance of data cubes and summary tables in a warehouse

Data warehouses contain large amounts of information, often collected from a variety of independent sources. Decision-support functions in a warehouse, such as on-line analytical processing (OLAP), involve hundreds of complex aggregate queries over large volumes of data. It is not feasible to compute these queries by scanning the data sets each time. Warehouse applications therefore build a large number of summary tables , or materialized aggregate views, to help them increase the system performance. As changes, most notably new transactional data, are collected at the data sources, all summary tables at the warehouse that depend upon this data need to be updated. Usually, source changes are loaded into the warehouse at regular intervals, usually once a day, in a batch window, and the warehouse is made unavailable for querying while it is updated. Since the number of summary tables that need to be maintained is often large, a critical issue for data warehousing is how to maintain the summary tables efficiently. In this paper we propose a method of maintaining aggregate views (the summary-delta table method), and use it to solve two problems in maintaining summary tables in a warehouse: (1) how to efficiently maintain a summary table while minimizing the batch window needed for maintenance, and (2) how to maintain a large set of summary tables defined over the same base tables. While several papers have addressed the issues relating to choosing and materializing a set of summary tables, this is the first paper to address maintaining summary tables efficiently.

Potential warehouse applications of industrial robots

The Third National Industrial Robot Conference and Exposition in Chicago, USA in November, 1978 indicated a lack of attention to warehousing applications and points up the need for coverage of this increasingly important subject for the benefit of those who may not be in a position to determine whether there may be valid applications in their own operations.

Design, Construction, and Performance of Slabs-on-Grade for an Industry

This paper presents the existing design and construction procedures for slabs-on-Grade for industrial warehouse applications. Design practices are included which have been succuessfully used for years. The paper includes information on the base and subgrade, on the concrete slab itself, on the construction details with emphasis on the integral footing, and comments on the performance of the resultant product. The basic theory and concepts of its use are included. Procedures that are presented are not proposed procedures. They are in use now and have a history of successful application. /Author/