Data Warehouse Development Research Articles

BEST PRACTICES FOR SUCCESSFUL DEVELOPMENT OF DATA WAREHOUSES FOR SMALL BUSINESSES

The movement to real-time is the latest development in business intelligence and data warehousing. In general terms, a data warehouse costs in excess of one million dollars and it is estimated that approximately 67% of the efforts fail. This is a significant factor when data warehouses are being developed for small businesses. Organizations are compelled to use their own data to produce reports that may indicate or better estimate tendencies in the market. This study examines six small businesses to determine best practices for creating data warehouses. Companies included in this analysis are in the fields of health care, manufacturing, education, and financial institutions. In this study, IT professionals and management were interviewed to determine which practices were highly valued in the development of data warehouses. In this paper, best practices for successful use of data warehouses are analyzed by examining six companies to determine the design, development and implementation.

Multidimensional Design Paradigms for Data Warehouses: A Systematic Mapping Study

Data warehouses (DW) must integrate information from the different areas and sources of an organization in order to extract knowledge relevant to decision-making. The DW development is not an easy task, which is why various design approaches have been put forward. These approaches can be classified in three different paradigms according to the origin of the information requirements: supply-driven, demand-driven, and hybrids of these. This article compares the methodologies for the multidimensional design of DW through a systematic mapping as research methodology. The study is presented for each paradigm, the main characteristics of the methodologies, their notations and problem areas exhibited in each one of them. The results indicate that there is no follow-up to the complete process of implementing a DW in either an academic or industrial environment; however, there is also no evidence that the attempt is made to address the design and development of a DW by applying and comparing different methodologies existing in the field.

Pembangunan Database Destinasi Pariwisata Indonesia dan Implementasinya pada Sistem Berbasis Web

Regarding to: (1) the increasing region’s need in developing tourism destinations; (2) the needs of tourists in selecting appropriate attractions according to specified criteria; (3) the need of travel businesses to offer sights of interest in accordance with the needs of potential tourists, (4) the need to deepen and continue our previous research titled "Development of Tourism Destination Media Potential and Utilizing Local Resources in the Era of Autonomy and Regional Expansion ", we need to develop a complete database of tourism destinations in Indonesia that can facilitate those needs. We build a web-based database that is capable of storing complete information about Indonesian tourism destinations in thorough, systematic, and structured way. It is also able to classify a variety of attractions based on attributes such as: location (the name of the island, province, district), type/ tourism products, how to achieve the object, cost, and a variety of informal information, such as the ins and outs of the attraction area incorporated by the local or tourist experiences. The research will focus on deepening and refinement of the model and database structure design and implementation with the collection, processing, and data entry of primary and secondary data which amounts to approximately 140 tourism destinations in Indonesia. The research is arranged in stages as follows: (1) designing models and the database structure, (2) making a web-based program, (3) installation and hosting ; (4) data collection, (5) data processing and data entry, (6) evaluation and improvement/ refinement. Once developed, the database can be used as a starting point in the development of Data Warehouse, Decision Support System, and Expert System for Indonesian tourism industry.

Adding semantic modules to improve goal-oriented analysis of data warehouses using I-star

The success rate of data warehouse (DW) development is improved by performing a requirements elicitation stage in which the users’ needs are modeled. Currently, among the different proposals for modeling requirements, there is a special focus on goal-oriented models, and in particular on the i* framework. In order to adapt this framework for DW development, we previously developed a UML profile for DWs. However, as the general i* framework, the proposal lacks modularity. This has a specially negative impact for DW development, since DW requirement models tend to include a huge number of elements with crossed relationships between them. In turn, the readability of the models is decreased, harming their utility and increasing the error rate and development time. In this paper, we propose an extension of our i* profile for DWs considering the modularization of goals. We provide a set of guidelines in order to correctly apply our proposal. Furthermore, we have performed an experiment in order to assess the validity our proposal. The benefits of our proposal are an increase in the modularity and scalability of the models which, in turn, increases the error correction capability, and makes complex models easier to understand by DW developers and non expert users.

Development of an anesthesia data warehouse: Preliminary results

Hospital information system manages patient's hospitalization information across different applications and databases. As statistics are performed with difficulties on these different databases, the university hospital of Lille developed an anesthesia data warehouse. This common structure stores data related to anesthesia procedures and patient hospital stay. In that way, the joint analysis on intervention's events and patient's outcome is possible. However, data quality remains one of the main issues in this kind of project. Indeed, errors in patient identifier result in difficulties to link data between the different sources. This problem will be approached in the next phase of the project.

Development of Secure XML Data Warehouses with QVT

ContextData warehouses are systems which integrate heterogeneous sources to support the decision making process. Data from the Web is becoming increasingly more important as sources for these systems, which has motivated the extensive use of XML to facilitate data and metadata interchange among heterogeneous data sources from the Web and the data warehouse. However, the business information that data warehouses manage is highly sensitive and must, therefore, be carefully protected. Security is thus a key issue in the design of data warehouses, regardless of the implementation technology. It is important to note that the idiosyncrasy of the unstructured and semi-structured data requires particular security rules that have been specifically tailored to these systems in order to permit their particularities to be captured correctly. Unfortunately, although security issues have been considered in the development of traditional data warehouses, current research lacks approaches with which to consider security when the target platform is based on XML technology. ObjectiveWe shall focus on defining transformations to obtain a secure XML Schema from the conceptual multidimensional model of a data warehouse. MethodWe have first defined the rationale behind the transformation rules and how they have been developed in natural language, and we have then established them clearly and formally by using the QVT language. Finally, in order to validate our proposal we have carried out a case study. ResultsWe have proposed an approach for the model driven development of Secure XML Data Warehouses, defining a set of QVT transformation rules. ConclusionThe main benefit of our proposal is that it is possible to model security requirements together with the conceptual model of the data warehouse during the early stages of a project, and automatically obtain the corresponding implementation for XML.

Methodological guidelines for reducing the complexity of data warehouse development for transactional blood bank systems

Over time, data warehouse (DW) systems have become more difficult to develop because of the growing heterogeneity of data sources. Despite advances in research and technology, DW projects are still too slow for pragmatic results to be generated. Here, we address the following question: how can the complexity of DW development for integration of heterogeneous transactional information systems be reduced? To answer this, we proposed methodological guidelines based on cycles of conceptual modeling and data analysis, to drive construction of a modular DW system. These guidelines were applied to the blood donation domain, successfully reducing the complexity of DW development.

Perancangan Data Warehouse Untuk Penerimaan Mahasiswa Baru

Data warehouse (DW) integrates information from numerous data sources not only for storage purposes but also for business analysis as well. As this information is to be used for business intelligent, it makes the design of DW difficult task. DW quality depends on the quality of all requirements, conceptual, logical and physical models used for DW design. Therefore , data quality has to be incorporated in the whole phase of DW development. This paper demonstrates the application of IRADAH (Integrated Requirements Analysis for Data Warehouse) method in implementing data quality into the whole DW development for stude n ts admission. Keywords: Data Warehouse, IRADAH, students admission

Core Methodologies in Data Warehouse Design and Development

Data warehouse is a system which can integrate heterogeneous data sources to support the decision making process. Data warehouse design is a lengthy, time-consuming, and costly process. There has been a high failure in data warehouse development projects. Thus how to design and develop a data warehouse have become important issues for information systems designers and developers. This paper reviews and discusses some of the core data warehouse design and development methodologies in information system development. The paper presents in particular the most recent and much heated hybrid approach which is a combination of data-driven and requirement-driven approaches.

Effective data warehouse for information delivery: a literature survey and classification

Data warehouse is playing an important role in strategic decision making process for complex business solutions. To gain competitive advantage, business executives are increasingly making use of data warehouse concepts as it plays a vital role in analysing, predicting future trends based on past and current scenarios. We as authors have surveyed the various techniques used in building of data warehouse and the methods used for the implementation of techniques. We have conducted an in-depth survey of existing literature from various known international journal papers to come up with a framework which will help the researchers to focus on specific and emerging areas in the field of data warehouse development as well as application of data warehouse in various business domains.

A business-oriented approach to data warehouse development

Several surveys have indicated that many data warehouses fail to meet business objectives or are outright failures. One reason for this is that requirement engineering is typically overlooked in real projects. This paper addresses data warehouse design from a business perspective by highlighting business strategy analysis, alignment between data warehouse objectives and a firm's strategy, goal-oriented information requirements' modelling and how an underlying multidimensional data warehouse model may be derived. A set of guidelines is provided allowing developers to design a data warehouse aligned with a prevailing business strategy. A classic case study is presented.

PS2-59: It Takes a Village to Raise a VDW Data Set: Creating Partnerships to Improve Research Data

Background Healthcare data is highly complex, and considerable effort is required to create rich data resources that are reliable, user-friendly and represent valid utilization. Challenges include identifying appropriate sources, interpreting the data in a given source, matching data between sources, and transforming source data to meet desired specifications. These challenges are shared by all healthcare analytic groups, including research, finance, HEDIS reporting, care delivery and membership management. Creating partnerships for data development both within and across institutions could make the process more efficient by pooling specialized knowledge and providing opportunities to share both development strategies and data products. Objective: Within KPMAS, our goal was to facilitate development of the Virtual Data Warehouse (VDW) by tapping into existing knowledge about source data and strategies for validating and refining data. Across KP sites, our goal was to reduce the effort required to build and use the VDW by sharing code and analytic infrastructure among sites using highly similar data sources.

Modeling Extraction Transformation Load Embedding Privacy Preservation using UML

Extraction Transformation Load plays an important phase in development of data warehouse due its complexity of selecting data from different location and having different structures. The recent industry of data warehouse is driven by Privacy Preserving Data Mining which ensures privacy of sensitive information during Mining and is a requirement of most Data Bases. Current approaches to modelling extraction transformation load do not include privacy representation in Conceptual Modelling. This paper proposes object-oriented approach to model Extraction Transformation Load embedding privacy preservation. The major components of extraction include Data Source, Source Identifier, Retrieval, Join, Privacy Preserving Area and Data Staging Area. All the above mentioned components have been modelled using Unified Modelling Language.

L5.3 Building an Information Platform for New Drug Discovery

Acceleration in the continuum of discovery, translation and delivery of personalized cancer care requires an integrated network information system to utilize the overwhelming amount of information being generated in basic, translational and clinical research. The stakeholders themselves: researchers, clinicians, administrators and patients must be part of developing and delivering a rapid learning, knowledge-based health system that creates value for the stakeholder, and provides evidence-based alternatives. For example, by having access to a large prospective database of patients, clinicians would be able to find patients most suitable for clinical trials based on their clinical and molecular characteristics which would result in shortening the time to reach adequate patient accrual, and patients would be most suitably matched to the “right” trial. In 2003, the Moffitt Cancer Center in Tampa, Florida launched an initiative in personalized cancer care that has now expanded to 18 health institutions in 10 states, called Total Cancer CareTM (TCC). Critical to the pursuit of personalized medicine is the development of a large regional cancer biorepository in parallel with the development of a relational data warehouse and an information system containing patient's clinical and molecular data. The Total Cancer CareTM Protocol, now with >89K consented patients, >32K tumors and >16K genetic profiles, provides the mechanism to accomplish these goals. A Health and Research Informatics platform facilitates the rapid identification of patient subjects eligible for specific trials. Several pre-clinical projects to define biomarker prevalence, concordance between gene expression and protein expression, and to validate biomarkers, are underway using the TCC database. Specific cohorts of subjects with various cancers identified at Moffitt and within the TCC consortium have been found to be clinically and molecularly eligible for trials involving a variety of biomarkers and targeted agents. Enrollment on these early phase studies that use CLIA defined biomarkers is well underway. This approach has the potential to dramatically decrease trial accrual times, require fewer subjects per trial, and result in increased response rates by specifically targeting molecular alterations in patient tumors.

A trace metamodel proposal based on the model driven architecture framework for the traceability of user requirements in data warehouses

The complexity of the data warehouse (DW) development process requires to follow a methodological approach in order to be successful. A widely accepted approach for this development is the hybrid one, in which requirements and data sources must be accommodated to a new DW model. The main problem is that we lose the relationships between requirements, elements in the multidimensional (MD) conceptual models and data sources in the process, since no traceability is explicitly specified. Therefore, this hurts requirements validation capability and increases the complexity of Extraction, Transformation and Loading processes. In this paper, we propose a novel trace metamodel for DWs and focus on the relationships between requirements and MD conceptual models. We propose a set of Query/View/Transformation rules to include traceability in DWs in an automatic way, allowing us to obtain a MD conceptual model of the DW, as well as a trace model. Therefore, we are able to trace every requirement to the MD elements, further increasing user satisfaction. Finally, we show the implementation in our Lucentia BI tool.

Enfoque MDA para el diseño de un data warehouse difuso

A Data Warehouse (DW) is a repository of data from different sources. Is a system for data analysis used to support decision making. In the area of data analysis is common to find relationships between data from fuzzy nature. In this context, the questions give me the marks quantity of good, average and bad cannot be obtained from the traditional DW because do not handle as qualitative information. Due to the limitations of traditional DW is that it can be extended to Fuzzy DW (DWD) using fuzzy logic. On the other hand in the DW development life cycle are considered different methodologies and models that have the disadvantage that do not integrate the different levels of abstraction or the DW design process. However, MDA is an alternative approach to integration for the development of DW as it gives a comprehensive development framework based on the architecture of models and transformations between different levels of abstraction. This article presents a Fuzzy MDA approach to design fuzzy measures, fuzzy relations and fuzzy levels. For which we propose a Fuzzy OLAP CWM metamodel, QVT rules and an example of rule. Is the sequence methodology for applying the rules QVT. Finally, it provides a case study analysis in the field of education.

A Model of Data Warehousing Process Maturity

Even though data warehousing (DW) requires huge investments, the data warehouse market is experiencing incredible growth. However, a large number of DW initiatives end up as failures. In this paper, we argue that the maturity of a data warehousing process (DWP) could significantly mitigate such large-scale failures and ensure the delivery of consistent, high quality, “single-version of truth” data in a timely manner. However, unlike software development, the assessment of DWP maturity has not yet been tackled in a systematic way. In light of the critical importance of data as a corporate resource, we believe that the need for a maturity model for DWP could not be greater. In this paper, we describe the design and development of a five-level DWP maturity model (DWP-M) over a period of three years. A unique aspect of this model is that it covers processes in both data warehouse development and operations. Over 20 key DW executives from 13 different corporations were involved in the model development process. The final model was evaluated by a panel of experts; the results strongly validate the functionality, productivity, and usability of the model. We present the initial and final DWP-M model versions, along with illustrations of several key process areas at different levels of maturity.

A Schema Selection Framework for Data Warehouse Design

Data schema represents the arrangement of fact table and dimension tables and the relations between them. In data warehouse development, selecting a right and appropriate data schema (Snowflake, Star, Star Cluster …) has an important Impact on performance and usability of the designed data warehouse. One of the problems that exists in data warehouse development is lack of a comprehensive and sound selection framework to choose an appropriate schema for the data warehouse at hand by considering application domain-specific conditions. In this paper, we present a schema selection framework that is based on a decision tree for solving the problem of choosing right schema for a data warehouse. The main selection criteria that are used in the presented decision tree are query type, attribute type, dimension table type and existence of index. To evaluate correctness and soundness of this framework, we have designed a test bed that includes multiple data warehouses and we have created all the possible states in decision tree of schema selection framework. Then we designed all types of queries and performed the designed queries on these data warehouses. The results confirm the correct functionality of the schema selection framework.

Transforming Conceptual Model into Logical Model for Temporal Data Warehouse Security: A Case Study

Extraction–transformation–loading (ETL) processes are responsible for the extraction of data from several sources, their cleansing, customization and insertion into a data warehouse. Data warehouse often store historical information which is extracted from multiple, heterogeneous, autonomous and distributed data sources, thereby, the survival of the organizations depends on the correct management, security and confidentiality of the information. In this paper, we are using the Model Driven Architecture (MDA) approach to represent logical model requirements for secure Temporal Data Warehouses (TDW). We use the Platform-Independent Model (PIM) which does not include information about specific platforms and technologies. Nowadays, the most crucial issue in MDA is the transformation between a PIM and Platform Specific Models PSM. Thus, OMG defines use the Query/View/Transformation (QVT) language, an approach for expressing these MDA transformations. This paper proposes a set of rules to transform PIM model for secure temporal data warehouse (TDW) to PSM model, we apply the QVT language to the development of a secure data warehouse by means of a case study.

Design and Implementation of a Data Warehouse for Benchmarking in Clinical Rehabilitation

Clinical Benchmarking provides comparative analysis among healthcare institutions in order to improve their quality and efficiency. This paper describes the development and implementation of a Data Warehouse, following Kimball's Business Dimensional Lifecycle framework, to support a dynamic Decision Support System for Benchmarking in Clinical Rehabilitation. For the Data Warehouse development, the identification of the most important concepts as well as the data modeling process are the most challenging and important tasks, requiring deep knowledge on the subjects in matter. Consequently, the developed Data Warehouse respects the specific characteristics related with Clinical Rehabilitation, addressed through a data modeling process using different Data Marts for each of the major business requirements. The evaluation of the solution, using real clinical records, demonstrated a clear orientation and suitability to the subject in matter as well as the capability to provide accurate current and historical analysis.