Building Data Warehouse Research Articles

Integration Methods and Advantages of Machine Learning with Cloud Data Warehouses

A data warehouse is a subject-oriented, integrated, relatively stable collection of data that reflects historical changes and is used to support management decisions. Common tools for building a data warehouse are IBM Cognos and SAP BO. However, both of the above use centralized single-node mode to build data warehouses. This type of data warehouse has poor scalability, and due to the rapid increase in the scale of the Internet, traditional data warehouses can no longer meet the actual needs of use. This paper mainly introduces the integration of cloud data warehouse and machine learning as well as the importance and application of parallel integration methods. First, the paper describes how the combination of cloud data warehousing and machine learning can promote business innovation and increase output. It then discusses the challenges of managing machine learning models in production environments, and introduces the role of cloud data warehouses in addressing these challenges. Subsequently, the cloud computing integration of Snowflake, as well as the implementation steps and processes of the parallel integration approach are also introduced in detail. Finally, the results of parallel integration method are analyzed, and it is considered that it has a good application prospect and development potential in cloud data warehouse.

ONTOLOGY-BASED DATA WAREHOUSE VISUALIZATION USING PROTÉGÉ FOR CORONA VIRUS SPREAD INCIDENCES

Corona Virus (Covid-19) is an infectious disease caused by a newly discovered coronavirus. At this time, there are no specific vaccines or treatments for COVID-19. death rates due Covid-19 in Indonesia reach 8,9%. The presence of differences between data released by the Covid-19 complaint website of West Java Province and data released by Subang Government indicates that’s still difficult to obtain accurate information. We need to build data warehouse and create ontology of Covid-19 incidences data to figure out the important information in Covid-19. we are using Protégé to build and visualize ontology of Covid-19 data.
 
 Keywords : ontology, Protégé, Covid-19

Building Data Warehouse and Dashboard of Church Congregation Data

A data warehouse is essential for an organization to process and analyze data coming from the organization. Hence, a data warehouse together with a dashboard to visualize the processed data are built to accommodate the need of the church administrator to analyze a large set of church congregation data. The data warehouse is built using the Kimball principle. This Kimball principle emphasizes the implementation of a dimensional model in the data warehouse, not a relational model used in a regular transactional database. An ETL process that contains extract, transform and load processes is used to retrieve all data from the regular transactional database and transform the data so the data can be loaded into the data warehouse. A dashboard is then built to visualize the data from the data warehouse so the users can view the processed data easily. Users can also export the processed data into an excel file that can be downloaded from the dashboard. A web service is built to get data from the data warehouse and return it to the dashboard.

Soil Monitoring and Recommendation System

Plant fertility is very dependent on the levels of nutrients present in the soil. Measurement of soil nutrients can be done through laboratory tests and can also be done using IoT technology. IoT equipment is supported by sensors that can be used to measure soil nutrients and the location of the soil is measured. Data from land measurements made through IoT technology can be stored and displayed in real-time through the cloud. This paper designs data sources derived from IoT-based nutrient measurement data to build data warehouse locations and soil nutrients. A data warehouse that is formed later can be done for analysis of the types of plants and fertilizers needed at a soil location. The agricultural yield primarily depends on soil fertility, the moisture level of soil and use of appropriate fertilizers. In the current scenario, the manual method of measuring the soil nutrients is less accurate because of the time difference of soil sample collected at the field and when it is measured in a laboratory. It becomes necessary to create a smarter agriculture practice through Internet of Things (IoT) to address this challenge. Soil nutrient analysis using sensor enables various application like remote monitoring of soil fertility, analysis, provide a selection of crop and build irrigation decision support systems. In the proposed system, the wireless sensors measure the macro nutrient of soil and transmit the data to the cloud. The user can view the soil fertility at the convenience on their mobile application. It is possible that the farmer may wish to grow a specific crop based on the economic interest. The software system has the intelligence to recommend the fertilizer that is required to be used to suit the needs of the desired crop, thus improving the quality of the soil and in turn, increase the yield. Overall, the proposed system helps farmers to gather real-time information about various soils, their fertility level, suggest crops and fertilizers at the convenience of the mobile app. Finally, this project effort will help farmers to make the right decision, gain better yield and economic.

Data warehousing for Open Data sharing and decision support in agriculture: a case study of the VDSA Knowledge Bank and its development process

Acceptance of Open Data by the governments, foundations, science councils and other sponsors of research across the world have prompted the trend towards Open Data. Many researchers and organizations have embraced the Open Data policy. Most often the datasets are released through Dataverse. However, just making datasets available is not enough. We need inbuilt user-friendly data extraction and transformation techniques, and analytical features that can facilitate analyses of research data and generate new knowledge and insights from such data. The Village Dynamics in South Asia (VDSA) Knowledge Bank, ( http://vdsakb.icrisat.ac.in/ ), is a unique data warehouse with user-friendly data retrieval and online analytical processing features for household level panel data on India and Bangladesh. It is the first of its kind in the world for managing household survey data. It exemplifies how digital innovation tools (Microsoft SQL and Business Intelligence tools) can be used in processing diverse, complex long-term databases and promoting the cause of Open Data and facilitates decision support needs. This paper has documented the development process, technical features and impacts of the VDSA Knowledge Bank. It provides an in-depth understanding about how to plan, partner with information technology firms, design and implement a data warehouse project. Thus, the main contribution of this paper is featuring how developments in information technology can be innovatively used to promote Open Data and enhance impacts. Hopefully, this study will stimulate interests among researchers and donors in building data warehouse for management and dissemination of Open Data.

Building Data Warehouse for Diseases Registry: First Step for Clinical Data Warehouse

Almost all the Decision Makers in medical path need a good platform which providing analytical results to use themin order to support their decisions.Due to vast amount of patient's records in Electronic Health Record(EHR) rather than Electronic Medical Records(EMR),Medical data can be used to get valuable information to support decisions of specialists in clinical path.Clinicaldata warehouse can combine different sources of clinicaldata into single repository and use it to produce analytical information to support decisions. Since Online Analytical Processing (OLAP) can be used in analyzing data and find the relationship between many factors and provide good view of data from many points of view so using data warehousing and OLAP techniques is the good choice for helping professionals and analysts in finding their goals.We will use diseases registry data (Part from EMR) to design and implement data warehouse as the first step that could be used later by OLAP or Data Mining to support Clinical Decisions. In this work we combined different sources of diseases registry dataintosingle repository.

A Model Driven Architecture Approach to Generate Multidimensional Schemas of Data Warehouses

Over the past decade, the concept of data warehousing has been widely accepted. The main reason for building data warehouses is to improve the quality of information in order to achieve specific business objectives such as competitive advantage or improved decision-making. However, there is no formal method for deriving a multidimensional schema from heterogeneous databases that is recognized as a standard by the OMG and the professionals of the field. Which is why, in this paper, we present a model-driven approach (MDA) for the design of data warehouses. To apply the MDA approach to the Data warehouse construction process, we describe a multidimensional meta-model and specify a set of transformations from a UML meta-model which is mapped to a multidimensional meta-model. The transformation rules are programmed by the Query View Transformation (QVT) language. A case study illustrates our approach. It demonstrates how it reinforces the components traceability and reusability and how it globally improves the modeler’s efficiency. Furthermore, the use of the UML, as a technique to build data warehouses, is an important facilitator which prepares our further work to automate this approach.

RESEARCH OF FUZHOU HISTORICAL BUILDING MANAGEMENT PLATFORM BASED ON DATA WAREHOUSE TECHNOLOGY

Abstract. In order to solve the problem of "data island" in the management of Fuzhou historical buildings. This paper analyzes historical building management needs and historical building management indicators. Then the data warehouse model of historic buildings is constructed by using dimension modeling method, and a set of data warehouse of Fuzhou historic buildings is designed and established. Finally, a historical building management platform was built on the basis of the historical building data warehouse, which realized the entry, standardization, extraction and cleaning of historical building management data. It provides data support for data mining of historic buildings in order to realize the protection, activation, utilization and standardized management of Fuzhou historic buildings.

Merge of X-ETL and XCube towards a Standard Hybrid Method for Designing Data Warehouses

There is no doubt that the hybrid approach is the best paradigm for designing effective multidimensional schemas. Its strength lies in its ability to combine the top-down and bottom-up approaches, thus exploiting the advantages of both approaches. In this paper, the authors try to identify and analyze the different hybrid methods developed for building data warehouses. The analysis revealed that the existing methods are too complicated and time consuming in the deployment phase. In order to solve this problem, the authors introduced a new hybrid method that is easy to use and saves a huge amount of deployment time. This new method consists of two main steps: the first data driven step allows an analysis of the source models by using the X-ETL method and gives rise to star models. The second requirements driven step performs a semantic analysis of the needs expressed in natural language by using the XCube Assist method. This analysis allows to improve the quality of star models generated by the X-ETL method without the intervention of a designer.

A Model Driven Architecture Approach to Generate Multidimensional Schemas of Data Warehouses

Over the past decade, the concept of data warehousing has been widely accepted. The main reason for building data warehouses is to improve the quality of information in order to achieve specific business objectives such as competitive advantage or improved decision-making. However, there is no formal method for deriving a multidimensional schema from heterogeneous databases that is recognized as a standard by the OMG and the professionals of the field. Which is why, in this paper, we present a model-driven approach (MDA) for the design of data warehouses. To apply the MDA approach to the Data warehouse construction process, we describe a multidimensional meta-model and specify a set of transformations from a UML meta-model which is mapped to a multidimensional meta-model. The execution of the transformation, programmed by the Query View Transformation (QVT) language, takes as input an instance of the UML Meta-Model to generate an instance of the Dimensional Meta-Model as output.

Configurable Web Warehouses construction through BPM Systems

The process of building Data Warehouses (DW) is well known with well defined stages but at the same time, mostly carried out manually by IT people in conjunction with business people. Web Warehouses (WW) are DW whose data sources are taken from the web. We define a flexible WW, which can be configured accordingly to different domains, through the selection of the web sources and the definition of data processing characteristics. A Business Process Management (BPM) System allows modeling and executing Business Processes (BPs) providing support for the automation of processes. To support the process of building flexible WW we propose a two BPs level: a configuration process to support the selection of web sources and the definition of schemas and mappings, and a feeding process which takes the defined configuration and loads the data into the WW. In this paper we present a proof of concept of both processes, with focus on the configuration process and the defined data.

생체신호 습득과 건강 모니터링을 위한 스마트 헬스케어 의복 개발

Recently, the wearable computing technology with bio-sensors has been rapidly developed and utilized in various areas such as personal health, care-giving for senior citizens who live alone, and sports activities. In particular, the wearable computing equipment to measure vital signs by means of digital yarns and bio sensors is noticeable. The wearable computing devices help users monitor and manage their health in their daily lives through the customized healthcare service. In this paper, we suggest a system for monitoring and analyzing vital signs utilizing smart healthcare clothing with bio-sensors. Vital signs that can be continuously acquired from the clothing is well-known as unstructured data. The amount of data is huge, and they are perceived as the big data. Vital sings are stored by Hadoop Distributed File System(HDFS), and one can build data warehouse for analyzing them in HDFS. We provide health monitoring system based on vital sings that are acquired by biosensors in smart healthcare clothing. We implemented a big data platform which provides health monitoring service to visualize and monitor clinical information and physical activities performed by the users.

A New ETL Approach Based on Data Virtualization

ETL (Extract-Transform-Load) usually includes three phases: extraction, transformation, and loading. In building data warehouse, it plays the role of data injection and is the most time-consuming activity. Thus it is necessary to improve the performance of ETL. In this paper, a new ETL approach, TEL (Transform-Extract-Load) is proposed. The TEL approach applies virtual tables to realize the transformation stage before extraction stage and loading stage, without data staging area or staging database which stores raw data extracted from each of the disparate source data systems. The TEL approach reduces the data transmission load, and improves the performance of query from access layers. Experimental results based on our proposed benchmarks show that the TEL approach is feasible and practical.

Design of Books Analysis System in University Library Based on Data Warehouse

The design of books analysis system is beneficial to sufficiently using the accumulated data resources to guide work for librarians and help library leaders make decisions. The paper presents a system architecture of library books based on data warehouse. It focuses on several key technologies in building data warehouse, analyze and design the client application, It uses the accumulated data of college library as data resources , simulate simple data warehouse and use OLAP technology. It makes manager analysis data on different levels and aspects and show the result in the form of forms and charts.

Using OLAP with Diseases Registry Warehouse for Clinical Decision Support

Diseases registry databases in Iraqi hospitals hold huge information which can be used to support strategic clinical decisions. The clinicians and professionals often need tools to get the valuable information. This information cannot be got with normal functions used with operational databases. Using On-Line Analytical Processing (OLAP) gives insight and clear view on the information from different points. It makes it very easy and fast to view results as reports. The researchers used with clinical data warehouse to give decision makers on-time results to assist them in support their decisions. Two database sources used to construct and build data warehouse in order to use OLAP to view multidimensional data.

Building data warehouse in Metropolia University of Applied Sciences

Universities use many different data systems that store data in different databases. To get maximum benefit from this precious data it is necessary to build a data warehouse. In this building project data from different sources is integrated and processed to form that can be easily used in reporting and analyzing. There are two different widespread data warehouse architectures: Inmon’s and Kimball’s. Metropolia started first with Inmon’s architecture. There were many problems and after many years we could not get much useful data to reporting. Later we changed to Kimball’s architecture and got good results with it.

Research on the Supplementary Analysis System Based on CPS Standard

CPS test standard has put forward new demands to the management of power grids. This paper analyzes the main influence factors of CPS, and introduces the way of building data warehouse and constructing supplementary analysis system on this basis, which can effectively help power grid workers to research the pros and cons in dispatch and operation. The application of the supplementary analysis system is helpful for Shanghai power grid to enhance AGC control and frequency control ability under the control performance standard.

Realistic Analysis of Data Warehousing and Data mining Application in Education Domain

increase in the number of branches and intake. Now a day, any reputed Institute’s enrollments count in to thousands. In view of these factors the challenges for the management are meeting the diverse needs of students and facing increased complexity in academic processes. The complexity of these challenges requires continual improvements in operational strategies based on accurate, timely and consistent information. The cost of building a data warehouse is expensive for any educational institution as it requires data warehouse tools for building data warehouse and extracting data using data mining tools from data warehouse. The present study provides an option to build data warehouse and extract useful information using data warehousing and data mining open source tools. In this paper we have explored the need of data warehouse / business intelligence for an educational institute, the operational data of an educational institution has been used for experimentation. The study may help decision makers of educational institutes across the globe for better decisions..

DESAIN ETL DENGAN CONTOH KASUS PERGURUAN TINGGI

Data Warehouse for higher education as a paradigm for helping high management in order to make an effective and efficient strategic decisions based on reliable and trusted reports which is produced from Data Warehouse itself. Data Warehouse is not a software, hardware or tool but Data Warehouse is an environment where the transactional database is modelled in other view for decision making purposes. ETL (Extraction, Transformation and Loading) is a bridge to build Data Warehouse and transform data from transactional database. In every fact and dimension table will be inserted with fields which represent the construction merge loading as an ETL (Extraction, Transformation and Loading) extraction. ETL needs an ETL table and ETL process where ETL table as table connectivity between tables in OLTP database and tables in Data Warehouse and ETL process will transform data from table in OLTP database into Data Warehouse table based on ETL table. The extraction process will be run with a table database as differentiate ETL process and an ETL algorithm which will be run automatically in idle transactional process, along with daily transactional database backup when the information system are not used.

Reconstruction of biological networks based on life science data integration.

For the implementation of the virtual cell, the fundamental question is how to model and simulate complex biological networks. Therefore, based on relevant molecular database and information systems, biological data integration is an essential step in constructing biological networks. In this paper, we will motivate the applications BioDWH--an integration toolkit for building life science data warehouses, CardioVINEdb--a information system for biological data in cardiovascular-disease and VANESA--a network editor for modeling and simulation of biological networks. Based on this integration process, the system supports the generation of biological network models. A case study of a cardiovascular-disease related gene-regulated biological network is also presented.