Data Management Architecture Research Articles

클라우드 서비스를 위한 고가용성 대용량 데이터 처리 아키텍쳐

These days, there are more and more IT research institutions which foresee cloud services as the predominant IT service in the near future and there, in fact, are actual cloud services provided by some IT leading vendors. Regardless of physical location of the service and environment of the system, cloud service can provide users with storage services, usage of data and software. On the other hand, cloud service has challenges as well. Even though cloud service has its edge in terms of the extent to which the IT resource can be freely utilized regardless of the confinement of hardware, the availability is another problem to be solved. Hence, this paper is dedicated to tackle the aforementioned issues; prerequisites of cloud computing for distributed file system, open source based Hadoop distributed file system, in-memory database technology and high availability database system. Also the author tries to body out the high availability mass distributed data management architecture in cloud service`s perspective using currently used distributed file system in cloud computing market.

A Study on the Organizational Architecture and Standard System of the Data Sharing Network of Earth System Science in China

The aim of this paper is to discuss the organizational architecture and standard system for sharing research data at the national level. The Data Sharing Network of Earth System Science (DSNESS) is one of the nine pilot projects of the Scientific Data Sharing Project in China that has become a long-term operational research data-sharing platform in the National Science and Technology Infrastructure (NSTI) of China. First, a data sharing union mechanism was designed with the core principle being, “data come from research and will be reused in research”. Second, a data sharing organizational architecture was constructed that consists of three sections: data resource architecture, data management architecture, and data services architecture. A physical data sharing network was constructed that includes one general center and 15 distributed sub-centers based on the architecture. Third, a series of data sharing standards and specifications were designed and implemented in the DSNESS. The reference model of the DSNESS standard system includes three levels of standards: directive standards, general standards, and application standards. In total, 21 high level standards and specifications were developed and implemented in the DSNESS. Several core standards and specifications, such as the extensible metadata standard, data quality control specifications, and so on, were analyzed in detail. Finally, the data service effect was summarized in three aspects: dataset services, standard and specification services, and international cooperation services. This research shows that the organizational architecture and standard system is a very important soft environment for research data sharing. The practices of DSNESS will provide useful experiences for multi-disciplinary data sharing in Earth science and will help to eliminate the data gap between the rich and poor at the national level.

Research on Simulation Data Management of Complicated Mechanical Product

Presently, simulation data management of complex mechanical products is a key issue on the aspect of CAE technical application. Based on the functional analysis of simulation data management platform, this research proposes a B/S-based architecture for data management, and then analyzes several key techniques of simulation data management. Finally, as a preliminary exploration, performs study on novel combinational solution, which is a simulation data management platform integrated with SimManager and Teamcenter 9.

PerLa: A Language and Middleware Architecture for Data Management and Integration in Pervasive Information Systems

A declarative SQL-like language and a middleware infrastructure are presented for collecting data from different nodes of a pervasive system. Data management is performed by hiding the complexity due to the large underlying heterogeneity of devices, which can span from passive RFID(s) to ad hoc sensor boards to portable computers. An important feature of the presented middleware is to make the integration of new device types in the system easy through the use of device self-description. Two case studies are described for PerLa usage, and a survey is made for comparing our approach with other projects in the area.

Adaptive data management in the ARC Grid middleware

The Advanced Resource Connector (ARC) Grid middleware was designed almost 10 years ago, and has proven to be an attractive distributed computing solution and successful in adapting to new data management and storage technologies. However, with an ever-increasing user base and scale of resources to manage, along with the introduction of more advanced data transfer protocols, some limitations in the current architecture have become apparent. The simple first-in first-out approach to data transfer leads to bottlenecks in the system, as does the built-in assumption that all data is immediately available from remote data storage. We present an entirely new data management architecture for ARC which aims to alleviate these problems, by introducing a three-layer structure. The top layer accepts incoming requests for data transfer and directs them to the middle layer, which schedules individual transfers and negotiates with various intermediate catalog and storage systems until the physical file is ready to be transferred. The lower layer performs all operations which use large amounts of bandwidth, i.e. the physical data transfer. Using such a layered structure allows more efficient use of the available bandwidth as well as enabling late-binding of jobs to data transfer slots based on a priority system. Here we describe in full detail the design and implementation of the new system.

A Product Data Management architecture for integrating hardware and software development

This paper proposes an architecture of Product Data Management (PDM) for integrating Software Configuration Management (SCM). The proposed architecture extends its data model to support version items for SCM. Its applications support both PDM and SCM functionalities, as well as integrate hardware parts and software items in product configurations and engineering change management. The architecture is implemented by using a commercial Product Lifecycle Management (PLM) system. The implementation enables hardware engineers and software programmers to share the same user environment, on a consistent database during the collaborative product development.

Architecture of RFID Spatio-Temporal Data Management

With the development of RFID technology, we can identify, locate, track and monitor items in supply chain, retail store, and asset management applications. RFID has become a key technology in the Internet of Things. But RFID data can’t be effectively managed by only using traditional data model because they have their own unique characteristics, such as aggregation, location, temporal and history-oriented, which have to be fully considered and integrated into the data model. Therefore, the architecture of RFID spatio-temporal data management is proposed in this paper. We provide a brief overview of RFID technology and highlight a few of the spatio-temporal data management challenges, such as RFID middleware, RFID event processing, In-memory cache.

A MultiCommodity Grid Logistics Model for Data Transportation in Federated Datagrid

Data Sharing is the process of making data resources available across geographically distributed institutions working on a common purpose. When there is need or request, these data resources are transported so as to co-allocate them at desired destinations. Distributed computing, particularly Grid computing environment involves transporting huge volume of data across geographically spread sources and destinations managed by an efficient data management architecture called the Datagrid. This affinity towards movement of data is to have the data and task collocated to eliminate network traffic and to improve performance by reducing latency. Participants of the datagrid are autonomous organizations, organized in a federated fashion that can request for resources from the federation as long as they are authentic and authorized and can fall into or fall out of the grid at their own will. Data Transportation mechanism in grid can be compared to the Multi-index transportation problem i.e. an n-dimensional problem (here n = 3) and usually referred to as multi-commodity transportation problem with an additional index for handling multiple resource types. This work proposes a novel method to transport resources taking various challenges posed by the dynamic grid environment into account.

A Sensor Data Management Scheme for Wireless Sensor Networks

One of the major difficulties of wireless sensor network (WSN) applications is how to efficiently manage the large amount of data produced by sensors. The differences from standard database source pose challenges of sensor data management. In this paper, we propose a novel sensor data management architecture based on our extensive discussion on existing works. The hierarchical system model consists of sensor network layer and proxy network layer. Sensor network layer performs limited computation and communication while being managed intelligently by the proxy network. The proxy network receives sensor data, manages sensor data and processes queries in a distributed manner. We also provide insight into possible research directions in this area.

Advertisement Data Management and Application Design in WBCs

This paper describes the advertisement data management and the corresponding intelligent application design implementation for wireless billboard channel (WBC) in the emerging ubiquitous consumer wireless world (UCWW). The algorithms and application runs at the application enabler sub-layer of a WBC service provider (WBC-SP)'s node for broadcasting wireless services to mobile terminals (MTs). The advertisement data is formatted by abstract syntax notation one (ASN.l) and organized by segments with WBC algorithms to reduce the mobile user (MU)'s access time. The intelligent application was used to run the data management algorithms, which is implemented with three tiers: a service discovery and maintenance tier acting as a client-server distributed system for data collection and organization; an intelligent application tier holding all business logic and common application programming interfaces (APIs); and a multi-agent systems (MAS) tier maintaining the advertisement, discovery and association (ADA) agents' lifecycle, and supplying directory facilitator services and message transport services. The performance evaluation of the proposed data management and application architecture details are provided.

A Proxy based Framework for Efficient Range Query Processing in a Cellular Network

Within the last decade, the continuous development of mobile technology has motivated an intense research in mobile data services. Among these services, location-based services and location-dependant queries have attracted a lot of attention from the software industry as well as from researchers. Although a number of locations based service methods have been proposed, the performance is not satisfactory in the case of dynamic data management problems that introduce high overhead in update and query operations. In this paper, we introduce a proxy based distributed dynamic data management architecture for location based services that reduces network cost with required quality of service (QoS). The system proposes a distributed hierarchical database structure with collocated proxy for efficient service delivery to location-dependant queries especially Range Queries (RQ). The system that we propose have the following advantages. i) It performs efficient processing of location dependant queries, especially Range Queries spanning over multiple network cells ii) It reduces wireless communication between mobile devices and servers. iii) The system is scalable where location data is distributed and arranged hierarchically. We use proxy objects to carry out processing task for each mobile user. Thus proxies play an important role and in charge of routing the queries to different (distributed) databases and present the results to queries efficiently. Simulation study shows the effectiveness of our proposed architecture.

Personalized Data Set For Analysis

Data Management portfolio within an organization has seen an upsurge in initiatives for compliance, security, repurposing and storage within and outside the organization. When such initiatives are being put to practice care must be taken while granting access to data repositories for analysis and mining activities. Also, initiatives such as Master Data Management, cloud computing and self service business intelligence have raised concerns in the arena of regulatory compliance and data privacy, especially when a large data set of an organization are being outsourced for testing, consolidation and data management. Here, an approach is presented where a new service layer is introduced, by data governance group, in the architecture for data management and can be used for preserving privacy of sensitive information.

Institutional Profile: The Emory Biomarker Service Center

The Emory Biomarker Service Center (EBSC) provides state-of-the-art analytical technologies to impact biomarker discovery, development and clinical translation of biomarker research in a spectrum of disease and therapeutic areas. EBSC technologies are primarily concentrated in medical genomics, microarray expression profiling and data analysis. The EBSC provides state-of-the-art services in nucleic acid extraction, genomic sequencing, custom and whole-genome genotyping, cytosine–phosphate–guanine dinucleotide (CpG) methylation, mRNA and miRNA expression analyses. The EBSC typically processes tens of thousands of samples per year, utilizing instrumentation and automation from Tecan, Qiagen, Applied Biosystems, Eppendorf and Beckman Coulter, which are supported by the Thermofisher Nautilus Laboratory Information Management System data-management architecture. The EBSC provides services and platforms for biomarker profiling of fresh, frozen or formalin-fixed paraffin-embedded tissues using both Affymetrix, Inc. and Illumina, Inc. platforms. It provides quality-control and data analysis in a high-throughput manner in a Clinical Laboratory Improvement Amendments certified laboratory. Bioinformatics support is provided to users by a team of analysts using a variety of standard and custom software tools.

PRESTO: Feedback-Driven Data Management in Sensor Networks

<para xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> This paper presents PRESTO, a novel two-tier sensor data management architecture comprising proxies and sensors that cooperate with one another for acquiring data and processing queries. PRESTO proxies construct time-series models of observed trends in the sensor data and transmit the parameters of the model to sensors. Sensors check sensed data with model-predicted values and transmit only deviations from the predictions back to the proxy. Such a model-driven push approach is energy-efficient, while ensuring that anomalous data trends are never missed. In addition to supporting queries on current data, PRESTO also supports queries on historical data using interpolation and local archival at sensors. PRESTO can adapt model and system parameters to data and query dynamics to further extract energy savings. We have implemented PRESTO on a sensor testbed comprising Intel Stargates and Telos Motes. Our experiments show that in a temperature monitoring application, PRESTO yields one to two orders of magnitude reduction in energy requirements over on-demand, proactive or model-driven pull approaches. PRESTO also results in an order of magnitude reduction in query latency in a 1% duty-cycled five hop sensor network over a system that forwards all queries to remote sensor nodes. </para>

Adaptable data management for systems biology investigations

BackgroundWithin research each experiment is different, the focus changes and the data is generated from a continually evolving barrage of technologies. There is a continual introduction of new techniques whose usage ranges from in-house protocols through to high-throughput instrumentation. To support these requirements data management systems are needed that can be rapidly built and readily adapted for new usage.ResultsThe adaptable data management system discussed is designed to support the seamless mining and analysis of biological experiment data that is commonly used in systems biology (e.g. ChIP-chip, gene expression, proteomics, imaging, flow cytometry). We use different content graphs to represent different views upon the data. These views are designed for different roles: equipment specific views are used to gather instrumentation information; data processing oriented views are provided to enable the rapid development of analysis applications; and research project specific views are used to organize information for individual research experiments. This management system allows for both the rapid introduction of new types of information and the evolution of the knowledge it represents.ConclusionData management is an important aspect of any research enterprise. It is the foundation on which most applications are built, and must be easily extended to serve new functionality for new scientific areas. We have found that adopting a three-tier architecture for data management, built around distributed standardized content repositories, allows us to rapidly develop new applications to support a diverse user community.

Advanced Digital Image Preservation Data Management Architecture

Digital data preservation is a difficult problem due to the high cost associated with the required technological development, rapid change and continuous anticipation of societal technology adaptation, and finally, the lack of interoperable standards for access and integration of diverse media content, software applications, and operating hardware systems over a decade-long timeline. While there is no single strategy to encompass all problems, the goal is to provide open preservation strategies between technologies. This paper presents a newly developed international standard technology called Multimedia Application Formats (MAF) from the ISO/IEC SC 29 WG 11 (MPEG) as the recommended digital image preservation data management architecture to migrate and preserve digital image content. This paper presents the implementation of such preservation strategy.

BayesStore

Several real-world applications need to effectively manage and reason about large amounts of data that are inherently uncertain. For instance, pervasive computing applications must constantly reason about volumes of noisy sensory readings for a variety of reasons, including motion prediction and human behavior modeling. Such probabilistic data analyses require sophisticated machine-learning tools that can effectively model the complex spatio/temporal correlation patterns present in uncertain sensory data. Unfortunately, to date, most existing approaches to probabilistic database systems have relied on somewhat simplistic models of uncertainty that can be easily mapped onto existing relational architectures: Probabilistic information is typically associated with individual data tuples, with only limited or no support for effectively capturing and reasoning about complex data correlations. In this paper, we introduce BayesStore, a novel probabilistic data management architecture built on the principle of handling statistical models and probabilistic inference tools as first-class citizens of the database system. Adopting a machine-learning view, BAYESSTORE employs concise statistical relational models to effectively encode the correlation patterns between uncertain data, and promotes probabilistic inference and statistical model manipulation as part of the standard DBMS operator repertoire to support efficient and sound query processing. We present BAYESSTORE's uncertainty model based on a novel, first-order statistical model , and we redefine traditional query processing operators, to manipulate the data and the probabilistic models of the database in an efficient manner. Finally, we validate our approach, by demonstrating the value of exploiting data correlations during query processing, and by evaluating a number of optimizations which significantly accelerate query processing.

Distributed musical performances

An increasing number of novel applications produce a rich set of different data types that need to be managed efficiently and coherently. In this article we present our experience with designing and implementing a data management infrastructure for a distributed immersive performance (DIP) application. The DIP project investigates a versatile framework for the capture, recording, and replay of video, audio, and MIDI (Musical Instrument Digital Interface) streams in an interactive environment for collaborative music performance. We are focusing on two classes of data streams that are generated within this environment. The first category consists of high-resolution isochronous media streams, namely audio and video. The second class comprises MIDI data produced by electronic instruments. MIDI event sequences are alphanumeric in nature and fall into the category of the data streams that have been of interest to data management researchers in recent years. We present our data management architecture, which provides a repository for all DIP data. Streams of both categories need to be acquired, transmitted, stored, and replayed in real time. Data items are correlated across different streams with temporal indices. The audio and video streams are managed in our own High-performance Data Recording Architecture (HYDRA), which integrates multistream recording and retrieval in a consistent manner. This paper reports on the practical issues and challenges that we encountered during the design, implementation and experimental phases of our prototype. We also present some analysis results and discuss future extensions for the architecture.

Real‐time acoustic monitoring of the deep‐ocean environment

ESONET is a European Network of Excellence (NoE) associating 50 partners (research centres, universities, industrials and SMEs) from 14 countries: France, Germany, Italy, UK, Spain, Portugal, Greece, Belgium, Ireland, the Netherlands, Norway, Sweden, Bulgaria, and Turkey. More than 300 scientists and engineers will participate to its activities. The aim of the ESONET NoE is the lasting integration of European research on deep‐sea multidisciplinary observatories. ESONET is particularly sensitive on the effects of noise on marine organisms. Because our knowledge is still quite limited, ESONET is developing a Demonstration Mission, called LIDO, Listening to the Deep‐Ocean Environment, a research program that will help establishing a scientific base to allow (1) the real‐time automatic identification and classification of nonbiological and biological sounds, (2) the monitoring of marine organisms and population dynamics, (3) the assessment and control of the long term effects of anthropogenic sources on marine organisms, (4) the education of the public, end‐users and the administration, and (5) the approval of ethical guidelines and procedures. This latter point constitutes a major criterion to award the ESONET LABEL to the observatories. This paper presents the data management architecture and the RT analysis processes that will be carried out at the observatories.

Seismological Data Warehousing and Mining

Earthquake data composes an ever increasing collection of earth science information for post-processing analysis. Earth scientists, local or national administration officers and so forth, are working with these data collections for scientific or planning purposes. In this article, we discuss the architecture of a so-called seismic data management and mining system (SDMMS) for quick and easy data collection, processing, and visualization. The SDMMS architecture includes, among others, a seismological database for efficient and effective querying and a seismological data warehouse for OLAP analysis and data mining. We provide template schemes for these two components as well as examples of their functionality towards the support of decision making. We also provide a comparative survey of existing operational or prototype SDMMS.