Autonomous Data Management Research Articles

Data Management across Geographically-Distributed Autonomous Systems: Architecture, Implementation, and Performance Evaluation

Supporting efficient geographically-distributed autonomous data management is one of the critical obstacles for opening up a big data era. It creates the need for investigating such kind of systems in big data environments. In this paper, a distributed autonomous data management system is put forward, exhibiting the following features. (1) A distributed architecture designed to meet the requirements of autonomous data management by allowing interconnection, intercommunication, and interoperation of multiple sites over the Internet. (2) An autonomous, multi-level, unstructured data storage system to meet high-efficiency storage needs, with reference to the distributed heterogeneous data storage theories. (3) A distributed autonomous data indexing and retrieval system to support metadata search & full-text searching, fast loading, remote access, and unified view. Experimental and industrial application results demonstrate that the proposed system has high potential to reduce access time and improve storage efficiency, while maintaining satisfactory availability and scalability.

Enterprise Platform and Integration Concepts Research at HPI

The Hasso Plattner Institute (HPI), academically structured as the independent Faculty of Digital Engineering at the University of Potsdam, unites computer science research and teaching with the advantages of a privately financed institute and a tuition-free study program. Founder and namesake of the institute is the SAP co-founder Hasso Plattner, who also heads the Enterprise Platform and Integration Concepts (EPIC) research center which focuses on the technical aspects of business software with a vision to provide the fastest way to get insights out of enterprise data. Founded in 2006, the EPIC combines three research groups comprising autonomous data management, enterprise software engineering, and data-driven decision support.

Retracted] Stakeholders’ Viewpoints toward Blockchain Integration within IoT‐Based Smart Cities

This study intends to assess the development of IoT‐based smart cities industry and the possibilities of blockchain integration from the perspective of industry stakeholders as the vision for a modern, integrated smart city future is predicated upon intelligence and the relationship between data‐rich connections and human activities. Although this ideal of an interconnected urban landscape is currently being tested and actively used by consumers spanning a range of connected nodes and service solutions, the scalability, interoperability, and security of this emergent cyber‐physical ideal has yet to be adequately resolved. This study used an exploratory study design following a mixed method design approach. A structured questionnaire survey (quantitative) and interviews (qualitative) were conducted for collecting data. IBM SPSS was used for the analysis of the data, which computed descriptive statistics, cross‐tabulation, Pearson correlation, and ANOVA for quantitative data and thematic analysis for qualitative data. Through an empirical assessment of the perceptions and expertise of 122 stakeholders from within the worldwide IoT smart city industry, conceptual support for blockchain integration into the IoT solution was acquired, highlighting the solution‐oriented, system‐centered advantages of a decentralised, autonomous data management backbone that could be applied to future IoT‐based smart city solutions. To meet the broad and diversified needs of the smart city and its future evolution, this study has confirmed that a commitment to decentralisation and blockchain intermediary data management is critical to scalable, secure, and autonomous negotiations of the IoT‐enabled smart city networks.

Towards autonomic data management for staging-based coupled scientific workflows

Emerging scientific workflows running at extreme scale are composed of multiple applications that interact and exchange data at runtime. While staging-based approaches, e.g. in-situ/in-transit processing, are promising, dynamic behaviors (e.g. data volumes and distributions) in coupled applications and varying resource constraints at runtime make the efficient use of these techniques challenging. Addressing these challenges requires fundamental changes in the way that workflows are executed at runtime. Specifically, it is required to monitor the operating environment and running applications, and then adapt and tune the application behaviors and resource allocations at runtime while meeting the data management requirements and constraints. In this paper, we propose a policy-based autonomic data management (ADM) approach that can adaptively respond at runtime to dynamic data management requirements. We first formulate the schematic abstraction of this ADM approach including its conceptual model and system elements. Then, we explore the realization of ADM runtime and demonstrate how to achieve adaptations in a cross-layer manner with pre-defined autonomic policies. We also prototype our ADM approach and evaluate its performance on the Intrepid IBM-BlueGene and Titan Cray-XK7 systems using Chombo-based AMR applications and a visualization application. The experimental results demonstrate its effectiveness in meeting user defined objectives and accelerating overall scientific discovery.

Autonomic Data Management System in Grid Environment

In grid environment, a job may execute on a node that is geographically far away from its data files. These files are stored in heterogeneous storage systems located at geographically distributed virtual organizations. The current approach includes explicit data file transfers to execution nodes, which forces users to deal with different administrative policies at each site and various data access mechanisms on each storage system. This implies a lot of human interventions in order to develop dedicated programs and scripts for data transfers for job execution. We have developed a framework, called GRAVY, which enables transparent data access between distributed file systems irrespective of their heterogeneity. This feature enables highlevel schedulers integrated with GRAVY to control data placements like computational jobs (i.e., they can be queued, scheduled and monitored). In particular, GRAVY provides reliable and efficient data placement across multiple protocols and has a mechanism for transfer failure recovery without any human interaction.

Autonomics and data management

AbstractTraditionally, database management systems have been associated with high‐cost, high‐quality functionalities. That is, powerful capabilities are provided, but only in response to careful design, procurement, deployment and administration. This has been successful in many contexts, but in an environment in which data are available in increasing quantities under the management of a growing collection of distributed applications, and where effective use of available data often provides a competitive edge, there is a requirement for various benefits of a comprehensive data management infrastructure to be made available with rather fewer of the costs. If this requirement is to be met, automation will need to be deployed much more widely and systematically in data management platforms. This paper reviews recent results on autonomic data management, makes a case that current practice presents significant opportunities for further development and argues that comprehensive support for automation should be central to future data management infrastructures, whether centralized or distributed. Copyright © 2008 John Wiley & Sons, Ltd.

Autonomous Distributed Genome Database and Its Application to Mouse cDNA and Human Genome Sequence Matching

The vast increase of genomic sequence data reveals the limitation of single computer systems in terms of both processing and storage capacity. The only realistic way to cope with this growth in the near future will consist in using several computers rather than a single one. Catching up with the steady growth of genomic sequence data is likely to require systems with an increasingly large number of nodes. For this purpose, solutions based on PC clusters are attractive because they offer good performance for the price, and provide reasonable hardware scalability. Nevertheless, building such a system raises at least three important concerns: (1) the whole system, including the database management, must be scalable and accommodate a very large number of server nodes (say, tens of thousands), (2) the system must tolerate the failure individual nodes because this happen frequently as their number increases, and (3) the whole system should appear to clients as a single database image. Considering this context, we sketch the overall architecture of an autonomous scalable genome database which should satisfy the requirements mentioned above. The system consists of partially independent databases, with a management system to provides a single database image. Our prototype system makes use of XML to store information, and of Java servlet/applet technology for autonomous data management and remote access. As an illustration, we have tested on BLAST search results between mouse cDNA and human chromosome sequence.