Local Database Management Systems Research Articles

Design and Analysis of a Relational Database for Behavioral Experiments Data Processing

In this paper, the results of a comparative analysis between different approaches to experimental data storage and processing are presented. Several studies related to the problem and some methods for solving it have been discussed. Different types of databases, ways of using them and the areas of their application are analyzed. For the purposes of the study, a relational database for storing and analyzing a specific data from behavioral experiments was designed. The methodology and conditions for conducting the experiments are described. Three different indicators were analyzed, respectively: memory required to store the data, time to load the data from an external file into computer memory and iteration time across all records through one cycle. The obtained results show that for storing a large number of records (in the order of tens of millions of rows), either dynamic arrays (stored on external media in binary file format), or an approach based on a local or remote database management system can be used. Regarding the data loading time, the fastest approach was the one that uses dynamic arrays. It outperforms significantly the approaches based on a local or remote database. The obtained results show that the dynamic arrays and the local data sets approaches iterated much faster across all data records than the remote database approach. The paper concludes with proposal for further developments towards using of web services.

BioDWH: a data warehouse kit for life science data integration.

This paper presents a novel bioinformatics data warehouse software kit that integrates biological information from multiple public life science data sources into a local database management system. It stands out from other approaches by providing up-to-date integrated knowledge, platform and database independence as well as high usability and customization. This open source software can be used as a general infrastructure for integrative bioinformatics research and development. The advantages of the approach are realized by using a Java-based system architecture and object-relational mapping (ORM) technology. Finally, a practical application of the system is presented within the emerging area of medical bioinformatics to show the usefulness of the approach. The BioDWH data warehouse software is available for the scientific community at http://sourceforge.net/projects/biodwh/.

BioDWH: A Data Warehouse Kit for Life Science Data Integration

SummaryThis paper presents a novel bioinformatics data warehouse software kit that integrates biological information from multiple public life science data sources into a local database management system. It stands out from other approaches by providing up-to-date integrated knowledge, platform and database independence as well as high usability and customization. This open source software can be used as a general infrastructure for integrative bioinformatics research and development. The advantages of the approach are realized by using a Java-based system architecture and object-relational mapping (ORM) technology. Finally, a practical application of the system is presented within the emerging area of medical bioinformatics to show the usefulness of the approach.The BioDWH data warehouse software is available for the scientific community at http://sourceforge.net/projects/biodwh/.

Execution of distributed queries in the web environment

Database query processing is one of the fundamental issues in data engineering, particularly when more and more databases become distributed over the Internet in the last decade. There have been several approaches to the problem of distributed query processing (DQP), some were successful in the client-server environment but not so in an environment with heterogeneous and autonomous databases. The challenge relies on the nature of the Internet and the Web that pose many difficulties to the problem in hand. In this paper, we propose a model that uses an XML standard and its equivalent in-memory data structure to achieve data sharing, and invokes XML Web services in asynchronous mode to allow local database management systems to efficiently process local data. This approach, like XJoin, allows progress to be made even when data sources experience delays. It builds a layer on top of existing database management systems to minimize the complexity and cost for processing distributed queries that is critical for making such a method practical.

Overcoming Heterogeneity and Autonomy in Multidatabase Systems

A multidatabase system (MDBS) is a software system for integration of preexisting and independent local database management systems (DBMSs). The transaction management problem in MDBSs consists of designing appropriate software, on top of local DBMSs, such that users can execute transactions that span multiple local DBMSs without jeopardizing database consistency. The difficulty in transaction management in MDBSs arises due to the heterogeneity of the transaction management algorithms used by the local DBMSs, and the desire to preserve their local autonomy. In this paper, we develop a framework for designing fault-tolerant transaction management algorithms for MDBS environments that effectively overcomes the heterogeneity- and autonomy-induced problems. The developed framework builds on our previous work. It uses the approach described in S. Mehrotra et al. (1992, in “Proceedings of ACM–SIGMOD 1992 International Conference on Management of Data, San Diego, CA”) to overcome the problems in ensuring serializability that arise due to heterogeneity of the local concurrency control protocols. Furthermore, it uses a redo approach to recovery for ensuring transaction atomicity (Y. Breitbart et al., 1990, in “Proceedings of ACM–SIGMOD 1990 International Conference on Management of Data, Atlantic City, NJ;” Mehrotra et al., 1992, in “Proceedings of the Eleventh ACM SIGACT–SIGMOD–SIGART Symposium on Principles of Database Systems, San Diego, CA;” and A. Wolski and J. Veijalainen, 1990, in “Proceedings of the International Conference on Databases, Parallel Architectures and Their Applications”, pp. 321–330), that strives to ensure atomicity of transactions without the usage of the 2PC protocol. We reduce the task of ensuring serializability in MDBSs in the presence of failures to solving three independent subproblems, solutions to which together constitute a complete strategy for failure-resilient transaction management in MDBS environments. We develop mechanisms with which each of the three subproblems can be solved without requiring any changes be made to the preexisting software of the local DBMSs and without compromising their autonomy.

Rule-Based Model for Traffic Accident Visualization and Analysis

This research develops and tests a computerized model for intersection accident data visualization and analysis. The model implemented a rule-based visualization module that automatically constructs the collision diagram according to traffic flow conflicts, where accident characteristics can be analyzed either individually or collectively. A data acquisition module was developed for accessing accident databases maintained in state transportation and public safety departments. Using a local database management system as part of the model, the user can select to analyze any accident data through Structured Query Language statements. In addition, this model has implemented a data analysis and graphing module for comparing similarities and identifying differences in accident data and for presenting the results of the analysis through charts and tables. The model was tested with accident data obtained from the Ohio Department of Public Safety in a number of scenario studies, and some of them were presented in this paper as examples. Because the model is very time efficient and user friendly, it can be developed into an effective engineering tool for field applications.

Integrated Networking and Visualization Approach to Crash Data Analysis

An integrated networking and visualization approach to crash data analysis is proposed and tested. A three-tier client server networking system is set up to serve as the backbone framework for data communication between the central database and distributed users. By using a local database-management system, a user at the client level can view the crash data through data mapping and attribute listing and can analyze the data through nested query and sorting operations. A visualization module, based on the types, movement characteristics, and points of impact of the vehicles involved in the crash, automatically constructs the collision diagram by using data selected for visual display. Data on both intersections and pipeline sections can be visualized and analyzed. The model was tested by using crash data obtained from the Ohio Department of Public Safety, and examples are presented. It was discovered that the proposed approach to crash data analysis not only is very effective but also is more accurate and time efficient than the conventional data-analysis approach.

Ensuring consistency in multidatabases by preserving two-level serializability

The concept of serializability has been the traditionally accepted correctness criterion in database systems. However in multidatabase systems (MDBSs), ensuring global serializability is a difficult task. The difficulty arises due to the heterogeneity of the concurrency control protocols used by the participating local database management systems (DBMSs), and the desire to preserve the autonomy of the local DBMSs. In general, solutions to the global serializability problem result in executions with a low degree of concurrency. The alternative, relaxed serializability, may result in data inconsistency. In this article, we introduce a systematic approach to relaxing the serializability requirement in MDBS environments. Our approach exploits the structure of the integrity constraints and the nature of transaction programs to ensure consistency without requiring executions to be serializable. We develop a simple yet powerful classification of MDBSs based on the nature of integrity constraints and transaction programs. For each of the identified models we show how consistency can be preserved by ensuring that executions are two-level serializable (2LSR). 2LSR is a correctness criterion for MDBS environments weaker than serializability. What makes our approach interesting is that unlike global serializability, ensuring 2LSR in MDBS environments is relatively simple and protocols to ensure 2LSR permit a high degree of concurrency. Furthermore, we believe the range of models we consider cover many practical MDBS environments to which the results of this article can be applied to preserve database consistency.

Multidatabase transaction management scheme supporting multiple subtransactions of a global transaction at a site

The purpose of multidatabase systems is to allow users to access and manipulate data from existing databases in the distributed environment without modifying the existing database management systems (DBMSs). In the multidatabase environment, determining if a schedule for a set of global transactions is globally serializable is difficult because each local DBMS operates autonomously. Thus, we adopt the approach that relaxes the isolation and atomicity requirements of transactions. Moreover, in some applications, multiple subtransactions of a global transaction need to be submitted to a local site. To support this new transaction model, we propose a new correctness criterion for global schedules that is based on the semantic compatibility of global transactions. The conditions that global transactions should satisfy to ensure the correct executions are investigated, and the concept of well-formed global transaction is defined. We also present a concurrency control scheme that ensures the proposed correctness criterion. Our scheme uses both compensation and retrial techniques to eliminate the requirement of local prepare-to-commit operations, and hence it allows the global subtransactions to be committed unilaterally at local sites before the decision of global commit.

An extended transaction to maintain consistency and recovery in multidatabase systems

A complete practical solution to transaction management preserving multidatabase consistency in the presence of multidatabase updates and failures is presented. The approach developed does not require support for the two-phase commit (2PC) protocol in the participating local database management systems (LDBMSs). Furthermore, it does not violate local autonomy; the source code of the LDBMSs is not modified in any way and the multidatabase system (MDBS) does not access or modify any control information of the LDBMS. The principles of the 2PC protocol in the process of global transaction commitment are adopted. The presented method does not rely on any specific concurrency control mechanism for LDBMSs. Consideration is given to global transaction failures due to subtransaction aborts by the LDBMSs and local site crashes. The recovery process is based on undo operations. While a global transaction is in progress, the tables accessed by subtransactions of this transaction at each local site are locked using specially initiated table locks. These locks are stored and maintained in the local database itself as control tables. The approach taken is easy to implement, and its limitations are discussed.

Performance issues in distributed query processing

The authors discuss various performance issues in distributed query processing. They validate and evaluate the performance of the local reduction (LR) the fragment and replicate strategy (FRS) and the partition and replicate strategy (PRS) optimization algorithms. The experimental results reveal that the choices made by these algorithms concerning which local operations should be performed, which relation should remain fragmented or which relation should be partitioned are valid. It is shown using experimental results that various parameters, such as the number of processing sites, partitioning speed relative to join speed, and sizes of the join relations, affect the performance of PRS significantly. It is also shown that the response times of query execution are affected significantly by the degree of site autonomy, interferences among processes, interface with the local database management systems (DBMSs) and communications facilities. Pipeline strategies for processing queries in an environment where relations are fragmented are studied. >

Overview of multidatabase transaction management

A multidatabase system (MDBS) is a facility that allows users access to data located in multiple autonomous database management systems (DBMSs). In such a system, global transactions are executed under the control of the MDBS. Independently, local transactions are executed under the control of the local DBMSs. Each local DBMS integrated by the MDBS may employ a different transaction management scheme. In addition, each local DBMS has complete control over all transactions (global and local) executing at its site, including the ability to abort at any point any of the transactions executing at its site. Typically, no design or internal DBMS structure changes are allowed in order to accommodate the MDBS. Furthermore, the local DBMSs may not be aware of each other and, as a consequence, cannot coordinate their actions. Thus, traditional techniques for ensuring transaction atomicity and consistency in homogeneous distributed database systems may not be appropriate for an MDBS environment. The objective of this article is to provide a brief review of the most current work in the area of multidatabase transaction management. We first define the problem and argue that the multidatabase research will become increasingly important in the coming years. We then outline basic research issues in multidatabase transaction management and review recent results in the area. We conclude with a discussion of open problems and practical implications of this research.

The concurrency control problem in multidatabases

A Multidatabase System (MDBS) is a collection of local database management systems, each of which may follow a different concurrency control protocol. This heterogeneity makes the task of ensuring global serializability in an MDBS environment difficult. In this paper, we reduce the problem of ensuring global serializability to the problem of ensuring serializability in a centralized database system. We identify characteristics of the concurrency control problem in an MDBS environment, and additional requirements on concurrency control schemes for ensuring global serializability. We then develop a range of concurrency control schemes that ensure global serializability in an MDBS environment, and at the same time meet the requirements. Finally, we study the tradeoffs between the complexities of the various schemes and the degree of concurrency provided by each of them.

Global checkpointing scheme for heterogeneous distributed database systems

We propose a transaction-consistent checkpointing scheme for heterogeneous distributed database systems. Using the global checkpoints generated by the checkpointing scheme, the heterogeneous distributed database system can efficiently reconstruct the database to the most recent consistent state from media failures. The checkpointing scheme never enforces termination of normal operations of transactions and changes of checkpointing schemes in local database management systems. Thus, the checkpointing scheme guarantees the design autonomy of local database management systems and increases the availability on the heterogeneous distributed database systems.

Architectural considerations for providing statistical analysis of distributed data

Many researchers have addressed the problem of providing statistical analysis for centralized databases. Perhaps surprisingly, however, no one has yet carefully examined the architectures and issues arising when data are distributed over many sites. The paper proposes two approaches to providing an architecture for a system that gives system support for statistical analyses in situations where the underlying raw data are distributed among different (heterogeneous) database management systems (DBMSs) on different computing facilities at different sites. The underlying local DBMSs are not required to provide any statistical capabilities, but advantage can be taken of such capability where it is present. The first approach uses a distributed database management system (DDBMS) to obtain the raw data from the local DBMSs and provides statistical analyses at the global level. Two system configurations are proposed for this approach. The second approach distributes as much as possible of the statistical analyses to the local sites and uses partially processed data from the sites as input to the global analysis procedures. A system has been implemented to illustrate the functionality of the first approach, but its major drawback is the amount of raw data that has to be shipped around the network. In the second approach, which is currently being used as the basis of an implementation, normally only summary data are moved between sites; nodes are required simply to present statistical participation views over which global statistical queries involving a wide range of statistical analyses can be evaluated. A number of challenging and interesting design decisions on modular details must be made to permit the architecture to be refined for applications in particular domains. The paper does not give detailed solutions to the problems identified but provides a discussion of these within an architectural framework that has evolved from work on the existing implementations. It promises to be more pragmatic than that used for current implementations.

Heterogeneous distributed database systems for production use

It is increasingly important for organizations to achieve additional coordination of diverse computerized operations. To do so, it is necessary to have database systems that can operate over a distributed network and can encompass a heterogeneous mix of computers, operating systems, communications links, and local database management systems. This paper outlines approaches to various aspects of heterogeneous distributed data management and describes the characteristics and architectures of seven existing heterogeneous distributed database systems developed for production use. The objective is a survey of the state of the art in systems targeted for production environments as opposed to research prototypes.

A natural language interface to a multiple databased office information system

As an organization grows through mergers and acquisitions and through the integration of independently designed information sub-systems, it could find itself faced with a collection of databases implemented on heterogeneous database management systems. This paper presents a system to provide integrated access to multiple, independently designed databases through a portable natural language front-end and a multiple database access system back-end.The overall system is divided into a natural language interface (NLI) front-end and a multiple database access system (MDAS) back-end. The MDAS integrates the various conceptual views into a single global external schema. In a prototype of the overall system, the NLI will accept a natural language query and produce a relational calculus query expression over a global external schema. The MDAS will construct the response to the query by accessing the separate constituent databases through the local database management systems (DBMS's).

Transaction management in distributed heterogeneous database management systems

Current distributed, heterogeneous database management systems (DBMSs) address the issue of distributed transaction management (DTM) in two different ways: some systems rely solely on unmodified local transaction managers (LTMs), thereby helping preserve local DBMS autonomy, but limit functionality (e.g. allow only unsynchronized retrievals, preclude distributed updates, etc.); others maintain full functionality but require the (re)design of the LTMs to enforce homogeneous DTM across all heterogeneous DBMSs, thereby giving up most of the local DBMS autonomy. The goal of the work presented here is to establish the minimum set of modifications to LTMs that allow synchronized retrievals and distributed updates (whenever semantic conflicts can be resolved), and will continue to maintain a high degree of local DBMS autonomy. The problems of: (1) distribution of responsibility between DTM and LTMs, (2) concatenation of functionally-equivalent LTM mechanisms, and (3) providing compensation mechanisms for functionally-limited LTMs are introduced. Solutions to the above problems are shown to exist. However, the interconnection of autonomous, heterogeneous DBMSs is significantly more difficult than anticipated, despite communication standardization and current optimism.