Commercial Database Management Systems Research Articles

The Catalog Archive Server Database Management System

The multiterabyte Sloan Digital Sky Survey's (SDSS's) catalog data is stored in a commercial relational database management system with SQL query access and a built-in query optimizer. The SDSS catalog archive server adds advanced data mining features to the DBMS to provide fast online access to the data.

Error minimization in approximate range aggregates

Histogram techniques have been used in many commercial database management systems to estimate a query result size. Recently, it has been shown that they are very effective to support approximation of query processing especially aggregates. In this paper, we investigate the problem of minimizing average errors of approximate aggregates using histogram techniques. Firstly, we present a novel linear-spline histogram model that is more accurate than the existing models. Secondly, we propose a novel histogram construction technique for minimizing such average errors, which is shown to generate a near optimal histogram. Our experiment results demonstrate that the new histogram construction techniques lead to a great accuracy improvement on the existing techniques.

Developing secure data warehouses with a UML extension

Data Warehouses (DWs), Multidimensional (MD) Databases, and On-Line Analytical Processing Applications are used as a very powerful mechanism for discovering crucial business information. Considering the extreme importance of the information managed by these kinds of applications, it is essential to specify security measures from the early stages of the DW design in the MD modeling process, and enforce them. In the past years, some proposals for representing main MD modeling properties at the conceptual level have been stated. Nevertheless, none of these proposals considers security issues as an important element in its model, so they do not allow us to specify confidentiality constraints to be enforced by the applications that will use these MD models. In this paper, we will discuss the specific confidentiality problems regarding DWs as well as present an extension of the Unified Modeling Language for specifying security constraints in the conceptual MD modeling, thereby allowing us to design secure DWs. One key advantage of our approach is that we accomplish the conceptual modeling of secure DWs independently of the target platform where the DW has to be implemented, allowing the implementation of the corresponding DWs on any secure commercial database management system. Finally, we will present a case study to show how a conceptual model designed with our approach can be directly implemented on top of Oracle 10g.

Database architectures: Current trends and their relationships to environmental data management

Ever increasing environmental information demands from customers, authorities, and governmental organizations as well as new business control functions are implemented and integrated to environmental information management systems (EIMSs). These systems are often based on traditional file techniques or, more recently, on commercial database management systems (DBMSs). With a production of huge data sets and their processing in real-time applications, the needs for environmental data management have grown significantly. Numerous examples from practice of EIMSs prove that the architecture of DBMS should be open for a permanent evolution. Current trends in database development and an associated research meet these challenges. New information and communication technologies and techniques influence today's DBMSs. They include, among other things, sensor networks, stream processing, processing uncertain and imprecise data, knowledge discovery and intelligent data analysis, as well as wireless broadcast and mobile computing. Both research and practice indicate that the traditional universal DBMS architecture hardly satisfies these trends and new solutions are needed. Rather separate specialized engines connected into networks are beneficial. The paper discusses recent advances in database technologies and attempts to highlight them with respect to requirements of EIMSs.

Scientific formats for object-relational database systems

Commercial database management systems (DBMSs) have historically seen very limited use within the scientific computing community. One reason for this absence is that previous database systems lacked support for the extensible data structures and performance features required within a high-performance computing context. However, database vendors have recently enhanced the functionality of their systems by adding object extensions to the relational engine. In principle, these extensions allow for the representation of a rich collection of scientific datatypes and common statistical operations. Utilizing these new extensions, this paper presents a study of the suitability of incorporating two popular scientific formats, NetCDF and HDF, into an object-relational system. To assess the performance of the database approach, a series of solution variables from a regional weather forecast model are used to build representative small, medium and large databases. Common statistical operations and array element queries are then performed using the object-relational database, and the execution timings are compared against native NetCDF and HDF operations.

The problem of missing data in geoscience databases

SQL is the (more or less) standardised language that is used by the majority of commercial database management systems. However, it is seriously flawed, as has been documented in detail by Date, Darwen, Pascal, and others. One of the most serious problems with SQL is the way it handles missing data. It uses a special value ‘NULL’ to represent data items whose value is not known. This can have a variety of meanings in different circumstances (such as ‘inapplicable’ or ‘unknown’). The SQL language also allows an ‘unknown’ truth value in logical expressions. The resulting incomplete three-valued logic leads to inconsistencies in data handling within relational database management systems. Relational database theorists advocate that a strict two-valued logic (true/false) be used instead, with prohibition of the use of NULL, and justify this stance by assertion that it is a true representation of the ‘real world’. Nevertheless, in real geoscience data there is a complete gradation between exact values and missing data: for example, geochemical analyses are inexact (and the uncertainty should be recorded); the precision of numeric or textual data may also be expressed qualitatively by terms such as ‘approximately’ or ‘possibly’. Furthermore, some data are by their nature incomplete: for example, where samples could not be collected or measurements could not be taken because of inaccessibility. It is proposed in this paper that the best way to handle such data sets is to replace the closed-world assumption and its concomitant strict two-valued logic, upon which the present relational database model is based, by the open-world assumption which allows for other logical values in addition to the extremes of ‘true’ and ‘false’. Possible frameworks for such a system are explored, and could use Codd's ‘marks’, Darwen's approach (recording the status of information known about each data item), or other approaches such as fuzzy logic.

Multidisciplinary aircraft design and evaluation software integrating CAD, analysis, database, and optimization

The development of an overall strategy to design the aircraft using analysis codes is presented. The procedure is automated through the integrated software MADE (multidisciplinary aircraft design and evaluation) developed in this research that enables the determination of an optimum set of aircraft configurations. The core ingredients of MADE are (i) analysis codes, which utilizes initial sizing, aerodynamics, mass, stability and control, propulsion, performance, and RCS (radar cross section) analyses, (ii) optimization utilizing gradient-based optimization technique, response surface modeling, and Carpet plot, (iii) database utilizing commercial Oracle 8i database management system (DBMS) and OCI (Oracle call interface) to save design parameters and aircraft configurations. The complexity of input and output containing massive amounts of data has previously placed severe limitations on the use of aircraft analysis codes by the general aerospace engineering community. By using C++ and the Microsoft Foundation Classes to develop a graphical user interface (GUI) and using DBMS, it is believed that use of MADE can avoid the additional cumbersome, and sometimes tedious, task of preparing the required input files manually and can make the transition to general usability in an aerospace engineering environment. For detail explanation, examples of E–R diagram, class diagram, and data flows between codes for the MADE are also presented.

The man who wasn’t there: The problem of partially missing data

Existing commercial database management systems offer little or no functionality to handle the complexity of geoscience data—and other environmental science data—particularly in respect of missing and partially missing (incomplete or imprecise) data items. The emphasis of both the relational theorists (Codd, Date, and others) and the developers of database systems is on commercial applications where only rudimentary treatment of missing data is required, in the form of NULLs, and even these are not handled properly by the SQL language.

VI-attached database storage

This work presents a Vl-attached database storage architecture to improve database transaction rates. More specifically, we examine how Vl-based interconnects can be used to improve I/O path performance between a database server and a storage subsystem. To facilitate the interaction between client applications and a Vl-aware storage system, we design and implement a software layer called DSA, that is layered between applications and VI. DSA takes advantage of specific VI features and deals with many of its shortcomings. We provide and evaluate one kernel-level and two user-level implementations of DSA. These implementations trade transparency and generality for performance at different degrees and, unlike research prototypes, are designed to be suitable for real-world deployment. We have also investigated many design trade offs in the storage cluster. We present detailed measurements using a commercial database management system with both microbenchmarks and industrial database workloads on a mid-size, 4 CPU, and a large, 32 CPU, database server. We also compare the effectiveness of Vl-attached storage with an iSCSI configuration, and conclude that storage protocols implemented using DSA over VI have significant performance advantages. More generally, our results show that Vl-based interconnects and user-level communication can improve all aspects of the I/O path between the database system and the storage back-end. We also find that to make effective use of VI in I/O intensive environments, we need to provide substantial additional functionality than what is currently provided by VI. Finally, new storage APIs that help minimize kernel involvement in the I/O path are needed to fully exploit the benefits of Vl-based communication.

Information Aggregation using the Cameleon# Web Wrapper

Cameleon# is a web data extraction and management tool that provides information aggregation with advanced capabilities that are useful for developing value-added applications and services for electronic business and electronic commerce. To illustrate its features, we use an airfare aggregation example that collects data from eight online sites, including Travelocity, Orbitz, and Expedia. This paper covers the integration of Cameleon# with commercial database management systems, such as MS SQL Server, and XML query languages, such as XQuery.

Database support for mobile route planning systems

The impact of tele and mobile information technology will increase the need for fast access to geodata significantly in the next years. However, hitherto many problems concerning the wireless access to geodata e.g. to be used in route planning clients are not yet solved. Beside transaction management the requirements of route planning systems to GIS technology mainly concern the modelling and management of graph-based geodata. New challenges to meet are the efficient storage of routes and the supply of location-based database queries. However, today's first commercial database management systems for mobile devices do not support spatial database queries. In this article we collect requirements and present a first implementation prototype of a mobile route planning system focusing on the support of spatial database queries. First experiences with a mobile bicycle route planning client coupled with a database management system are presented. We show that at the server side the computation and storage of different route types as well as the generation of routes under consideration of semantic information like tourist information or roads with certain points of interest should make up the main functionality of such a system. The mobile client should supply database support for the location-based visualization of routes and an offline data management of route data.

Tools for loading MEDLINE into a local relational database

BackgroundResearchers who use MEDLINE for text mining, information extraction, or natural language processing may benefit from having a copy of MEDLINE that they can manage locally. The National Library of Medicine (NLM) distributes MEDLINE in eXtensible Markup Language (XML)-formatted text files, but it is difficult to query MEDLINE in that format. We have developed software tools to parse the MEDLINE data files and load their contents into a relational database. Although the task is conceptually straightforward, the size and scope of MEDLINE make the task nontrivial. Given the increasing importance of text analysis in biology and medicine, we believe a local installation of MEDLINE will provide helpful computing infrastructure for researchers.ResultsWe developed three software packages that parse and load MEDLINE, and ran each package to install separate instances of the MEDLINE database. For each installation, we collected data on loading time and disk-space utilization to provide examples of the process in different settings. Settings differed in terms of commercial database-management system (IBM DB2 or Oracle 9i), processor (Intel or Sun), programming language of installation software (Java or Perl), and methods employed in different versions of the software. The loading times for the three installations were 76 hours, 196 hours, and 132 hours, and disk-space utilization was 46.3 GB, 37.7 GB, and 31.6 GB, respectively. Loading times varied due to a variety of differences among the systems. Loading time also depended on whether data were written to intermediate files or not, and on whether input files were processed in sequence or in parallel. Disk-space utilization depended on the number of MEDLINE files processed, amount of indexing, and whether abstracts were stored as character large objects or truncated.ConclusionsRelational database (RDBMS) technology supports indexing and querying of very large datasets, and can accommodate a locally stored version of MEDLINE. RDBMS systems support a wide range of queries and facilitate certain tasks that are not directly supported by the application programming interface to PubMed. Because there is variation in hardware, software, and network infrastructures across sites, we cannot predict the exact time required for a user to load MEDLINE, but our results suggest that performance of the software is reasonable. Our database schemas and conversion software are publicly available at .

APPLYING DATABASE TECHNOLOGY TO CLINICAL AND BASIC RESEARCH BIOINFORMATICS PROJECTS

This paper describes the application of database technology to medical information with the goal of providing medical and clinical researchers with the tools necessary to plan bioinformatics projects. Commercial database management systems were utilized, standard database design practices were applied, a user interface was created, data entered, and the development of analysis tools, including data mining technologies is underway. Databases were constructed based on animal and cell culture models of diabetes and clinical data. Bioinformatics is a useful tool in both basic research and clinical settings. The advantages of relational databases and an approach to managing bioinformatics projects are discussed.

Encapsulation of geometric functions for ship structural CAD using a STEP database as native storage

It is difficult to support collaborative design with a conventional ship CAD system that manages design information using files. The file storage, however, can be replaced with a commercial database management system. This paper describes OpenDIS, which is an interface between the geometric modeling kernel and the DBMS. The main purpose of OpenDIS is to implement a CAD system that has the STEP database as the native storage. A prototype CAD system has been implemented using the OpenCascade geometric modeling kernel and ObjectStore, a commercial object-oriented DBMS. The STEP methodology is used for the database schema. This CAD system has been applied to the hull design process of a ship in order to verify the usefulness of the interface.

Data management challenges for molecular and cell biology: an industry perspective.

OMICS: A Journal of Integrative BiologyVol. 7, No. 1 Original PapersData Management Challenges for Molecular and Cell Biology: An Industry PerspectiveVictor M. MarkowitzVictor M. MarkowitzSearch for more papers by this authorPublished Online:8 Jul 2004https://doi.org/10.1089/153623103322006760AboutSectionsPDF/EPUB Permissions & CitationsPermissionsDownload CitationsTrack CitationsAdd to favorites Back To Publication ShareShare onFacebookTwitterLinked InRedditEmail "Data Management Challenges for Molecular and Cell Biology: An Industry Perspective." , 7(1), pp. 121–122FiguresReferencesRelatedDetails Volume 7Issue 1Jan 2003 To cite this article:Victor M. Markowitz.Data Management Challenges for Molecular and Cell Biology: An Industry Perspective.OMICS: A Journal of Integrative Biology.Jan 2003.121-122.http://doi.org/10.1089/153623103322006760Published in Volume: 7 Issue 1: July 8, 2004PDF download

A Retrospection On Niche Database Technologies.

Today commercial relational database management systems (RDBs) and supporting tools (database design tools, graphical database browsers, application generators, query tools, etc.) from several major vendors are used to suit most data management needs around the world. The database market is now dominated by the Big Three: IBM, Microsoft, and Oracle. Initially, commercial RDBs supported the first normal form relational data model and SQL query language, along with basic transaction management and access authorization facilities. They mostly ran on a few UNIX platforms. The scope and sophistication of the functionality they support, and the performance they deliver have all substantially matured during the past twenty years. They have also been made to run on most computers and operating systems. However, the often frustratingly long periods that it took major vendors to shore up various shortcomings of RDBs have led to the introduction of what I will call “niche database” technologies during this period of maturation. Each niche database technology came about to address typically one major shortcoming in then-existing RDBs. The technologies the vendors introduced and the efforts they made to market the technologies often provided a key incentive to major vendors to upgrade their RDBs with the same technologies. The financial wherewithal, customer loyalty, and marketing prowess of the major RDB vendors have simply overpowered vendors of niche database technologies. For these and other reasons, except for about a dozen vendors, niche database technology vendors have generally failed to grow into sustained large business entities. Today the maturity of RDBs, the expectation of the customer base, and the domination of the market by the Big Three make it very difficult for the entry of new niche database technology vendors.

Project management system for structural and functional proteomics: Sesame.

A computing infrastructure (Sesame) has been designed to manage and link individual steps in complex projects. Sesame is being developed to support a large-scale structural proteomics pilot project. When complete, the system is expected to manage all steps from target selection to data-bank deposition and report writing. We report here on the design criteria of the Sesame system and on results demonstrating successful achievement of the basic goals of its architecture. The Sesame software package, which follows the client/server paradigm, consists of a framework, which supports secure interactions among the three tiers of the system (the client, server, and database tiers), and application modules that carry out specific tasks. The framework utilizes industry standards. The client tier is written in Java2 and can be accessed anywhere through the Internet. All the development on the server tier is also carried out in Java2 so as to accommodate a wide variety of computer platforms. The database tier employs a commercial database management system. Each Sesame application module consists of a simple user interface in the client tier, corresponding objects in the server tier, and relevant data stored in the centralized database. For security, access to stored data is controlled by access privileges. The system facilitates both local and remote collaborations. Because users interact with the system using Java Web Start or through a web browser, access is limited only by the availability of an Internet connection. We describe several Sesame modules that have been developed to the point where they are being utilized routinely to support steps involved in structural and functional proteomics. This software is available to parties interested in using it and assisting to guide its further development.

Processing OLAP queries in hierarchically clustered databases

On-Line Analytical Processing (OLAP) is a technology that encompasses applications requiring a multidimensional and hierarchical view of data. OLAP applications often require fast response time to complex grouping/aggregation queries on enormous quantities of data. Commercial relational database management systems use mainly multiple one-dimensional indexes to process OLAP queries that restrict multiple dimensions. However, in many cases, multidimensional access methods outperform one-dimensional indexing methods. We present an architecture for multidimensional databases that are clustered with respect to multiple hierarchical dimensions. It is based on the star schema and is called CSB star. We focus on processing OLAP queries over this schema using multidimensional access methods. Users can still formulate their queries over a traditional star schema, which are then rewritten by the query processor over the CSB star. We exploit the different clustering features of the CSB star to efficiently process a class of typical OLAP queries. We detect cases where the construction of an evaluation plan can be simplified, and other cases where additional processing techniques can be applied.

Experiences with VI communication for database storage

This paper examines how VI-based interconnects can be used to improve I/O path performance between a database server and the storage subsystem. We design and implement a software layer, DSA, that is layered between the application and VI. DSA takes advantage of specific VI features and deals with many of its shortcomings. We provide and evaluate one kernel-level and two user-level implementations of DSA. These implementations trade transparency and generality for performance at different degrees, and unlike research prototypes are designed to be suitable for real-world deployment. We present detailed measurements using a commercial database management system with both micro-benchmarks and industrial database workloads on a mid-size, 4 CPU, and a large, 32 CPU, database server.Our results show that VI-based interconnects and user-level communication can improve all aspects of the I/O path between the database system and the storage back-end. We also find that to make effective use of VI in I/O intensive environments we need to provide substantial additional functionality than what is currently provided by VI. Finally, new storage APIs that help minimize kernel involvement in the I/O path are needed to fully exploit the benefits of VI-based communication.

Global lock escalation in database management systems

Since database management systems (DBMSs) have limited lock resources, transactions requesting locks beyond the limit must be aborted, degrading the performance abruptly. Lock escalation can be effectively used in such circumstances to alleviate the problem. Many lock escalation methods have been proposed and implemented in commercial DBMSs. However, they have certain problems due to the local nature of their decisions on when to execute lock escalation. In this paper, we propose a new lock escalation method, global lock escalation, that makes such decision globally based on the total number of locks. Through extensive simulation, we show that the global lock escalation method outperforms the existing ones significantly. Especially, we show that the number of concurrent transactions allowable increases by 2–16 times. We believe our method can be easily implemented in the commercial DBMSs enhancing the performance significantly under excessive lock requests.