Internal Data Representation Research Articles

ObsPy: a bridge for seismology into the scientific Python ecosystem

The Python libraries NumPy and SciPy are extremely powerful tools for numerical processing and analysis well suited to a large variety of applications. We developed ObsPy (http://obspy.org), a Python library for seismology intended to facilitate the development of seismological software packages and workflows, to utilize these abilities and provide a bridge for seismology into the larger scientific Python ecosystem. Scientists in many domains who wish to convert their existing tools and applications to take advantage of a platform like the one Python provides are confronted with several hurdles such as special file formats, unknown terminology, and no suitable replacement for a non-trivial piece of software. We present an approach to implement a domain-specific time series library on top of the scientific NumPy stack. In so doing, we show a realization of an abstract internal representation of time series data permitting I/O support for a diverse collection of file formats. Then we detail the integration and repurposing of well established legacy codes, enabling them to be used in modern workflows composed in Python. Finally we present a case study on how to integrate research code into ObsPy, opening it to the broader community. While the implementations presented in this work are specific to seismology, many of the described concepts and abstractions are directly applicable to other sciences, especially to those with an emphasis on time series analysis.

An optimized stereo vision implementation for embedded systems: application to RGB and infra-red images

The aim of this paper is to demonstrate the applicability and the effectiveness of a computationally demanding stereo-matching algorithm in different low-cost and low-complexity embedded devices, by focusing on the analysis of timing and image quality performances. Various optimizations have been implemented to allow its deployment on specific hardware architectures while decreasing memory and processing time requirements: (1) reduction of color channel information and resolution for input images; (2) low-level software optimizations such as parallel computation, replacement of function calls or loop unrolling; (3) reduction of redundant data structures and internal data representation. The feasibility of a stereo vision system on a low-cost platform is evaluated by using standard datasets and images taken from infra-red cameras. Analysis of the resulting disparity map accuracy with respect to a full-size dataset is performed as well as the testing of suboptimal solutions.

Towards effective GML data management: framework and prototype

Geography Markup Language (GML) is composed of non-spatial information as properties of geographical features and spatial information as geometry. With the feature of XML-encoding and well-formed specification, GML has been widely used in various application systems as internal representation of geo-spatial data for its efficient interoperability in heterogeneous systems. This arises the demand to effectively manipulate GML data. The separate GML processing technology cannot provide a whole geo-spatial data management service for users. In this paper, we propose a GML data management framework and analyse how the relative technologies cooperate to meet the demand of geo-spatial data representing, processing and exchanging. Storage, index and query approaches, and corresponding integration technology are presented based on the proposed framework. The prototype of GML data management system (GDMS) was implemented based on the proposed framework.

Mapping between heterogeneous XML and OWL transaction representations in B2B integration

XML-based standards have been widely used to enable and ease Business-to-Business (B2B) integration. Examples of standards include cXML, CIDX and ebXML. While these XML-based standards are syntactic, contemporary organizations have available new means to structure their internal data representations using semantic descriptions, such as RDF(S) and OWL. This scenario poses an interesting challenge: “How to reconcile external XML-based standards and internal OWL-based representations in B2B integration scenarios?” In this paper, we present a conceptual approach, and its implementation, to integrate external syntactic data representations with organizational internal semantic data representations by using the notion of heterogeneous mappings which are established between the two types of representations. The application developed, B2BISS, enables an effective management of mappings. As the number of mappings stored in the repository increases over time, organizations can gradually rely on a semi-automatic to automatic B2B integration.

A relational modal logic for higher-order stateful ADTs

The method of logical relations is a classic technique for proving the equivalence of higher-order programs that implement the same observable behavior but employ different internal data representations. Although it was originally studied for pure, strongly normalizing languages like System F, it has been extended over the past two decades to reason about increasingly realistic languages. In particular, Appel and McAllester's idea of step-indexing has been used recently to develop syntactic Kripke logical relations for ML-like languages that mix functional and imperative forms of data abstraction. However, while step-indexed models are powerful tools, reasoning with them directly is quite painful, as one is forced to engage in tedious step-index arithmetic to derive even simple results. In this paper, we propose a logic LADR for equational reasoning about higher-order programs in the presence of existential type abstraction, general recursive types, and higher-order mutable state. LADR exhibits a novel synthesis of features from Plotkin-Abadi logic, Gödel-Löb logic, S4 modal logic, and relational separation logic. Our model of LADR is based on Ahmed, Dreyer, and Rossberg's state-of-the-art step-indexed Kripke logical relation, which was designed to facilitate proofs of representation independence for "state-dependent" ADTs. LADR enables one to express such proofs at a much higher level, without counting steps or reasoning about the subtle, step-stratified construction of possible worlds.

First-order complete and computationally complete query languages for spatio-temporal databases

We address a fundamental question concerning spatio-temporal database systems: “What are exactly spatio-temporal queries?” We define spatio-temporal queries to be computable mappings that are also generic , meaning that the result of a query may only depend to a limited extent on the actual internal representation of the spatio-temporal data. Genericity is defined as invariance under groups of geometric transformations that preserve certain characteristics of spatio-temporal data (e.g., collinearity, distance, velocity, acceleration, …). These groups depend on the notions that are relevant in particular spatio-temporal database applications. These transformations also have the distinctive property that they respect the monotone and unidirectional nature of time. We investigate different genericity classes with respect to the constraint database model for spatio-temporal databases and we identify sound and complete languages for the first-order and the computable queries in these genericity classes. We distinguish between genericity determined by time-invariant transformations, genericity notions concerning physical quantities and genericity determined by time-dependent transformations.

Canonical Abstract Syntax Trees

This paper presents GOM, a language for describing abstract syntax trees and generating a Java implementation for those trees. GOM includes features allowing the user to specify and modify the interface of the data structure. These features provide in particular the capability to maintain the internal representation of data in canonical form with respect to a rewrite system. This explicitly guarantees that the client program only manipulates normal forms for this rewrite system, a feature which is only implicitly used in many implementations.

Teaching cleanroom software engineering with object-oriented data abstraction

This paper describes the benefit of incorporating two ideas from Design-by-Contract [3] into the teaching of Cleanroom Software Engineering [4][5], namely the specification of method behavior in terms of inspector methods, and inheritance of method specifications. The inspectors serve as a specification of the transformation between the abstract data type represented by a class and its internal data representation. By inheriting method specifications, the behavior of augmented polymorphic methods can be specified. Pedagogical examples and student code are presented and discussed.

Numerics of High Performance Computers and Benchmark Evaluation of Distributed Memory Computers

The internal representation of numerical data, their speed of manipulation to generate the desired result through efficient utilisation of central processing unit, memory, and communication links are essential steps of all high performance scientific computations. Machine parameters, in particular, reveal accuracy and error bounds of computation, required for performance tuning of codes. This paper reports diagnosis of machine parameters, measurement of computing power of several workstations, serial and parallel computers, and a component-wise test procedure for distributed memory computers. Hierarchical memory structure is illustrated by block copying and unrolling techniques. Locality of reference for cache reuse of data is amply demonstrated by fast Fourier transform codes. Cache and register-blocking technique results in their optimum utilisation with consequent gain in throughput during vector-matrix operations. Implementation of these memory management techniques reduces cache inefficiency loss, which is known to be proportional to the number of processors. Of the two Linux clusters-ANUP16, HPC22 and HPC64, it has been found from the measurement of intrinsic parameters and from application benchmark of multi-block Euler code test run that ANUP16 is suitable for problems that exhibit fine-grained parallelism. The delivered performance of ANUP16 is of immense utility for developing high-end PC clusters like HPC64 and customised parallel computers with added advantage of speed and high degree of parallelism.

Reducibility between Classes of Port Graph Grammar

This paper introduces a general notion of static-port graph grammar (PGG) that encompasses existing formalisms that have been independently proposed, such as linear graph grammars, interaction nets and partial sharing graphs. These formalisms provide a computational framework for asynchronous computation that respects a very rigorous notion of local interaction, and have proven a suitable basis for the internal representation of program and data in the compilation of high-level programming languages for distributed execution. The general class of PGGs introduced in this paper provides a more expressive computational framework, still possessing asynchronous concurrency but which has a reduction mechanism that is complex and non-local. The increased expressiveness of this calculus is shown by means of a motivating example of concurrent pattern matching. However, the non-locality of the calculus renders it unsuitable for direct implementation. The paper introduces a class of simple PGGs, a local rewriting calculus suitable for distributed implementation. The central result of the paper is to show how the general class of PGGs may be reduced to the simple PGGs, in a manner that fully preserves all the potential for concurrent rewriting present in the original.

Real-time software specification and validation with Transnet

The paper presents the executable specification method for real-time (embedded) systems Transnet. The method is based on an extension to Petri nets. A specification is developed by a problem decomposition into a set of parallel processes. Each process is defined by an extended Petri net with functions assigned to transitions, and conditions and time constants assigned to arcs. It is shown that Transnet matches with the characteristics of the intended class of applications. The available specification languages and the internal representation of data which describe the functional as well as nonfunctional requirements are outlined. The advantages of the Petri net-based representation and the possibilities of automatic net analysis and early validation are discussed.

Neural network models of nuclear systematics

We describe a novel phenomenological approach to many-body systems based on multilayer feedforward neural networks. When subjected to appropriate training schemes, such networks are capable of learning the systematics of atomic masses and nuclear spins and parities with high accuracy while forming an efficient internal representation of the training data. When tested on nuclei outside the training set, these neural-network models demonstrate a predictive power competitive with that of traditional theoretical approaches, provided the test nuclei are not too different from those of the training set. The relative performance on training and test sets may be used as a measure of the homology of nuclei with respect to given observables.

Feature taxonomies for rotational parts: a review and proposed taxonomies

Abstract A major element of computer-integrated manufacturing is the link between computer-aided design (CAD) on one side, and computer-aided manufacturing (CAM) and process planning on the other. This paper is concerned with a key aspect of this link, that is the definition of a feature taxonomy for rotational parts and the specification of the features and their associated parameters. The aims of this paper are to review existing taxonomies and to define new feature taxonomies for rotational parts using the basic structure of the Product Data Exchange Specification (PDES). PDES is intended to be an internal data representation scheme and, as such, is relatively complex. The feature taxonomies presented in this paper provide, in effect, a natural interface to the PDES internal data representation. The work done on feature-based CAD systems is reviewed. The background to rotational part features is described and an overview of PDES is given. Two feature taxonomies for rotational parts are then described. ...

Implementation and design of PVD: An interactive protocol specification and validation environment

The Protocol Validation Development system (PVD) is based on the Finite State Machine (FSM) model to provide an interactive protocol specification and validation software environment, It is used to help the protocol designer to design and validate communication protocols. The PVD supports a specification tool to specify two processes FSM communication protocols by using graphic schema as the protocol input stage, and integrates several tools that implement two validation methods of the conventional exhaustive state exploration [9, 12] and the random walk state exploration [10, 11]. By the utilization of these validation tools to analyze the given protocol, one can find out whether the communication protocol meets the logical correctness properties, such as deadlock freeness, unspecified reception freeness and channel overflow freeness. The PVD system utilizes the relational approach [7] to control and manage its internal data representation. This system has been used to design and validate a version of the CCITT transport class 2 protocol [3]. The validation result is to be reported.

Efficient Prolog: a practical tutorial

This tutorial presents twelve guidelines for making Prolog programs efficient. Topics covered include procedural versus declarative programming, internal representation of data, first-argument indexing, cuts, and ways to avoid consing.

Unix music tools at bellcore

A number of Bellcore projects have required software to manipulate musical data at a range of different conceptual levels from notes to whole classes of pieces. In the absence of suitable preexisting software for these projects, a set of modules was written in house to run under the Unix® operating system. These programs were designed to be tools, each performing a specific subtask with little or no preconception of the particular combinations in which they would be used. Toward that end, a standard internal data representation was chosen, and all programs were either written to process that format directly or, as more conceptual needs arose, to process a format that could be converted to the standard. This report describes our experience in the development of software to implement languages and data descriptions specialized to the manipulation of musical entities. It explains why we did not settle for existing software and gives some examples of the utility of well-designed music tools. We give an overview of the scope of the software and describe our solutions to the problems of interfacing music synthesizers with computers.

Object-instead of function-oriented data management for tool management as an example application

Today's situation in data processing in production shows substantial heterogeneity. All data processing systems implemented usually maintain their own databases. CIM strategies often turn out to be bilateral connections between systems. In this way transparency of events as well as the possiblity of reacting quickly cannot surely be enforced. The “transit times for information” are still unsatisfactory, and organization stays slow. The importance of data management as a strategic problem becomes more and more apparent. A common database can serve as an integration kernel for a CIM concept. This database should be generated and maintained independently from specific applications. The internal representation of data in the database system is of fundamental importance. On one hand it should comply with the requirements of the user programs and should be a comprehensive picture of reality; on the other hand it should be easy to survey and render possible safe data maintenance. A suitable approach for modelling data is to orient the model to the objects of the real world. This way a system for data maintenance can be created which supplies all users with comprehensive information. Fast access to information that is always up to date guarantees the ability to react quickly and is a fundamental prerequisite for the application of highly developed data processing systems. Computer-aided production will remain but a wish for many enterprises unless due attention is paid to data management.

A uniform approach toward handling atomic and structured information in the nested relational database model

The algebras and query languages for nested relations defined thus far do not allow us to “flatten” a relation scheme by disregarding the internal representation of data. In real life, however, the degree in which the structure of certain information, such as addresses, phone numbers, etc., is taken into account depends on the particular application and may even vary in time. Therefore, an algebra is proposed that does allow us to simplify relations by disregarding the internal structure of a certain class of information. This algebra is based on a careful manipulation of attribute names. Furthermore, the key operator in this algebra, called “copying,” allows us to deal with various other common queries in a very uniform manner, provided these queries are interpreted as operations on classes of semantically equivalent relations rather than individual relations. Finally, it is shown that the proposed algebra is complete in the sense of Bancilhon and Paredaens.

Transfer of large data sets from biomedical research: Proposal for a universal data header (UNIDAT 1.0)

A universal data header for computerized data transfer between different research centers has been developed and successfully tested. The structure of this data header allows for data interchange irrespective of the internal data representation of the processor used. The first 128 byte are used for description of data structure followed by dynamically allocated space for string variables, event data, and measured data. Test programs written in Turbopascal and C are available on floppy disk (formatted for MS-DOS or OS/9) from the authors upon request.

System 75:System Management

System management is the task of installing, administering, and maintaining a communications system. Customer and technician access to software-based administration and maintenance capabilities in the System 75 office communication system occurs through an on-line video display terminal called the System Access Terminal (SAT). With the SAT, the user may install, test, rearrange, and change equipment and services. The SAT hides the internal complexity of the system while presenting all the capabilities in as simple a manner as possible. A layered software architecture is used to perform data view mapping from the user view to the internal data representation.