A Language for Spatial Data Manipulation

Abstract

Environmental Observation and Forecasting Systems (EOFS) create new opportunities and challenges for generation and use of environmental data products. The number and diversity of these data products, however, has been artificially constrained by the lack of a simple descriptive language for expressing them. Data products that can be described simply in English take pages of obtuse scripts to generate. The scripts obfuscate the original intent of the data product, making it difficult for users and scientists to understand the overall product catalog. The problem is exacerbated by the evolution of modern EOFS into data product “factories” subject to reliability requirements and daily production schedules. New products must be developed and assimilated into the product suite as quickly as they are imagined. Reliability must be maintained despite changing hardware, changing software, changing file formats, and changing staff. We present a language for naturally expressing data product recipes over structured and unstructured computational grids of arbitrary dimension. Informed by relational database theory, we have defined a simple data model and a handful of operators that can be composed to express complex visualizations, plots, and transformations of gridded datasets. The language provides a medium for design, discussion, and development of new data products without commitment to particular data structures or algorithms. In this paper, we provide a formal description of the language and several examples of its use to express and analyze data products. The context of our research is the CORIE system, an EOFS supporting the study of the Columbia River Estuary.