Using Preaggregation to Speed Up Scaling Operations on Massive Spatio-temporal Data

Abstract

AbstractA frequent operation in e-Science is downscaling of some data item or part thereof, such as obtaining a 1 GB overview from a 10 TB dataset. Scaling is expensive as it normally requires a full scan of the area. Speeding up such operations, therefore, is performance critical.A common optimization technique used for map imagery is to materialize selected downscaled versions. However, there is no support for 3D, such as x/y/t timeseries or x/y/z geophysics data. To overcome this, we propose a preaggregation technique for multi-dimensional gridded (”raster”) data. Preaggregates are selected based on a given query workload while considering disk space constraints. Upon evaluation, queries use the next best preaggregate and perform the remaining scaling.We present the preaggregate selection algorithm and argue its efficiency based on a performance analysis covering 2-D and 3-D use cases. Further, we show how our approach outperforms the well-known 2-D image pyramids widely used in Web mapping.KeywordsGeographic Information SystemStorage SpaceImage PyramidScale VectorQuery WorkloadThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.