Finding large‐scale spatial trends in massive, global, environmental datasets

Abstract

AbstractAs technology progresses, the availability of massive environmental data with global spatial coverage has become quite common. An example of such data is total column ozone (TCO) remotely sensed from a satellite. In their raw form, these data are often spatially (and temporally) dense, but irregular. The problem considered here is one of detecting a large‐scale spatial trend at a given time point (actually, in a given time interval). We propose a sequential aggregation method, producing different levels of coarser (spatial) resolution data and, at the same time, preserving both the local information content and the locations of the raw data. In estimating the large‐scale trend, we consider different types of smooth spatial trend surfaces on the sphere, all linear combinations of smooth basis functions and satisfying the topological constraints imposed by the sphere. These include trend surfaces based on tensor products of splines, spherical harmonic basis functions, smoothing spherical splines and a new trend surface that we call fixed‐rank smoothing (FRS). The FRS trend surfaces can be based on any set of smooth basis functions on the sphere and are estimated via penalized weighted‐least‐squares regression (ridge regression) using a data‐adaptive penalty term. In comparing the various trend surfaces considered, we look at data fidelity, trend‐surface consistency when fitted to data at different resolutions, and small‐ and large‐scale spatial behavior of the resulting detrended fine‐resolution data. An application to the TCO data reveals that the large‐scale spatial trend can be detected and effectively estimated using coarse‐resolution data. The FRS trend surfaces are seen to achieve better data fidelity than other trend surfaces considered, and, in terms of trend‐surface consistency and small‐ and large‐scale residual behavior, FRS is seen to have as good, and sometimes better, performance. Copyright © 2003 John Wiley & Sons, Ltd.