Big Earth Observation Research Articles

From Geospatial Data Cube to AI Cube: the Open Geospatial Engine (OGE) Approach

Abstract. The Earth Observation (EO) analytics are moving from local systems to online cloud computing platforms such as Google Earth Engine (GEE) and Open Geospatial Engine (OGE). A typical approach in existing efforts is to leverage geospatial data cubes with cloud computing to support large-scale big EO data analytics in Digital Earth systems. While online analytical processing (OLAP) can be enabled using the cube approach, it is still not clear how geospatial artificial intelligence (GeoAI) can be incorporated in data cubes to benefit the cube infrastructure. Such an investigation can consolidate the vision of an AI-ready SDI (Spatial Data Infrastructure). The paper presents a systematic approach to incorporate GeoAI models into geospatial data cubes to help create an AI Cube. It covers on-demand model retrieval, cube data and model integration, and distributed model inference. The approach is demonstrated in OGE, which is an EO cloud computing platform layered on the GeoCube implementation. The results show that such an AI Cube enriches a cube infrastructure with GeoAI capabilities, facilitates the on-demand coupling of cube data and GeoAI models, and improves the performance of GeoAI inference.

Time-first approach for land cover mapping using big Earth observation data time-series in a data cube – a case study from the Lake Geneva region (Switzerland)

ABSTRACT Accurate, consistent, and high-resolution Land Use & Cover (LUC) information is fundamental for effectively monitoring landscape dynamics and better apprehending drivers, pressures, state, and impacts on land systems. Nevertheless, the availability of such national products with high thematic accuracy is still limited and consequently researchers and policymakers are constrained to work with data that do not necessarily reflect on-the-ground realities impending to correctly capture details of landscape features as well as limiting the identification and quantification of drivers and rate of change. Hereafter, we took advantage of the Switzerland’s official LUC statistical sampling survey and dense time-series of Sentinel-2 data, combining them with Machine and Deep Learning methods to produce an accurate and high spatial resolution land cover map over the Lake Geneva region. Findings suggest that time-first approach is a valuable alternative to space-first approaches, accounting for the intra-annual variability of classes, hence improving classification performances. Results demonstrate that Deep Learning methods outperform more traditional Machine Learning ones such as Random Forest, providing more accurate predictions with lower uncertainty. The produced land cover map has a high accuracy, an improved spatial resolution, while at the same time preserving the statistical significance (i.e. class proportion) of the official national dataset. This work paves the way towards the objective to produce a yearly high resolution land cover map of Switzerland and potentially implement a continuous land change monitoring capability. However further work is required to properly address challenges such as the need for increased temporal resolution for LUC information or the quality of training samples.

Chinese sociotechnical imaginaries of Earth observation: From sight to foresight

Although Earth observation (EO) is considered a universal scientific technique with a hegemonic gaze, national sociotechnical imaginaries shape its practice. Historically established and organized by national governments, EO, which is commonly pursued via satellite remote sensing, depends on domestic perceptions of the technique's scope and applications. EO is also a political technology with the capacity to mediate and potentially regulate life on Earth at a range of scales. In China, the state and scientists endeavor to transform EO into a data-driven, inductive, and predictive method with a focus on social coordination, especially in cities—a key scale for scientific study and the organization and reproduction of state power. Contributing to China's reimagining of EO as a governance and governmentality tool are advances in social sensing, which leverages big data continuously generated by individuals’ devices. By analyzing big EO and social data, rather than only observe past and present changes to the Earth's surface, the Chinese state and scientists seek to predict future social events. While these changes are narrated as a popularization of remote sensing enabling more responsive, human-centric governance, they may augur the rise of antipolitical technologies that attempt to prevent dissent by determining and even guiding future behavior. As China, a satellite power, aims to export its services for observing and predicting Earth and upscale them through global governance, critique of their associated sociotechnical imaginaries is crucial. Such research reveals the heterogeneity of the satellite gaze and may identify where and how people hold power within EO systems.

Data Compression-Aware Performance Analysis of Dask and Spark for Earth Observation Data Processing

High-performance computing is a good choice for handling Big Earth Observation data, allowing the processing of the data in a distributed and performance-efficient way using in-memory computing frameworks. The data compression technique reduces the amount of storage and network transfer time and improves processing performance. The article aims to investigate the effectiveness of widely used distributed data processing frameworks in conjunction with lossless data compression techniques, to find the optimal compression method and processing framework for specific earth observation workflows. Normalized Difference Vegetation Index has been evaluated for the territory of Armenia, obtaining data from the Sentinel satellite and considering the supported compression methods to compare the performance of in-memory Dask and Spark frameworks. Experiments show that the Zstandard compression method and the Dask framework are the best choices for such workflows.

Scalable Data Processing Platform for Earth Observation Data Repositories

Earth observation (EO) satellite data is essential to environmental monitoring. At a national and regional level, the open data cubes harness the power of satellite data by providing application programming interfaces and services to the end-users. The volume and the complexity of satellite observations are increasing, demanding novel approaches for data storing, managing, and processing. High-performance computing (HPC) and cloud platforms may improve Big EO data processing performance. However, it is necessary to consider several vital aspects for efficient and flexible EO data processing, such as the interoperability from cloud-HPC and EO data repositories, automatic provisioning and scaling of cloud-HPC resources, cost-effectiveness, support of new EO data formats and open-source packages, or linkage with data cube platforms. The article proposes a scalable EO data processing platform interoperable from cloud-HPC and EO data repositories. The platform enables linking any data repository supporting web coverage service or SpatioTemporal Asset Catalog Application Programming Interfaces (STAC-API), and any cloud or HPC resource supporting scheduling system API for providing access to the cluster backends.

Insights into the vulnerability of vegetation to tephra fallouts from interpretable machine learning and big Earth observation data

Abstract. Although the generally high fertility of volcanic soils is often seen as an opportunity, short-term consequences of eruptions on natural and cultivated vegetation are likely to be negative. The empirical knowledge obtained from post-event impact assessments provides crucial insights into the range of parameters controlling impact and recovery of vegetation, but their limited coverage in time and space offers a limited sample of all possible eruptive and environmental conditions. Consequently, vegetation vulnerability remains largely unconstrained, thus impeding quantitative risk analyses. Here, we explore how cloud-based big Earth observation data, remote sensing and interpretable machine learning (ML) can provide a large-scale alternative to identify the nature of, and infer relationships between, drivers controlling vegetation impact and recovery. We present a methodology developed using Google Earth Engine to systematically revisit the impact of past eruptions and constrain critical hazard and vulnerability parameters. Its application to the impact associated with the tephra fallout from the 2011 eruption of Cordón Caulle volcano (Chile) reveals its ability to capture different impact states as a function of hazard and environmental parameters and highlights feedbacks and thresholds controlling impact and recovery of both natural and cultivated vegetation. We therefore conclude that big Earth observation (EO) data and machine learning complement existing impact datasets and open the way to a new type of dynamic and large-scale vulnerability models.

Think global, cube local: an Earth Observation Data Cube’s contribution to the Digital Earth vision

ABSTRACT The technological landscape for managing big Earth observation (EO) data ranges from global solutions on large cloud infrastructures with web-based access to self-hosted implementations. EO data cubes are a leading technology for facilitating big EO data analysis and can be deployed on different spatial scales: local, national, regional, or global. Several EO data cubes with a geographic focus (“local EO data cubes”) have been implemented. However, their alignment with the Digital Earth (DE) vision and the benefits and trade-offs in creating and maintaining them ought to be further examined. We investigate local EO data cubes from five perspectives (science, business and industry, government and policy, education, communities and citizens) and illustrate four examples covering three continents at different geographic scales (Swiss Data Cube, semantic EO data cube for Austria, DE Africa, Virginia Data Cube). A local EO data cube can benefit many stakeholders and players but requires several technical developments. These developments include enabling local EO data cubes based on public, global, and cloud-native EO data streaming and interoperability between local EO data cubes. We argue that blurring the dichotomy between global and local aligns with the DE vision to access the world’s knowledge and explore information about the planet.

A multi-source spatio-temporal data cube for large-scale geospatial analysis

Data management and analysis are challenging with big Earth observation (EO) data. Expanding upon the rising promises of data cubes for analysis-ready big EO data, we propose a new geospatial infrastructure layered over a data cube to facilitate big EO data management and analysis. Compared to previous work on data cubes, the proposed infrastructure, GeoCube, extends the capacity of data cubes to multi-source big vector and raster data. GeoCube is developed in terms of three major efforts: formalize cube dimensions for multi-source geospatial data, process geospatial data query along these dimensions, and organize cube data for high-performance geoprocessing. This strategy improves EO data cube management and keeps connections with the business intelligence cube, which provides supplementary information for EO data cube processing. The paper highlights the major efforts and key research contributions to online analytical processing for dimension formalization, distributed cube objects for tiles, and artificial intelligence enabled prediction of computational intensity for data cube processing. Case studies with data from Landsat, Gaofen, and OpenStreetMap demonstrate the capabilities and applicability of the proposed infrastructure.

Multidimensional architecture using a massive and heterogeneous data: Application to drought monitoring

The rapid increase in the number of Earth Observation (EO) systems generates a massive amount of heterogeneous data. It has raised big issues in collecting, preprocessing, storing, and the visualization these data. However, traditional techniques are facing serious challenges when dealing with big EO data dimensions (i.e., Volume, Veracity, Variety, and Velocity), especially in natural hazards management. Therefore, big data techniques and tools attract more attention. In this paper we propose a multidimensional model framework for Big EO data warehousing. This framework includes 3 parts: (1) Data collection and preprocessing, being responsible for collecting data and improving their quality; (2) Data loading and storage, performing the ingestion task which consists of transferring the data from external resources to the Big data platform for storage; and (3) Visualization and interpretation, aiming to provide spatio-temporal analysis. This framework could be useful for decision-makers in monitoring the effects of drought disasters and, consequently, planning the mitigation and remediation measures. Experiments are carried out on drought monitoring in China along the period 2000–2020. The input data include remote sensing data, biophysical data, and climatological data. The results reveal that the proposed framework has a higher retrieval speed and a greater elasticity with different kinds (i.e. spatial, temporal, or spatiotemporal) of requests compared to traditional frameworks, indicating its superiority.

Can linguistic features extracted from geo-referenced tweets help building function classification in remote sensing?

The fusion of two or more different data sources is a widely accepted technique in remote sensing while becoming increasingly important due to the availability of big Earth Observation satellite data. As a complementary source of geo-information to satellite data, massive text messages from social media form a temporally quasi-seamless, spatially multi-perspective stream, but with unknown and diverse quality. Despite the uncontrolled quality: can linguistic features extracted from geo-referenced tweets support remote sensing tasks? This work presents a straightforward decision fusion framework for very high-resolution remote sensing images and Twitter text messages. We apply our proposed fusion framework to a land-use classification task – the building function classification task – in which we classify building functions like commercial or residential based on linguistic features derived from tweets and remote sensing images. Using building tags from OpenStreetMap (OSM), we labeled tweets and very high-resolution (VHR) images from Google Maps. We collected English tweets from San Francisco, New York City, Los Angeles, and Washington D.C. and trained a stacked bi-directional LSTM neural network with these tweets. For the aerial images, we predicted building functions with state-of-the-art Convolutional Neural Network (CNN) architectures fine-tuned from ImageNet on the given task. After predicting each modality separately, we combined the prediction probabilities of both models building-wise at a decision level. We show that the proposed fusion framework can improve the classification results of the building type classification task. To the best of our knowledge, we are the first to use semantic contents of Twitter messages and fusing them with remote sensing images to classify building functions at a single building level.

Monitoring Complex Integrated Crop–Livestock Systems at Regional Scale in Brazil: A Big Earth Observation Data Approach

Due to different combinations of agriculture, livestock and forestry managed by rotation, succession and intercropping practices, integrated agriculture production systems such as integrated crop–livestock systems (iCL) constitute a very complex target and a challenge for automatic mapping of cropping practices based on remote sensing data. The overall objective of this study was to develop a classification strategy for the annual mapping of integrated Crop–Livestock systems (iCL) at a regional scale. This strategy was designed and tested in the six agro-climatic regions of Mato Grosso, the largest Brazilian soybean producer state, using MODIS satellite time-series images acquired between 2012 and 2019, ground data with heterogeneous distribution in space and time and a Random Forest classifier. The results showed that: 1. the use of unbalanced training samples with a class composition close to the real one was the right classifier training strategy; 2. the use of a single training database (pooling samples from different years and regions) to classify each region and year individually proved to be robust enough to provide similar classification accuracies in comparison to those based on the use of a database acquired for each region and for each year. The final hierarchical classification overall accuracy was 0.89 for Level 1, the cropping pattern level (single and double crops DC); 0.84 for Level 2, the DC category level (integrated system iCL soy-pasture/brachiaria, soy-cotton and soy-cereal); 0.77 for Level 3, the iCL level (iCL1 soy-pasture and iCL2 soy-pasture mixed with corn). The F-scores for DC, iCL and iCL1 cropping systems presented high accuracy (0.89, 0.85 and 0.84), while iCL2 was more difficult to classify (0.63). This approach will next be applied across the entire Brazilian soybean corridor, leading to an operational tool for monitoring the adoption of sustainable intensification practices recognized by Brazil’s Agriculture Low Carbon Plan (ABC PLAN).

The Austrian Semantic EO Data Cube Infrastructure

Big optical Earth observation (EO) data analytics usually start from numerical, sub-symbolic reflectance values that lack inherent semantic information (meaning) and require interpretation. However, interpretation is an ill-posed problem that is difficult for many users to solve. Our semantic EO data cube architecture aims to implement computer vision in EO data cubes as an explainable artificial intelligence approach. Automatic semantic enrichment provides semi-symbolic spectral categories for all observations as an initial interpretation of color information. Users graphically create knowledge-based semantic models in a convergence-of-evidence approach, where color information is modelled a-priori as one property of semantic concepts, such as land cover entities. This differs from other approaches that do not use a-priori knowledge and assume a direct 1:1 relationship between reflectance values and land cover. The semantic models are explainable, transferable, reusable, and users can share them in a knowledgebase. We provide insights into our web-based architecture, called Sen2Cube.at, including semantic enrichment, data models, knowledge engineering, semantic querying, and the graphical user interface. Our implemented prototype uses all Sentinel-2 MSI images covering Austria; however, the approach is transferable to other geographical regions and sensors. We demonstrate that explainable, knowledge-based big EO data analysis is possible via graphical semantic querying in EO data cubes.

Assembly of a Coreset of Earth Observation Images on a Small Quantum Computer

Satellite instruments monitor the Earth’s surface day and night, and, as a result, the size of Earth observation (EO) data is dramatically increasing. Machine Learning (ML) techniques are employed routinely to analyze and process these big EO data, and one well-known ML technique is a Support Vector Machine (SVM). An SVM poses a quadratic programming problem, and quantum computers including quantum annealers (QA) as well as gate-based quantum computers promise to solve an SVM more efficiently than a conventional computer; training the SVM by employing a quantum computer/conventional computer represents a quantum SVM (qSVM)/classical SVM (cSVM) application. However, quantum computers cannot tackle many practical EO problems by using a qSVM due to their very low number of input qubits. Hence, we assembled a coreset (“core of a dataset”) of given EO data for training a weighted SVM on a small quantum computer, a D-Wave quantum annealer with around 5000 input quantum bits. The coreset is a small, representative weighted subset of an original dataset, and its performance can be analyzed by using the proposed weighted SVM on a small quantum computer in contrast to the original dataset. As practical data, we use synthetic data, Iris data, a Hyperspectral Image (HSI) of Indian Pine, and a Polarimetric Synthetic Aperture Radar (PolSAR) image of San Francisco. We measured the closeness between an original dataset and its coreset by employing a Kullback–Leibler (KL) divergence test, and, in addition, we trained a weighted SVM on our coreset data by using both a D-Wave quantum annealer (D-Wave QA) and a conventional computer. Our findings show that the coreset approximates the original dataset with very small KL divergence (smaller is better), and the weighted qSVM even outperforms the weighted cSVM on the coresets for a few instances of our experiments. As a side result (or a by-product result), we also present our KL divergence findings for demonstrating the closeness between our original data (i.e., our synthetic data, Iris data, hyperspectral image, and PolSAR image) and the assembled coreset.

Yield forecasting with machine learning and small data: What gains for grains?

Forecasting crop yields is important for food security, in particular to predict where crop production is likely to drop. Climate records and remotely-sensed data have become instrumental sources of data for crop yield forecasting systems. Similarly, machine learning methods are increasingly used to process big Earth observation data. However, access to data necessary to train such algorithms is often limited in food-insecure countries. Here, we evaluate the performance of machine learning algorithms and small data to forecast yield on a monthly basis between the start and the end of the growing season. To do so, we developed a robust and automated machine-learning pipeline which selects the best features and model for prediction. Taking Algeria as case study, we predicted national yields for barley, soft wheat and durum wheat with an accuracy of 0.16–0.2 t/ha (13-14% of mean yield) within the season. The best machine-learning models always outperformed simple benchmark models. This was confirmed in low-yielding years, which is particularly relevant for early warning. Nonetheless, the differences in accuracy between machine learning and benchmark models were not always of practical significance. Besides, the benchmark models outperformed up to 60% of the machine learning models that were tested, which stresses the importance of proper model calibration and selection. For crop yield forecasting, like for many application domains, machine learning has delivered significant improvement in predictive power. Nonetheless, superiority over simple benchmarks is fully achieved after extensive calibration, especially when dealing with small data.

Satellite Image Time Series Analysis for Big Earth Observation Data

The development of analytical software for big Earth observation data faces several challenges. Designers need to balance between conflicting factors. Solutions that are efficient for specific hardware architectures can not be used in other environments. Packages that work on generic hardware and open standards will not have the same performance as dedicated solutions. Software that assumes that its users are computer programmers are flexible but may be difficult to learn for a wide audience. This paper describes sits, an open-source R package for satellite image time series analysis using machine learning. To allow experts to use satellite imagery to the fullest extent, sits adopts a time-first, space-later approach. It supports the complete cycle of data analysis for land classification. Its API provides a simple but powerful set of functions. The software works in different cloud computing environments. Satellite image time series are input to machine learning classifiers, and the results are post-processed using spatial smoothing. Since machine learning methods need accurate training data, sits includes methods for quality assessment of training samples. The software also provides methods for validation and accuracy measurement. The package thus comprises a production environment for big EO data analysis. We show that this approach produces high accuracy for land use and land cover maps through a case study in the Cerrado biome, one of the world’s fast moving agricultural frontiers for the year 2018.

An AI-Enabled Framework for Real-Time Generation of News Articles Based on Big EO Data for Disaster Reporting

In the field of journalism, the collection and processing of information from different heterogeneous sources are difficult and time-consuming processes. In the context of the theory of journalism 3.0, where multimedia data can be extracted from different sources on the web, the possibility of creating a tool for the exploitation of Earth observation (EO) data, especially images by professionals belonging to the field of journalism, is explored. With the production of massive volumes of EO image data, the problem of their exploitation and dissemination to the public, specifically, by professionals in the media industry, arises. In particular, the exploitation of satellite image data from existing tools is difficult for professionals who are not familiar with image processing. In this scope, this article presents a new innovative platform that automates some of the journalistic practices. This platform includes several mechanisms allowing users to early detect and receive information about breaking news in real-time, retrieve EO Sentinel-2 images upon request for a certain event, and automatically generate a personalized article according to the writing style of the author. Through this platform, the journalists or editors can also make any modifications to the generated article before publishing. This platform is an added-value tool not only for journalists and the media industry but also for freelancers and article writers who use information extracted from EO data in their articles.

Cloud Detection From Remote Sensing Imagery Based on Domain Translation Network

Cloud detection in optical imagery has drawn remarkable attention in the era of big Earth observation data analytic. While multiple supervised learning models have been developed for such purpose, large volumes of paired training samples annotated at the pixel level are essential to ensure the model’s generalization capacity. However, constructing a comprehensive cloud detection training database is a tedious and time-consuming process. To tackle this dilemma, we simply regard cloud-contaminated remote sensing (RS) imagery as the combination of cloud and background domains and propose a cloud detection framework based on image-to-image domain translation network (DTNet) to separate cloud-contaminated RS imagery into two target domains of cloud and background object images without using any paired and pixel-level annotation training data. The framework was evaluated with multispectral images from two types of sensors, Landsat-8 Operational Land Imager (OLI) (30 m) and GaoFen-1 (16 m), and demonstrated superior or comparable performance compared with several state-of-the-art cloud detection models.

One GUI to Rule Them All: Accessing Multiple Semantic EO Data Cubes in One Graphical User Interface

Spatio-temporal analysis capabilities of big Earth observation (EO) data are possible now on various infrastructures, but the transferability and interoperability of analyses remain challenging. This contribution describes an approach for interacting with multiple semantic EO data cubes, where for each observation, at least one nominal (i.e., categorical) interpretation is available and can be queried in the same instance. Our in-house developed Web-based graphical user interface (GUI) provides technical access to multiple semantic EO data cubes, regardless of what infrastructure they are implemented on. It is designed to create semantic models using a graphical language, and an inference engine is able to evaluate these models against existing semantic EO data cubes based on a user’s defined area and timespan of interest. Querying on a semantic level allows the transferability of semantic models across EO data cubes. Our contribution shows an approach towards solving this open research gap and discusses relevant challenges such as transferability of semantic models, on-demand instantiation, and federated EO data cubes. We believe that this approach offers new opportunities for improved semantic and syntactic interoperability in EO analyses and is better positioned to allow semantically-enabled queries possible in a federated EO data cube context.

Multi-Temporal Land Cover Change Mapping Using Google Earth Engine and Ensemble Learning Methods

The study deals with the application of Google Earth Engine (GEE), Landsat data and ensemble-learning methods (ELMs) to map land cover (LC) change over a decade in the Kaski district of Nepal. As Nepal has experienced extensive changes due to natural and anthropogenic activities, monitoring such changes are crucial for understanding relationships and interactions between social and natural phenomena and to promote better decision-making. The main novelty lies in applying the XGBoost classifier for LC mapping over Nepal and monitoring the decadal changes of LC using ELMs. To map the LC change, a yearly cloud-free composite Landsat image was selected for the year 2010 and 2020. Combining the annual normalized difference vegetation index, normalized difference built-up index and modified normalized difference water index, with elevation and slope data from shuttle radar topography mission, supervised classification was performed using a random forest and extreme gradient boosting ELMs. Post classification change detection, validation and accuracy assessment were executed after the preparation of the LC maps. Three evaluation indices, namely overall accuracy (OA), Kappa coefficient, and F1 score from confusion matrix reports, were calculated for all the points used for validation purposes. We have obtained an OA of 0.8792 and 0.875 for RF and 0.8926 and 0.8603 for XGBoost at the 95% confidence level for 2010 and 2020 LC maps, which are better for mountainous terrain. The applied methodology could be significant in utilizing the big earth observation data and overcoming the traditional computational challenges using GEE. In addition, the quantification of changes over time would be helpful for decision-makers to understand current environmental dynamics in the study area.

On the semantics of big Earth observation data for land classification

This paper discusses the challenges of using big Earth observation data for land classification. The approach taken is to consider pure data-driven methods to be insufficient to represent continuous change. We argue for sound theories when working with big data. After revising existing classification schemes such as FAO's Land Cover Classification System (LCCS), we conclude that LCCS and similar proposals cannot capture the complexity of landscape dynamics. We then investigate concepts that are being used for analyzing satellite image time series; we show these concepts to be instances of events. Therefore, for continuous monitoring of land change, event recognition needs to replace object identification as the prevailing paradigm. The paper concludes by showing how event semantics can improve data-driven methods to fulfil the potential of big data.