Crowd, Crops and Machines: How crowdsourced annotations can help towards crops classification

Abstract

<p>The use of artificial intelligence and specifically deep learning (DL) approaches in the domain of remote sensing is increasing. Such methods provide excellent results and show great potential for future applications. Earth observation sensors are able to deliver data with higher spatial, spectral and temporal resolutions. In this project, we use Sentinel-2 multispectral data and couple this input with a crowd annotated very high resolution (VHR) map which is generated in the video-game Cerberus, developed by the company BlackShore. In Cerberus, players are able to map features, like buildings, forest and specific types of crop fields, that are subsequently used as input for the Machine Learning (ML) pipeline. The ML pipeline is applied to classify crop fields in a larger region.</p><p>The main objective of this research is to study the accuracy of a model in detecting and describing the type of crop and whether the addition of a temporal dimension increases the accuracy. We will be experimenting with different methods that take their root in DL. The study region shown to Cerberus-players is Oromia in Ethiopia, south of the capital Addis Ababa. Using Sentinel-2 data, we aim to extend the generated maps to cover Ethiopia.</p><p>First, we will implement two DL methods; Random Forest (RF), and a 3D Convolutional Neural Network (CNN) that do not make use of the temporal dimension in order to have a baseline of the expected accuracy from a single multi-spectral image. Next, we will investigate four models that make use of time series: 1) a hybrid convolutional neural network-random forest (CNN-RF); 2) a 3D CNN that takes as input the output of a stack of 3D CNNs; 3) a model based on Recurrent Neural Networks (RNNs) performing pixel-based classification; and 4) an innovative method that combines the strength of RNNs, CNNs and Generative Adversarial Networks.<span> </span></p><p>We are now implementing the methods and shall report on results at EGU April 2021. For future research, it could be a very interesting case to study the possibility of generalizing the combined approach of crowd annotated training data with extended classification over larger regions and generalizing to other areas.</p>