Digital taxonomist: Identifying plant species in community scientists’ photographs

Riccardo De Lutio,Yihang She,Stefano D’Aronco,Stefania Russo,Philipp Brun,Jan D Wegner,Konrad Schindler

doi:10.1016/j.isprsjprs.2021.10.002

Riccardo De Lutio, Yihang She + Show 5 more

Open Access

https://doi.org/10.1016/j.isprsjprs.2021.10.002

Copy DOI

Abstract

Automatic identification of plant specimens from amateur photographs could improve species range maps, thus supporting ecosystems research as well as conservation efforts. However, classifying plant specimens based on image data alone is challenging: some species exhibit large variations in visual appearance, while at the same time different species are often visually similar; additionally, species observations follow a highly imbalanced, long-tailed distribution due to differences in abundance as well as observer biases. On the other hand, most species observations are accompanied by side information about the spatial, temporal and ecological context. Moreover, biological species are not an unordered list of classes but embedded in a hierarchical taxonomic structure. We propose a multimodal deep learning model that takes into account these additional cues in a unified framework. Our Digital Taxonomist is able to identify plant species in photographs better than a classifier trained on the image content alone, the performance gained is over 6 percent points in terms of accuracy.

Highlights

Biodiversity describes the diversity of life in terms of species’ numbers, similarity, abundance, and distribution across spatial scales (Barrotta and Gronda, 2020; Gaston and Spicer, 2004)
We focus on the case of recognising plant species in data collected via community science applications such as iNaturalist or Info Flora (Info Flora, 2021)
Improvements from location context and hierarchical labels are largely orthogonal, as ex pected, since they leverage different types of information. These results indicate a clear benefit of complementing visual cues from community science images with additional sources of information

Summary

Introduction

Biodiversity describes the diversity of life in terms of species’ numbers, similarity, abundance, and distribution across spatial scales (Barrotta and Gronda, 2020; Gaston and Spicer, 2004). Its spatio-temporal distribution needs to be well understood, which requires efficient monitoring schemes. One viable way to complement professional biodiversity monitoring is the community science approach. The community science paradigm aims at involving the general public in scientific observations and in vestigations, and is useful in cases where the experiment is characterized by a large spatial and/or temporal scale (Silvertown, 2009). The community science approach has a long history in biodiversity monitoring (Dickinson et al, 2010). Volun teers have participated in the annual Christmas Bird Counts of the Na tional Audubon Society in the USA since 1900 (Butcher and Niven, 2007)

Methods

Results

Conclusion