Annotated Video Footage for Automated Identification and Counting of Fish in Unconstrained Seagrass Habitats

Ellen M Ditria,Rod M Connolly,Eric L Jinks,Sebastian Lopez-Marcano

doi:10.3389/fmars.2021.629485

Abstract

Technological advances are improving the collection, processing and analysis of ecological data. One of these technologies that has been adopted in recent studies by ecologists is computer vision (CV). CV is a rapidly developing area of machine learning that aims to infer image content at the same level humans can by extracting information from pixels (LeCun et al., 2015; Weinstein, 2018). CV in ecology has gained much attention as it can quickly and accurately process image from remote video imagery while allowing scientists to monitor both individuals and populations at unprecedented spatial and temporal scales. Automated analysis of imagery through CV has also become more accurate and streamlined with the implementation of deep learning (a subset of machine learning) models that have improved the capacity to processes raw images compared to traditional machine learning methods (LeCun et al., 2015; Villon et al., 2016). As the use of camera systems for monitoring fish abundances is common practice in conservation ecology (Gilby et al., 2017; Whitmarsh et al., 2017; Langlois et al., 2020), deep learning allows for the automated processing of big data from video or images, a step which usually creates a bottleneck when these data must be analyzed manually.

Highlights

Technological advances are improving the collection, processing and analysis of ecological data
The mAP50 metric was used to evaluate model performance in Ditria et al (2020a), which assigns a true positive when a predicted segmentation mask (Figure 1) overlaps the ground-truth annotated segmentation by at least 50%. mAP50 was calculated as follows: mean average precision value (mAP) = P (R) dR. The results using these metrics and the proposed dataset can be found in Ditria et al (2020a), where the mAP50 and F1 are both >92% for detecting the target species and counting abundance
By providing the datasets in different modalities, we propose that it can be used to understand fish dynamics in seagrass ecosystems, develop novel fish counting methods and for understanding and exploring different methods to improve accuracy for implementation in an ecological context

Summary

Introduction

Technological advances are improving the collection, processing and analysis of ecological data. Image dataset of 6 different fish species from 3 locations in Pakistan ∼80k labeled crop images (sps.) ∼45k bounding box annotations (fish/no fish) ∼4k classification images

Results

Conclusion