CoralSeg: Learning coral segmentation from sparse annotations

Tali Treibitz,Iñigo Alonso,Gal Eyal,Ana C Murillo,Matan Yuval

doi:10.1002/rob.21915

Abstract

AbstractRobotic advances and developments in sensors and acquisition systems facilitate the collection of survey data in remote and challenging scenarios. Semantic segmentation, which attempts to provide per‐pixel semantic labels, is an essential task when processing such data. Recent advances in deep learning approaches have boosted this task's performance. Unfortunately, these methods need large amounts of labeled data, which is usually a challenge in many domains. In many environmental monitoring instances, such as the coral reef example studied here, data labeling demands expert knowledge and is costly. Therefore, many data sets often present scarce and sparse image annotations or remain untouched in image libraries. This study proposes and validates an effective approach for learning semantic segmentation models from sparsely labeled data. Based on augmenting sparse annotations with the proposed adaptive superpixel segmentation propagation, we obtain similar results as if training with dense annotations, significantly reducing the labeling effort. We perform an in‐depth analysis of our labeling augmentation method as well as of different neural network architectures and loss functions for semantic segmentation. We demonstrate the effectiveness of our approach on publicly available data sets of different real domains, with the emphasis on underwater scenarios—specifically, coral reef semantic segmentation. We release new labeled data as well as an encoder trained on half a million coral reef images, which is shown to facilitate the generalization to new coral scenarios.

Highlights

Advances in robotics have facilitated the acquisition of data in challenging environments, such as underwater [Bryant et al, 2017, Gonzalez-Rivero et al, 2014] and aerial [Koh and Wich, 2012] surveys
It enables the application of recent developments in deep learning for semantic segmentation in a wider range of domains including coral segmentation demonstrated here
Depthrelated zonation, a predominant characteristic of coral reefs [Huston, 1985], adds to the challenge. Such dissimilarities must be taken into consideration in benthic image analysis, and highlight the need for an adaptive identification tool that is robust to different underwater scenes and can be utilized across an assortment of datasets

Summary

Introduction

Advances in robotics have facilitated the acquisition of data in challenging environments, such as underwater [Bryant et al, 2017, Gonzalez-Rivero et al, 2014] and aerial [Koh and Wich, 2012] surveys. Many robotic applications benefited from these improvements, e.g., autonomous driving [Luc et al, 2017] and object detection and manipulation [Wong et al, 2017] These methods require extensive amounts of pixel-level labeled data. Scleractinian corals are known to display morphological plasticity, i.e., intra-specific variations in the shape and form of colonial units [Todd, 2008] These variations represent the feedback between the organism’s developmental plan and the surrounding ecological context and settings [Schlichting et al, 1998]. Depthrelated zonation, a predominant characteristic of coral reefs [Huston, 1985], adds to the challenge Such dissimilarities must be taken into consideration in benthic image analysis, and highlight the need for an adaptive identification tool that is robust to different underwater scenes and can be utilized across an assortment of datasets

Objectives

Methods

Results