Crowdsourced generation of annotated video datasets: A zebrafish larvae dataset for video segmentation and tracking evaluation

Abstract

Video segmentation research has emerged over the last decade for biomedical image and video processing, especially in biological organism tracking. However, due to the difficulties in generating the video segmentation ground truth, the general lack of segmentation datasets with annotated ground-truth severely limits the evaluation of segmentation algorithms. This paper proposes an efficient and scalable crowdsourced approach to generate video segmentation ground-truth to facilitate database generation for general biological organism segmentation and tracking algorithm evaluation. To illustrate the proposed approach, an annotated zebrafish larvae video segmentation dataset has been generated and made freely available online. To enable the evaluation of algorithms against a ground-truth, a set of segmentation evaluation metrics are also presented. The segmentation performance of five leading segmentation algorithms is then evaluated by the metrics on the generated zebrafish video segmentation dataset.