Validation of the F-POD—A fully automated cetacean monitoring system

Abstract

The F-POD, an echolocation-click logging device, is commonly used for passive acoustic monitoring of cetaceans. This paper presents the first assessment of the error-rate of fully automated analysis by this system, a description of the F-POD hardware, and a description of the KERNO-F v1.0 classifier which identifies click trains. Since 2020, twenty F-POD loggers have been used in the BlackCeTrends project by research teams from Bulgaria, Georgia, Romania, Türkiye, and Ukraine with the aim of investigating trends of relative abundance in populations of cetaceans of the Black Sea. Acoustic data from this project analysed here comprises 9 billion raw data clicks in total, of which 297 million were classified by KERNO-F as Narrow Band High Frequency (NBHF) clicks (harbour porpoise clicks) and 91 million as dolphin clicks. Such data volumes require a reliable automated system of analysis, which we describe. A total of 16,805 Detection Positive Minutes (DPM) were individually inspected and assessed by a visual check of click train characteristics in each DPM. To assess the overall error rate in each species group we investigated 2,000 DPM classified as having NBHF clicks and 2,000 DPM classified as having dolphin clicks. The fraction of NBHF DPM containing misclassified NBHF trains was less than 0.1% and for dolphins the corresponding error-rate was 0.97%. For both species groups (harbour porpoises and dolphins), these error-rates are acceptable for further study of cetaceans in the Black Sea using the automated classification without further editing of the data. The main sources of errors were 0.17% of boat sonar DPMs misclassified as harbour porpoises, and 0.14% of harbour porpoise DPMs misclassified as dolphins. The potential to estimate the rate at which these sources generate errors makes possible a new predictive approach to overall error estimation.