Software-dependent biases in the recognition of di- and tri-syllabic bird songs can create false interpretations of bird abundance and singing activity

Abstract

The use of autonomous recording units (ARUs) for passive acoustic monitoring has recently gained a lot of importance. It is now widely used for scientific purposes and has been increasingly applied in the field of conservation biology. However, while ARUs may greatly increase cost-effectiveness in monitoring, they are not exempt from possible biases. In this study, we compared the performances of three different software in recognizing bi- and tri-syllabic bird songs and tested if their results varied significantly in a real setting. We focused on the interpretation of the relative abundance among habitats and the time of vocal activity, also using different numbers of training records to assess if the results of the software improved. Even with the quality measures of the recognizer being consistent with those of previously published studies, in several cases the results produced significantly different interpretations depending on the software used. While increasing the number of training records slightly improved the overlap in the estimates of activity among the different software, differences in the relative abundance of each song were present even with a fairly solid number of training records. Our results suggest that reliable biological interpretations might only be attained with a large number of training records and very high values of precision, recall, and the F-score. In fact, even with very high precision and F-scores, lower values for recall led not only to statistically significant differences, but also to different interpretations of relative abundance and time of activity between the two best-performing software. Thus, great caution must be taken in interpreting the results of bioacoustics studies that use automated recognition, and standards of minimum quality should be created for recognizers in scientific and technical studies.