Annotating otoliths with a deep generative model

Abstract

Abstract Otoliths are a central information source for fish ecology and stock management, conveying important data about age and other life history for individual fish. Traditionally, interpretation of otoliths has required skilled expert readers, but recently deep learning classification and regression models have been trained to extract fish age from images of otoliths from a variety of species. Despite high accuracy in many cases, the adoption of such models in fisheries management has been slow. One reason may be that the underlying mechanisms the model uses to derive its results from the data are opaque, and this lack of legibility makes it challenging to build sufficient trust in the results. Here, we implement a deep learning model that instead of age predicts the location of annotation marks for each of the annuli. This allows an expert to evaluate the model’s performance in detail. The quality of the annotations was judged by a panel of four expert otolith readers in a double-blinded randomized survey. Using a scale from 1 to 5, the generated marks received an average quality score of 4.22, whereas expert annotations received an average score of 4.33. By counting the marks to determine fish age, we obtained an agreement between expert and model annotations of 64% on our test set, which running the model stochastically increased to 69%. Stochastic sampling yields further benefits, including an explicit measure of the model’s uncertainty, the post hoc likelihood of the different age classes for each otolith, and a set of alternative annotation sequences that highlight the structure of the annuli.