Hierarchical multi-label taxonomic classification of carbonate skeletal grains with deep learning

Abstract

Recent advances in Artificial Intelligence (AI), particularly the rise of deep learning, are revolutionizing data collection and analysis in many aspects of the Earth Sciences, including paleontology. Rapid digital transformation of paleontological information enables automation of various paleontological tasks using AI technology. One such task is identifying and classifying skeletal grains at different levels of the Linnaean taxonomic hierarchy, both macro- and microscopically. Unfortunately, fossil classification remains largely untouched by AI due to the expertise and time required to generate high-quality, large-scale labeled training data. This task is particularly challenging when part of the compositional analysis of limestone, where three-dimensional objects (i.e., skeletal grains) are observed in a two-dimensional cross-section. It is compounded by the imbalanced data and limited number of images available, approximately four orders smaller than the number necessary to train deep neural networks. Recent efforts to classify such fossil images using deep learning returned a single output/label classification, limiting the information output from the model, and hindering its further applications to replicate paleontologist-level identification. Here, we couple a multi-head deep convolutional neural network architecture called TaxonNet that adopts the Branch Convolutional Neural Networks and advanced image augmentation to overcome these barriers. This model uses prior knowledge of hierarchical category relationships to output multiple hierarchical taxonomic predictions from a single petrographic image. In this study, we developed and evaluated four different strategies of hierarchical classification: (i) forward (coarse to fine); (ii) reverse (fine to coarse); (iii) parallel (coarse and fine simultaneously), and (iv) hybrid. The forward and reverse strategies yield the most accurate prediction of ~95 % accuracy for coarse-level and fine-level recognitions across major fossil types in carbonate petrographic images, including algae, mollusks, and corals. These results represent an important step toward applying AI to advance fossil grain classification and exhibit a promising real-world geological application of a hierarchical network in paleontology, sedimentary petrography, and other image-based geological analysis requiring hierarchical, multilabel output.