Spatio-temporal contrastive learning for acoustic data

Abstract

Large volumes of water column sonar data have been generated from acoustic surveys of living marine resources. They provide valuable information about marine ecosystems. To leverage them for acoustic target identification, scarcity of annotations is usually an issue. While some annotations have been generated by scientists via manual scrutiny or limited automation, they are limited to certain species. To fill in this gap, we propose a spatio-temporal contrastive learning approach for acoustic data. This unsupervised deep learning technique leverages both acoustic and non-acoustic (spatial and temporal) information for acoustic target classification. We firstly employ the Simple Linear Iterative Clustering (SLIC) algorithm to extract superpixels from acoustic data. Then the spatial and temporal similarity between two superpixels is computed, and the proposed spatio-temporal contrastive learning approach is applied to learn a semantically meaningful representation for superpixels. Finally, the learned representations of superpixels are used in acoustic target classification. We demonstrate that this approach outperforms previous methods with acoustic data collected by the NOAA National Marine Fisheries Service in the Gulf of Maine and Georges Bank region over 10 years. This dataset is archived at NOAA National Centers for Environmental Information and accessed for free through AWS and NOAA Big Data Program.