Automatic plankton image classification—Can capsules and filters help cope with data set shift?

Abstract

AbstractThe general task of image classification seems to be solved due to the development of modern convolutional neural networks (CNNs). However, the high intraclass variability and interclass similarity of plankton images still prevents the practical identification of morphologically similar organisms. This prevails especially for rare organisms. Every CNN requires a vast amount of manually validated training images which renders it inefficient to train study‐specific classifiers. In most follow‐up studies, the plankton community is different from before and this data set shift (DSS) reduces the correct classification rates. A common solution is to discard all uncertain images and hope that the remains still resemble the true field situation. The intention of this North Sea Video Plankton Recorder (VPR) study is to assess if a combination of a Capsule Neural Network (CapsNet) with probability filters can improve the classification success in applications with DSS. Second, to provide a guideline how to customize automated CNN and CapsNet deep learning image analysis methods according to specific research objectives. In community analyses, our approach achieved a discard of uncertain predictions of only 5%. CapsNet and CNN reach similar precision scores, but the CapsNet has lower recall scores despite similar discard ratios. This is due to a higher discard ratio in rare classes. The recall advantage of the CNN decreases with increasing DSS. We present an alternative method to handle rare classes with a CNN achieving a mean recall of 96% by manually validating an average of 6.5% of the original images.