Plankton Recognition Research Articles

Survey of automatic plankton image recognition: challenges, existing solutions and future perspectives

Planktonic organisms including phyto-, zoo-, and mixoplankton are key components of aquatic ecosystems and respond quickly to changes in the environment, therefore their monitoring is vital to follow and understand these changes. Advances in imaging technology have enabled novel possibilities to study plankton populations, but the manual classification of images is time consuming and expert-based, making such an approach unsuitable for large-scale application and urging for automatic solutions for the analysis, especially recognizing the plankton species from images. Despite the extensive research done on automatic plankton recognition, the latest cutting-edge methods have not been widely adopted for operational use. In this paper, a comprehensive survey on existing solutions for automatic plankton recognition is presented. First, we identify the most notable challenges that make the development of plankton recognition systems difficult and restrict the deployment of these systems for operational use. Then, we provide a detailed description of solutions found in plankton recognition literature. Finally, we propose a workflow to identify the specific challenges in new datasets and the recommended approaches to address them. Many important challenges remain unsolved including the following: (1) the domain shift between the datasets hindering the development of an imaging instrument independent plankton recognition system, (2) the difficulty to identify and process the images of previously unseen classes and non-plankton particles, and (3) the uncertainty in expert annotations that affects the training of the machine learning models. To build harmonized instrument and location agnostic methods for operational purposes these challenges should be addressed in future research.

Toward efficient deep learning system for in-situ plankton image recognition

Plankton is critical for the structure and function of marine ecosystems. In the past three decades, various underwater imaging systems have been developed to collect in-situ plankton images and image processing has been a major bottleneck that hinders the deployment of plankton imaging systems. In recent years, deep learning methods have greatly enhanced our ability of processing in-situ plankton images, but high-computational demands and longtime consumption still remain problematic. In this study, we used knowledge distillation as a framework for model compression and improved computing efficiency while maintaining original high accuracy. A novel inter-class similarity distillation algorithm based on feature prototypes was proposed and enabled the student network (small scale) to acquire excellent ability for plankton recognition after being guided by the teacher network (large scale). To identify the suitable teacher network, we compared emerging Transformer neural networks and convolution neural networks (CNNs), and the best performing deep learning model, Swin-B, was selected. Utilizing the proposed knowledge distillation algorithm, the feature extraction ability of Swin-B was transferred to five more lightweight networks, and the results had been evaluated in taxonomic dataset of in-situ plankton images. Subsequently, the chosen lightweight model and the Bilateral–Sobel edge enhancement were tested to process in-situ images with high level of noises captured from coastal waters of Guangdong, China and achieved an overall recall rate of 91.73%. Our work contributes to effective deep learning models and facilitates the deployment of underwater plankton imaging systems by promoting both accuracy and speed in recognition of plankton targets.

Identification and abundance of plankton as an indicator of the presence of mahseer fish (Tor sp.) in the Leuser River Lokop Area, East Aceh, Aceh

The abundance of plankton can describe aquatic life. The Jurung fish is a plankton feeder freshwater fish which has high economic value. This study aimed to identify plankton recognition as an indicator of the presence of Jurung fish (Tor sp.) in the Leuser area of the Lokop River in Aceh and analyze the physico-chemical factors associated with attracting plankton in the Lokop River. This research was conducted from February to March 2021. The research showed that the number of types of plankton found in the Lokop river was 17 types (13 phytoplankton and 4 zooplankton). Diversity index value of 1.538364 – 2.133691 and evennes index of 0.542975 - 0.753099 were classified as mesotropic waters with moderate fertility levels. In general, based on observations of plankton in the Lokop river, Jurung fish could be cultivated from upstream to downstream. However, it is better to be at station 1 (upstream) because there are still a lot of plankton species so that the food needs of the Jurung fish can be fulfilled.

Automatic plankton quantification using deep features

Abstract The study of marine plankton data is vital to monitor the health of the world’s oceans. In recent decades, automatic plankton recognition systems have proved useful to address the vast amount of data collected by specially engineered in situ digital imaging systems. At the beginning, these systems were developed and put into operation using traditional automatic classification techniques, which were fed with hand-designed local image descriptors (such as Fourier features), obtaining quite successful results. In the past few years, there have been many advances in the computer vision community with the rebirth of neural networks. In this paper, we leverage how descriptors computed using convolutional neural networks trained with out-of-domain data are useful to replace hand-designed descriptors in the task of estimating the prevalence of each plankton class in a water sample. To achieve this goal, we have designed a broad set of experiments that show how effective these deep features are when working in combination with state-of-the-art quantification algorithms.

Enhanced convolutional neural network for plankton identification and enumeration.

Despite the rapid increase in the number and applications of plankton imaging systems in marine science, processing large numbers of images remains a major challenge due to large variations in image content and quality in different marine environments. We constructed an automatic plankton image recognition and enumeration system using an enhanced Convolutional Neural Network (CNN) and examined the performance of different network structures on automatic plankton image classification. The procedure started with an adaptive thresholding approach to extract Region of Interest (ROIs) from in situ plankton images, followed by a procedure to suppress the background noise and enhance target features for each extracted ROI. The enhanced ROIs were classified into seven categories by a pre-trained classifier which was a combination of a CNN and a Support Vector Machine (SVM). The CNN was selected to improve feature description and the SVM was utilized to improve classification accuracy. A series of comparison experiments were then conducted to test the effectiveness of the pre-trained classifier including the combination of CNN and SVM versus CNN alone, and the performance of different CNN models. Compared to CNN model alone, the combination of CNN and SVM increased classification accuracy and recall rate by 7.13% and 6.41%, respectively. Among the selected CNN models, the ResNet50 performed the best with accuracy and recall at 94.52% and 94.13% respectively. The present study demonstrates that deep learning technique can improve plankton image recognition and that the results can provide useful information on the selection of different CNN models for plankton recognition. The proposed algorithm could be generally applied to images acquired from different imaging systems.

Deep learning and transfer learning features for plankton classification

Plankton are the most fundamental components of ocean ecosystems, due to their function at many levels of the oceans food chain. Studying and monitoring plankton distribution is vital for global climate and environment protection. Currently, much research is concentrated on the automated recognition of plankton and several imaging-based technologies have been developed for collecting plankton images continuously using underwater image sensors. In this paper, we present a study about an automated plankton recognition system, which is based on the fusion of different deep learning methods. In this work we study both the fine tuning of several deep learned models and transfer learning from the same models with the aim of exploiting their diversity in designing an ensemble of classifiers, we deal with: (i) the ability of fine-tuning pre-trained CNN for plankton classification, (ii) the possibility of using pre-trained CNN for transfer learning, (iii) the possibility of coupling pre-processing to CNN in order to improve their feature extraction capability. The combination of such different descriptors/methods in a heterogeneous ensemble grants a substantial performance improvement with respect to other state-of-the-art approaches based on feature selection and classification. The experimental evaluation on three large publicly available datasets demonstrates high classification accuracy and f-measure of our ensemble with respect to other classifiers on the same datasets. One of the main contributions of this work is a collection of classification models and a wide experimental evaluation to report performance of both single CNN and ensemble of CNNs in different available plankton datasets with a given testing protocol. Moreover, we show how to combine different CNN in order to improve the performance. To encourage future comparisons the MATLAB source code is available in the GitHub repository: https://github.com/LorisNanni.

Binary SIPPER plankton image classification using random subspace

Plankton recognition plays an important role in the ocean environmental research. In this paper, we propose a random subspace based algorithm to classify the plankton images detected in real time by the Shadowed Image Particle Profiling and Evaluation Recorder. The difficulty of such classification is multifold because the data sets are not only much noisier but the plankton are deformable, projection-variant, and often in partial occlusion. In addition, the images in our experiments are binary, thus are lack of texture information. Using random sampling, we construct a set of stable classifiers to take full advantage of nearly all the discriminative information in the feature space of plankton images. The combination of multiple stable classifiers is better than a single classifier. We achieve over 93% classification accuracy on a collection of more than 3000 images, making it comparable with what a trained biologist can achieve by using conventional manual techniques.

Optimizing the number of classes in automated zooplankton classification

Zooplankton biomass and abundance estimation, based on surveys or time-series, is carried out routinely. Automated or semi-automated image analysis processes, combined with machine-learning techniques for the identification of plankton, have been proposed to assist in sample analysis. A difficulty in automated plankton recognition and classification systems is the selection of the number of classes. This selection can be formulated as a balance between the number of classes identified (zooplankton taxa) and performance (accuracy; correctly classified individuals). Here, a method is proposed to evaluate the impact of the number of selected classes, in terms of classification performance. On the basis of a data set of classified zooplankton images, a machine-learning method suggests groupings that improve the performance of the automated classification. The end-user can accept or reject these mergers, depending on their ecological value and the objectives of the research. This method permits both objectives to be equally balanced: (i) maximization of the number of classes and (ii) performance, guided by the end-user.

Human and machine factors in algae monitoring performance

We all take our visual systems for granted, and often assume we are always ‘near perfect’ observers. This is not the case; expert visual recognition is complex and can be error prone. Starting with examples that define the problem I will explore some of the issues of recognition where expert judgements are required. In addition to ‘expert’ effects, there are a number of cognitive factors that can severely affect performance, including fatigue, boredom, recency effects, positivity bias and short-term memory effects. Experimental evidence of the impact of these on performance are presented and discussed. The specimen identifications generated by experts are useful not only to ecology, but to researchers developing systems for automatic labelling of marine plankton. Comparisons of performance are presented, where human experts have been pitted against machines to label plankton. Consensus of opinion is important in reducing errors, yet it is the norm for experts to operate alone. The shortcomings of man and machines engaged in plankton recognition are reviewed and the future of automation is assessed.

Automatic plankton image recognition with co-occurrence matrices and Support Vector Machine

A long-standing problem in plankton ecology is sparseness of taxa-specific data. New optical imaging systems are becoming available which can acquire high-resolution data on the abun- dance and biomass of plankton taxa. The Video Plankton Recorder (VPR) has been designed and used for automatic sampling and visualization of major planktonic taxa at sea in real time, providing high-resolution data over a broad range of scales. Although these optical systems produce digital images of plankton that can be automatically identified by computer, the limited accuracy of auto- matic classification methods can reduce confidence in subsequent abundance estimates, especially in areas where a taxon is in low relative abundance. This paper describes an improved classification system for automatic identification of plankton taxa from digital images. Classifiers are trained from a set of images that were classified by human experts. The data set used to verify the classification system contains over 20 000 planktonic images manually sorted into 7 different categories. The new method uses co-occurrence matrices (COM) as the feature, and a Support Vector Machine (SVM) as the classifier. This new method is compared against a previous plankton recognition system, which used moment invariants, Fourier descriptors and granulometry as features and a learning vector quantization neural network as a classifier. The new method reduced the classification error rate from 39 to 28%. Subsequent plankton abundance estimates are improved by more than 50% in regions of low relative abundance. In general, the reduction in classification error was due to a combination of the use of COM and SVM.

Fat neural network for recognition of position-normalised objects

The design of a recognition system for natural objects is difficult, mainly because such objects are subject to a strong variability that cannot be easily modelled: planktonic species possess such highly variable forms. Existing plankton recognition systems usually comprise feature extraction processing upstream of a classifier. Drawbacks of such an approach are that the design of relevant feature extraction processes may be very difficult, especially if classes are numerous and if intra-class variability is high, so that the system becomes specific to the problem for which features have been tuned. The opposite course that we take is based on a structured multi-layer neural network with no shared weights, which generates its own features during training. Such a large parameterised—fat—network exhibits good generalisation capabilities for pattern recognition problems dealing with position-normalised objects, even with as many as one thousand weights as training examples. The advantage of such large networks, in terms of generalisation efficiency, adaptability and classification time, is demonstrated by applying the network to three plankton recognition and face recognition problems. Its ability to perform good generalisation with few training examples, but many weights, is an open theoretical problem.