Fish Images Research Articles

Fish-Finder: A robust small target detection method for aquaculture fish in low-quality underwater images.

Underwater fish object detection serves as a pivotal research direction in marine biology, aquaculture management, and computer vision, yet it poses substantial challenges due to the complexity of underwater environments, occultations, and the small-sized and frequently moving fish in aquaculture. Addressing these challenges, we propose a novel underwater fish object detection algorithm named Fish-Finder. First, we engendered a structure titled "C2fBF," utilizing the dual-path routing attention protocol of BiFormer. The primary objective of this structure is to alleviate the perturbations induced by underwater intricacies during the phase of downsampling in the backbone network, thereby discerning and conserving finer contextual features. Subsequently, we co-opted the RepGFPN method within our neck network-a distinctive approach that adeptly merges high-level semantic constructs with low-level spatial specifics, thus fortifying its multi-scale detection prowess. Then, in an endeavor to diminish the sensitivity toward positional aberrations during the detection of diminutive aquatic creatures, we incorporated a novel bounding box regression loss function, the Wasserstein loss, to the existing CIoU. This innovative function gauges the congruity between the predicted bounding box Gaussian distribution and the reference bounding box Gaussian distribution. Finally, in regard to the dataset, we independently assembled a specific dataset termed "SmallFish." This unique dataset, meticulously designed for the detection of small-scale fish within intricate underwater settings, includes 5000 annotated images of small fish. Experimental results demonstrate that, compared to the state-of-the-art detection methods, our proposed method improves the accuracy by and , and mean average precision (mAP) increases and in public dataset Kaggle-Fish and our SmallFish dataset, respectively.

Assessment of fish health status for waterless transportation based on image features and deep learning models

In the current aquaculture industry, the detection of physiological stress indicators in fish during waterless transportation is considered to be an effective method to monitor their health status. However, commonly used detection methods are invasive and interfere with the accuracy of detection results. In order to solve this problem, this study takes grouper as an example, combines multi-sensor physiological stress indicators monitoring, and designs a health status assessment and classification model based on image features and deep learning models, which inputs real-time fish image feature information data acquired during low-temperature waterless transportation into the optimised CNN-BiGRU classification model based on whale optimisation algorithm (WOA) proposed in this study, so as to achieve an accurate assessment of the health status of the fish. By inputting the image feature datasets based on time series and classified into the WOA-CNN-BiGRU, BiGRU, and GRU classification models for training, validation, and testing, respectively, the effectiveness of the classification performance of the models was verified by using accuracy, precision, recall and F1-value composite scores as the metrics. It was found that the WOA-CNN-BiGRU model had excellent classification performance, followed by the BiGRU and GRU models, especially in classifying the health status assessment of small size grouper. This study proposes an efficient and inexpensive method for real-time monitoring and accurate assessment of the health status of fish during waterless transportation, and also provides a reference for vigour monitoring and health assessment of other similar aquatic products.

A Convolutional Neural Network Approach for Precision Fish Disease Detection

Background: Detecting and classifying fish diseases is crucial for maintaining the health and sustainability of aquaculture systems. This study employs deep learning techniques, particularly Convolutional Neural Networks (CNNs), to automate the detection of various fish diseases using image data. Methods: The study utilizes a carefully curated dataset sourced from the Kaggle database, comprising images representing seven distinct types of fish diseases, along with images of healthy fish. Data preprocessing techniques, including resizing, rescaling, denoising, sharpening, and smoothing, are applied to enhance image quality and facilitate accurate disease detection. Data augmentation is employed to increase the model's ability to generalize to unseen data. The CNN architecture is designed with cascading convolutional layers, ReLU activation functions, and pooling operations to extract high-level features associated with fish infections. The model architecture, implemented using the Keras Sequential API, includes convolutional layers, max pooling layers, and densely connected layers for classification. Results: Experimental results demonstrated promising performance across various disease categories, with high accuracy and balanced precision and recall values for most classes. The study also discussed the impact of climate change on fish disease incidence and underscores the importance of effective monitoring and management practices facilitated by technological innovations such as Big Data, IoT, sensors, and robotics in ensuring sustainable fisheries management.

Leveraging deep learning and computer vision technologies to enhance management of coastal fisheries in the Pacific region

This paper presents the design and development of a coastal fisheries monitoring system that harnesses artificial intelligence technologies. Application of the system across the Pacific region promises to revolutionize coastal fisheries management. The program is built on a centralized, cloud-based monitoring system to automate data extraction and analysis processes. The system leverages YoloV4, OpenCV, and ResNet101 to extract information from images of fish and invertebrates collected as part of in-country monitoring programs overseen by national fisheries authorities. As of December 2023, the system has facilitated automated identification of over six hundred nearshore finfish species, and automated length and weight measurements of more than 80,000 specimens across the Pacific. The system integrates other key fisheries monitoring data such as catch rates, fishing locations and habitats, volumes, pricing, and market characteristics. The collection of these metrics supports much needed rapid fishery assessments. The system’s co-development with national fisheries authorities and the geographic extent of its application enables capacity development and broader local inclusion of fishing communities in fisheries management. In doing so, the system empowers fishers to work with fisheries authorities to enable data-informed decision-making for more effective adaptive fisheries management. The system overcomes historically entrenched technical and financial barriers in fisheries management in many Pacific island communities.

Fishing event detection and species classification using computer vision and artificial intelligence for electronic monitoring

Fisheries regulations require detailed catch reporting on commercial fishing vessels. Vital components for the sustainable management of fish stocks include a robust estimate of the number of fish caught and the species composition. Catch recording is often done manually by human observers on fishing vessels. Human observers are costly, and consistent data streams can be subject to observer availability and the weather. On-vessel cameras (electronic monitoring, EM) are a growing alternative to human observers. However, on-land human auditors are required to review hundreds of hours of videos recorded during fishing trips that can last for weeks. In this paper, a framework is presented to automatically detect fish in EM videos, count the total fishing events, and classify the fish species. For this purpose, a deep learning and computer vision-based model is developed to efficiently detect fish and fishers onboard a vessel. Secondly, a vision-based tracking pipeline tracks the detected fish and counts the total fishing events in the videos. Thirdly, the extracted fishing events are classified through a deep learning-based fish species classifier, to provide the distribution of different fish species caught for a fishing trip. For our experiments, the datasets were prepared using the electronic monitoring data of multiple fishing trips of a fishing vessel. The videos were recorded on Australian longline vessels targeting tunas and billfish. For the fish detection task, video frames were extracted and labelled manually to provide a digital ground-truth. For the fish species classification task, hundreds of fish images of multiple species were cropped to provide a training dataset for the fish classifier. For the fish counting task, manual counts for the fishing events of individual fish species were generated for the test fishing trips. The developed fish and fisher detector achieves a mean Average Precision of 87.0 % for fish and 94.0 % for fishers on test video frames. The fishing event detection pipeline achieves an Average Precision of 81.0 % and an Average Recall of 74.5 % on test videos. The fish species classifier achieves an Accuracy (Top-1) of 91.11 % for the classification of cropped fish images and 89.05 % for the classification of extracted fishing events from the videos. Experimental results show that our proposed computer vision and artificial intelligence-based solution for video analysis has great potential to automate the auditing process from electronic monitoring footage and contribute to the sustainable management of fish stocks.

You only look once v8 for fish species identification

<p>This research aims to test the performance of you only look once (YOLOv8) in identifying fish species in Indonesian waters. Fish image data is obtained from various sources to conduct testing. The results show that YOLOv8 is able to identify fish species with a mAP accuracy rate of 97%. These results reveal the great potential of deep learning technology in supporting the preservation of marine biodiversity as well as the development of various applications, such as fisheries monitoring, conservation, and marine-based tourism development in Indonesia. With its efficient object detection and classification capabilities, YOLOv8 can simplify and accelerate the process of identifying fish species, even on a large scale. Thus, this technology is a highly effective solution to overcome the challenges of manual fish species identification, which requires a lot of time and effort. The results of this study provide valuable insights into the potential utilization of Indonesia's natural resources in the context of scientific development, the tourism industry, and the fisheries sector, which is vital to the country's economy.</p>

FishAI: Automated hierarchical marine fish image classification with vision transformer

AbstractTo address the issues of high demand for efficiently recognizing fish species in marine scientific research, such as impact assessments on biodiversity and monitoring, an automated hierarchical image classification web‐based platform, named FishAI, was developed. Trained with marine fish images collected from the World Register of Marine Species, FishAI used the Vision Transformer (ViT) model, to classify fish. The model considers hierarchy levels, covering 3 classes, 38 orders, 154 families, 438 genera, and 808 species. The FishAI achieved accuracies of 0.975 (Class), 0.798 (Order), 0.743 (Family), 0.638 (Genus), and 0.626 (Species) on test images, respectively, by using the hyperparameter optimization. Comparison between ViT and other baseline backbones proves its superiority by capturing long‐distance dependency. In addition, FishAI yields the top‐5 prediction accuracies of 1.000 (Class), 0.887 (Order), 0.816 (Family), 0.729 (Genus), and 0.727 (Species), respectively. In order to further enhance the practicality of FishAI, the user‐friendly graphic interface (http://www.csbio.sjtu.edu.cn/bioinf/FishAI/) facilitates its easy‐to‐use application. Furthermore, interpretability analysis by Grad‐CAM provides a visual explanation of the highlighted regions on the images for FishAI's prediction among different hierarchies.

Underwater image enhancement using contrast correction

AbstractLight‐induced degeneration of underwater images occurs by physical features of seawater. According to the wavelength of the colour spectrum, light reduces intensity significantly when it moves through water. The greatest wavelength of light that is visible gets absorbed first. Red and blue absorb the most and least, respectively. Because of the reducing consequences of the light spectrum, underwater images having poor contrast can be obtained. As a result, the crucial data contained inside these images will not be effectively retrieved for later analysis. The recent research suggests a novel approach to enhance the contrast while decreasing noise in underwater images. The recommended approach involves image histogram transformation using two significant colour spaces, Red‐Green‐Blue (RGB) and Hue‐Saturation‐Value (HSV). The histogram of the dominant colour channel (blue channel) in the RGB colour model is extended towards the lower level, containing a maximum limitation of 95%, while the inferior red colour channel has been extended towards the upper side, containing a minimum limitation of 5%. During the entire dynamic range, the green colour channel having the dominant and inferior colour channels expands in each direction. The Rayleigh distribution has been utilized for developing various stretching actions within the RGB colour space. The image has been converted to the HSV colour space, having the S and V elements adjusted within 1% of their minimum and maximum values. The suggested approach is examined in both qualitative and quantitative analysis. According to qualitative analysis, the recommended approach substantially boosts image contrast, lowers its blue and green effect, and minimizes over‐enhanced and under‐enhanced sections in the final resultant underwater image. The quantitative examination of 500 large scale underwater images dataset reveals that the suggested technique generates better results. The dataset images are grouped into small fish images, blue coral images, stone wall images, and coral branch images. The quantitative examination of all these four groups have been evaluated and shown. The average mean square error, peak signal to noise ratio, underwater image quality measurement, and underwater colour image quality evaluation values of dataset images are 76.69, 31.25, 3.85, and 0.64, respectively. These values of our proposed work outperform six other previous methods.

SISTEM DETEKSI PENGGUNAAN FORMALIN PADA PANGAN IKAN ASIN MENGGUNAKAN ALGORITMA CONVOLUTIONAL NEURAL NETWORK (CNN)

Salted fish is a food that is very popular in the community. In addition to its delicious taste, salted fish can be purchased at an affordable price. Salted fish is fish that has gone through a preservation process using salt preservatives. The process of preserving salted fish is very dependent on natural factors such as weather and sunlight, so this preservation process cannot be used all the time. This makes salted fish producers suffer losses, so salted fish producers do everything to gain profits, one of which is using the dangerous preservative formaldehyde. The effects of consuming formalin preservatives include abdominal pain, diarrhea, nervous depression, and circulatory disorders. Based on these problems, a tool is needed that can detect the use of formalin in salted fish. In this research, one of the deep learning methods is used, namely Convolutional Neural Networks (CNN) as an image identification which is proven to be efficient in classifying images. The method is implemented by classifying formalin salted fish based on the images inputted by the user. In this study, training was carried out 3 times to find the best value. The data used in this study were 200 images of salted fish captured using a smartphone camera consisting of 100 images of formalin salted fish and 100 images of non-formalin salted fish. The results of this study are that the system can recognize objects with an accuracy rate of 92.5% with 40 times of testing which can detect 3 times identifying wrong images and 37 times correctly identifying images.

Detection and Tracking of Underwater Fish Using the Fair Multi-Object Tracking Model: A Comparative Analysis of YOLOv5s and DLA-34 Backbone Models

Modern aquaculture utilizes computer vision technology to analyze underwater images of fish, contributing to optimized water quality and improved production efficiency. The purpose of this study is to efficiently perform underwater fish detection and tracking using multi-object tracking (MOT) technology. To achieve this, the FairMOT model was employed to simultaneously implement pixel-level object detection and re-identification (Re-ID) functions, comparing two backbone models: FairMOT+YOLOv5s and FairMOT+DLA-34. The study constructed a dataset targeting the popular black porgy in Korean aquaculture, using underwater video data from five different environments collected from the internet. During the training process, the FairMOT+YOLOv5s model rapidly reduced train loss and demonstrated stable performance. The FairMOT+DLA-34 model showed better results in ID tracking performance, with an accuracy of 44.1%, an IDF1 of 11.0%, an MOTP of 0.393, and an IDSW of 1. In contrast, the FairMOT+YOLOv5s model recorded an accuracy of 43.8%, an IDF1 of 14.6%, an MOTP of 0.400, and an IDSW of 10. The results of this study indicate that the FairMOT+YOLOv5s model demonstrated higher IDF1 and MOTP scores compared to the FairMOT+DLA-34 model, while the FairMOT+DLA-34 model showed superior performance in ID tracking accuracy and had fewer ID switches.

CATNet: Cascaded attention transformer network for marine species image classification

Complex physicochemical environmental effects result in the underwater species images’ highly intricate and diverse backgrounds, which poses significant challenges for identifying marine species. Deep learning techniques are used extensively for image classification thanks to their exceptional feature extraction capabilities. To address the above issues, we present a cascaded attention transformer network for marine species image classification named CATNet. More specifically, we first utilize the efficient channel attention module with depthwise convolution, which can efficiently extract the essential features of the image and reduce the computational parameters of the traditional attention module. Subsequently, we design a stacked transformer module to focus on the different features among a large amount of feature information, thereby reducing redundancy and improving?the generalization performance of CATNet. To comprehensively exploit the features extracted from different modules, we integrate the efficient channel attention and stacked transformer modules to construct a feature-cascaded module. Meanwhile, we construct a large-scale Marine Species Image Dataset (MSID) including 8605 underwater images with different species. Concretely, it consists of 712 coral reef images, 1270 fish images, 1082 people images, 2414 sea urchin images, 1219 squid images, 837 sea cucumber images, and 1071 turtle images. Comprehensive experimentation on our built dataset underscores the proposed CATNet’s superiority over the existing state-of-the-art methods in marine species image classification. The data is available. https://www.researchgate.net/publication/381961074_2024MSID.

CLIB: Contrastive learning of ignoring background for underwater fish image classification.

Aiming at the problem that the existing methods are insufficient in dealing with the background noise anti-interference of underwater fish images, a contrastive learning method of ignoring background called CLIB for underwater fish image classification is proposed to improve the accuracy and robustness of underwater fish image classification. First, CLIB effectively separates the subject from the background in the image through the extraction module and applies it to contrastive learning by composing three complementary views with the original image. To further improve the adaptive ability of CLIB in complex underwater images, we propose a multi-view-based contrastive loss function, whose core idea is to enhance the similarity between the original image and the subject and maximize the difference between the subject and the background, making CLIB focus more on learning the core features of the subject during the training process, and effectively ignoring the interference of background noise. Experiments on the Fish4Knowledge, Fish-gres, WildFish-30, and QUTFish-89 public datasets show that our method performs well, with improvements of 1.43-6.75%, 8.16-8.95%, 13.1-14.82%, and 3.92-6.19%, respectively, compared with the baseline model, further validating the effectiveness of CLIB.

Selective decision-making and collective behavior of fish by the motion of visual attention.

Collective motion provides a spectacular example of self-organization in Nature. Visual information plays a crucial role among various types of information in determining interactions. Recently, experiments have revealed that organisms such as fish and insects selectively utilize a portion, rather than the entirety, of visual information. Here, focusing on fish, we propose an agent-based model where the direction of attention is guided by visual stimuli received from the images of nearby fish. Our model reproduces a branching phenomenon where a fish selectively follows a specific individual as the distance between two or three nearby fish increases. Furthermore, our model replicates various patterns of collective motion in a group of agents, such as vortex, polarized school, swarm, and turning. We also discuss the topological nature of the visual interaction, as well as the positional distribution of nearby fish and the map of pairwise and three-body interactions induced by them. Through a comprehensive comparison with existing experimental results, we clarify the roles of visual interactions and issues to be resolved by other forms of interactions.

A lightweight underwater fish image semantic segmentation model based on U‐Net

AbstractSemantic segmentation of underwater fish images is vital for monitoring fish stocks, assessing marine resources, and sustaining fisheries. To tackle challenges such as low segmentation accuracy, inadequate real‐time performance, and imprecise location segmentation in current methods, a novel lightweight U‐Net model is proposed. The proposed model acquires more segmentation details by applying a multiple‐input approach at the first four encoder levels. To achieve both lightweight and high accuracy, a multi‐scale residual structure (MRS) module is proposed to reduce parameters and compensate for the accuracy loss caused by the reduction of channels. To improve segmentation accuracy, a multi‐scale skip connection (MSC) structure is further proposed, and the convolution block attention mechanism (CBAM) is introduced at the end of each decoder level for weight adjustment. Experimental results demonstrate a notable reduction in model volume, parameters, and floating‐point operations by 94.20%, 94.39%, and 51.52% respectively, compared to the original model. The proposed model achieves a high mean intersection over union (mIOU) of 94.44%, mean pixel accuracy (mPA) of 97.03%, and a frame rate of 43.62 frames per second (FPS). With its high precision and minimal parameters, the model strikes a balance between accuracy and speed, making it particularly suitable for underwater image segmentation.

Underwater Fish Object Detection with Degraded Prior Knowledge

Understanding fish distribution, behavior, and abundance is crucial for marine ecological research, fishery management, and environmental monitoring. However, the distinctive features of the underwater environment, including low visibility, light attenuation, water turbidity, and strong currents, significantly impact the quality of data gathered by underwater imaging systems, posing considerable challenges in accurately detecting fish objects. To address this challenge, our study proposes an innovative fish detection network based on prior knowledge of image degradation. In our research process, we first delved into the intrinsic relationship between visual image quality restoration and detection outcomes, elucidating the obstacles the underwater environment poses to object detection. Subsequently, we constructed a dataset optimized for object detection using image quality evaluation metrics. Building upon this foundation, we designed a fish object detection network that integrates a prompt-based degradation feature learning module and a two-stage training scheme, effectively incorporating prior knowledge of image degradation. To validate the efficacy of our approach, we develop a multi-scene Underwater Fish image Dataset (UFD2022). The experimental results demonstrate significant improvements of 2.4% and 2.5%, respectively, in the mAP index compared to the baseline methods ResNet50 and ResNetXT101. This outcome robustly confirms the effectiveness and superiority of our process in addressing the challenge of fish object detection in underwater environments.

Adaptive density guided network with CNN and Transformer for underwater fish counting

Accurate assessment of high-density underwater fish resources is vital to the aquaculture industry. It is directly related to the formulation of fishery insurance strategies and the implementation of breeding plans. However, accurately counting fish in high-density environments becomes challenging due to the uneven distribution of fish density and individual fish’s different sizes and postures. To break through this technical bottleneck, we developed an advanced adaptive density-guided high-density fish counting network. In detail, first of all, the network adopts a multi-layer feature fusion structure similar to UNet, which significantly enhances the matching between fish targets of different scales and feature pyramid levels, effectively alleviating the problems caused by scale changes and morphological deformations. Secondly, the network also introduces a density-guided adaptive selection module, which can intelligently judge the applicability of Convolutional Neural Network and Transformer blocks in different density areas, thereby achieving robust information transfer and interaction between blocks. Finally, to verify the effectiveness of this method, we also specially constructed two high-density data sets: a simulated high-density underwater fish image data set (SHUFD) and a real high-density underwater fish image data set (RHUFD). The proposed method has significant improvements over the state-of-the-art method (CUT) on SHUFD and RHUFD datasets, with the mean absolute error, mean square error, background region bias, foreground region bias and density map bias indicators improving by 3.44% and 6.47%, 11.43% and 4.41%, 23.91% and 29.48%, 4.43% and 10.33%, 8.3% and 13.14%, respectively.

Research on Measuring the Bodies of Underwater Fish with Inclined Positions Using the YOLOv8 Model and a Line-Laser System

Fish body measurement is essential for monitoring fish farming and evaluating growth. Non-destructive underwater measurements play a significant role in aquaculture management. This study involved annotating images of fish in aquaculture settings and utilized a line laser for underwater distance calibration and fish body inclined-angle calculation. The YOLOv8 model was employed for fish identification and key-point detection, enabling the determination of actual body dimensions through a mathematical model. The results show a root-mean-square error of 6.8 pixels for underwater distance calibration using the line laser. The pre-training YOLOv8-n, with its lower parameter counts and higher MAP values, proved more effective for fish identification and key-point detection, considering speed and accuracy. Average body length measurements within 1.5 m of the camera showed a minor deviation of 2.46% compared to manual measurements. The average relative errors for body length and width were 2.46% and 5.11%, respectively, with corresponding average absolute errors. This study introduces innovative techniques for fish body measurement in aquaculture, promoting the digitization and informatization of aquaculture processes.

Spatial confinement affects the heterogeneity and interactions between shoaling fish

Living objects are able to consume chemical energy and process information independently from others. However, living objects can coordinate to form ordered groups such as schools of fish. This work considers these complex groups as living materials and presents imaging-based experiments of laboratory schools of fish to understand how activity, which is a non-equilibrium feature, affects the structure and dynamics of a group. We use spatial confinement to control the motion and structure of fish within quasi-2D shoals of fish and use image analysis techniques to make quantitative observations of the structures, their spatial heterogeneity, and their temporal fluctuations. Furthermore, we utilize Monte Carlo simulations to replicate the experimentally observed data which provides insight into the effective interactions between fish and confirms the presence of a confinement-based behavioral preference transition. In addition, unlike in short-range interacting systems, here structural heterogeneity and dynamic activities are positively correlated as a result of complex interplay between spatial arrangement and behavioral dynamics in fish collectives.

DyFish-DETR: Underwater Fish Image Recognition Based on Detection Transformer

Due to the complexity of underwater environments and the lack of training samples, the application of target detection algorithms to the underwater environment has yet to provide satisfactory results. It is crucial to design specialized underwater target recognition algorithms for different underwater tasks. In order to achieve this goal, we created a dataset of freshwater fish captured from multiple angles and lighting conditions, aiming to improve underwater target detection of freshwater fish in natural environments. We propose a method suitable for underwater target detection, called DyFish-DETR (Dynamic Fish Detection with Transformers). In DyFish-DETR, we propose a DyFishNet (Dynamic Fish Net) to better extract fish body texture features. A Slim Hybrid Encoder is designed to fuse fish body feature information. The results of ablation experiments show that DyFishNet can effectively improve the mean Average Precision (mAP) of model detection. The Slim Hybrid Encoder can effectively improve Frame Per Second (FPS). Both DyFishNet and the Slim Hybrid Encoder can reduce model parameters and Floating Point Operations (FLOPs). In our proposed freshwater fish dataset, DyFish-DETR achieved a mAP of 96.6%. The benchmarking experimental results show that the Average Precision (AP) and Average Recall (AR) of DyFish-DETR are higher than several state-of-the-art methods. Additionally, DyFish-DETR, respectively, achieved 99%, 98.8%, and 83.2% mAP in other underwater datasets.

Improving fish image detection speed with hybrid VGG16 and darknet

Improving fish image detection speed with hybrid VGG16 and darknet