Non-destructive classification of sturgeon stress using cross-modal data fusion and multi-input deep learning models

Abstract

In the aquaculture phase, ensuring the safe transportation of sturgeon is crucial. The stress levels experienced during transit directly impact the fish quality and the economic returns for farmers. To address this, distributors enlist fishery farming experts to evaluate sturgeon stress. Our investigation identified three critical parameters for grading: sturgeon physiological, environmental, and visual characteristics. This study aims to develop a Cross-Modal Stress Classification Network (CM-SCN) model. It integrates information from three sensing systems to assess sturgeon stress levels. The model is built upon the architectures of AlexNet and ANN, skillfully combining both image and non-image data sources. This integration enables the model to effectively categorize sturgeon stress into four classes: A, B, C, D (representing mild, minor, moderate, and severe stress levels). The results demonstrated the model's high performance with an accuracy of 88.96%, precision of 90.06%, recall of 89.43%, and an F1 score of 89.49%. Notably, the CM-SCN model surpassed both the unimodal visual stress model and the bimodal physiological-environmental stress model. This study introduces an efficient and dependable method for monitoring sturgeon health, offering promising advancements in the field.