EMViT-Net: A novel transformer-based network utilizing CNN and multilayer perceptron for the classification of environmental microorganisms using microscopic images

Abstract

Environmental microbes are certainly present in our surroundings since they are essential to the growth and survival of human advancement. The detailed analysis of environmental microorganisms (EMs) is very important to recognize, understand and make use of microbes as well and prevent damage. Extracting the discriminatory features from a limited-size dataset is very challenging for a deep learning model and a pure transformer-based network cannot achieve good classification results on a limited-size dataset due to the lack of muti-scale features. In this study, a novel vision transformer-based deep neural network is proposed by integrating the transformer with CNN for the classification of EM using microscopic images. The proposed network EMViT-Net has three main modules: a transformer module, a CNN module and a multilayer perceptron module. The transformer model extracted multiscale features to generate more discriminatory information from the images. A new separable convolutional parameter-sharing attention (SCPSA) block is integrated with the CNN module in the core of EMViT-Net, which makes the model robust to capture the local and global features, and simultaneously reduces the computational complexity of the model. The data augmentation is performed to introduce the variability in the dataset and counter the problem of overfitting and data imbalance. After extensive experiments and detailed analysis, it has been determined that the proposed model EMViT-Net outperforms the other existing methods and achieves state-of-the-art results with an accuracy of 71.17% which proves the effectiveness of the model for the classification of environmental microbes.