A comparative study of KAN transformer and traditional vision transformer for ultrasound image-based diagnosis

Abstract

Abstract. In recent years, the MLP architecture has almost been monopolized in the field of deep learning, and its success is undeniable, but at the same time there are some problems. Kolmogorov-Arnold Network (KAN) is a new neural network architecture based on Kolmogorov-Arnold theory implementation. Compared to traditional MLPs, KANs have higher interpretability, faster training, and more efficient usability. In this paper, based on the theory of KAN, the author try to replace the Multi-Layer Perceptron (MLP) architecture in Vision Transformer (ViT) with the better performing KAN,author conducted scoring classification experiments on clinical acute respiratory distress syndrome (ARDS) and pneumonia image datasets provided by the Emergency Department of Changzheng Hospital in Shanghai, China, in order to validate the feasibility of the application of KAN+ViT in assisting clinical medical ultrasound diagnosis as well as its high efficiency in comparison with the traditional MLP+ViT structure. And classification experiments on the Cifar-10 dataset are used to validate the superiority of this new architecture over the traditional ViT architecture.