Transformers and their application to medical image processing: A review

Abstract

Transformers perform well in natural language processing tasks and have made many breakthroughs in computer vision. In medical image processing, transformers are successfully used in image segmentation, classification, reconstruction, and diagnosis. In this paper, we mainly expound on the transformer principle and its application in medical imaging. Specifically, we first introduce the basic principles and model structure of transformers. Then, we summarize the improvement mechanism of the transformer's network including combining the transformer with the Unet network, creating a transformer lightweight variant network, strengthening the cross-fast link mechanism, and building a large model with the transformer as the skeleton. Second, extensive discussion is given to medical image segmentation, reconstruction, classification, and other applications. Finally, the main challenges transformers face in the medical image processing field and future development prospects. Furthermore, we systematically summarize the latest research progress of transformers and their application in medical image processing, which has significant reference value for transformer research in the medical field.