Recent advances of Transformers in medical image analysis: A comprehensive review

Abstract

AbstractRecent works have shown that Transformer's excellent performances on natural language processing tasks can be maintained on natural image analysis tasks. However, the complicated clinical settings in medical image analysis and varied disease properties bring new challenges for the use of Transformer. The computer vision and medical engineering communities have devoted significant effort to medical image analysis research based on Transformer with especial focus on scenario‐specific architectural variations. In this paper, we comprehensively review this rapidly developing area by covering the latest advances of Transformer‐based methods in medical image analysis of different settings. We first give introduction of basic mechanisms of Transformer including implementations of selfattention and typical architectures. The important research problems in various medical image data modalities, clinical visual tasks, organs and diseases are then reviewed systemically. We carefully collect 276 very recent works and 76 public medical image analysis datasets in an organized structure. Finally, discussions on open problems and future research directions are also provided. We expect this review to be an up‐to‐date roadmap and serve as a reference source in pursuit of boosting the development of medical image analysis field.