Generating Lymphoma Ultrasound Image Description with Transformer Model

Abstract

AbstractLymphoma is a malignant tumor originating from lymphatic hematopoietic system and is the most common type of hematologic tumor. High-resolution ultrasound can display the size, morphology, internal echo structure and its changes of lymphoma, providing rich diagnostic information, clinicians can rely on its microscopic ultrasound performance to screen out suspected tumors and then obtain a definite pathological diagnosis through puncture biopsy. The ultrasonographic manifestations of lymphoma are complex and varied, which affects the accurate judgment of the nature of lymphoma by the sonographer. To solver these problems, this paper proposes a lymphoma ultrasound image description generation model based on Transformer [11] to provide auxiliary advice for ultrasound doctors in screening. In this paper, deep stable learning [8] was integrated into the model, and the dependence between features was removed by learning and training sample weights to make the model more focused on lymphoma. In order to make ultrasound doctors better understand the reason of the model description, the mapping of the prediction sequence to the input image in the cross attention layer was visualized to show the ultrasonic image basis of each prediction word in the model. The experimental results on the ultrasonic diagnosis dataset of lymphoma in 696 patients from Shanghai Ruijin Hospital show that the prediction effect of the proposed model is superior to that of the relevant methods mentioned in the literature.KeywordsLymphomaImage captioningDeep stable learningTransformer