Improved Transformer Based Hashing for Cased-based Pneumonia Retrieval

Abstract

The deep hash method has been proved to be the most effective nearest neighbor search technique in large-scale image retrieval. However, up to now, deep hashing in medical image retrieval has been dominated by convolution network architecture, such as Resnet. Inspired by Transformer, we integrate Transformer into the deep learning framework. Our framework is mainly composed of two main modules: (1) In order to solve the problem of big difference among different subclasses of medical image categories, the Transformer model is improved in this paper. The external attention is used to replace the self-attention in the original Transformer model, and the feature acquisition ability of the model is improved by capturing the correlation between samples. (2) In addition, in order to learn compact binary coding, we adopt Bayesian framework for end-to-end learning. We conduct experiments on ChestX-ray Pneumonia dataset, and compare the proposed algorithm with the commonly used depth hashing method. Experimental results show that the MAP values under different hash bit codes are better than the existing deep hash methods, which proves the effectiveness of the proposed method.