Deep learning for ultrasound image caption generation based on object detection

Abstract

Deep learning for image caption generation makes great progress in the field of natural images. However, there are still lack of effective methods for detailed analysis and automatic description of diseases content information in ultrasound image understanding. In order to find the location of focus areas, and understand the content of focus areas conveniently, we propose a novel method of ultrasound image captioning generation based on region detection. The method simultaneously detects and encodes the focus areas in ultrasound images, then utilizes the LSTM to decode the encoding vectors and generate annotation text information to describe the diseases content information in ultrasound images. The experimental results show that the method can accurately detect the location of the focus area, and also improves 1% the scores of BLEU-1, BLEU-2 with less parameters and running time, which compared with the full-size-image captioning model for ultrasound images.