Generation of echocardiographic 2D images of the heart using cGAN

Abstract

One of the most significant tasks of echocardiography is the automatic delineation of the cardiac structures from 2D echocardiographic images. Over the past decades, the automation of this task has been the subject of intense research. One of the most effective approaches is based on the deep convolutional neural networks. Nonetheless, it is necessary to use echocardiogram frames of the cardiac muscle, which show the boundaries of the cardiac structures labeled/annotated by experts/cardiologists to train it. However, the number of databases containing the necessary information is relatively small. Therefore, generated echocardiogram frames are used to increase the amount of training samples. This process is based on the ultrasound images of the heart, annotated by experts. The article proposes an improved method for generating echocardiograms using a generative adversarial neural network (GAN) with a patch-based conditional discriminator. It has been demonstrated that it is possible to improve the quality of generated echocardiogram frames in both two and four chamber views (AP4C, AP2C) using the masks of cardiac segmentation with sub-pixel convolution layer (pixel shuffle). It is demonstrated that the proposed approach makes it possible to generate ultrasound images, the structure of which corresponds to the specified segmentation masks. It is expected that this method will improve the accuracy of solving the direct problem of automatic segmentation of the left ventricle.