GAN-aided Serial Dependence Study in Medical Image Perception

Abstract

Medical imaging has been critically important for the health and well-being of millions of patients. Although deep learning has been widely studied in medical imaging area and the performance of deep learning has exceeded human's performance in certain medical diagnostic tasks, detecting and diagnosing lesions still depends on the visual system of human observers (radiologists), who completed years of training to scrutinize anomalies. Routinely, radiologists sequentially read batches of medical images one after the other. A basic underlying assumption of radiologists' precise diagnosis is that their perceptions and decisions on a current medical image are completely independent from the previous reading history of medical images. However, recent research proposed that the human visual system has visual serial dependencies (VSDs) at many levels. VSD means that what was seen in the past influences (and captures) what is seen and reported at this moment. Our pilot data via naive artificial stimuli has shown that VSD has a disruptive effect in radiologic searches that impairs accurate detection and recognition of tumors or other structures. However, the naive artificial stimuli have been noted by both untrained observers and expert radiologists to be less authentic. In this project, we will generate authentic medical images via Generative Adversarial Networks (GANs) in order to replace the simple stimuli in future experiments. The rationale for the proposed research project is that once it is known how serial dependence arises and how it impacts visual search, we can understand how to control for it. Hence, the accuracy of diagnosis via medical imaging can significantly improve. The specific goals of this project are to establish, identify and mitigate the impact of VSD on visual search tasks in clinical settings.