Counterfactual can be strong in medical question and answering

Abstract

Medical question and answering is a crucial aspect of medical artificial intelligence, as it aims to enhance the efficiency of clinical diagnosis and improve treatment outcomes. Despite the numerous methods available for medical question and answering, they tend to overlook the data generation mechanism’s imbalance and the pseudo-correlation caused by the task’s text characteristics. This pseudo-correlation is due to the fact that many words in the question and answering task are irrelevant to the answer but carry significant weight. These words can affect the feature representation and establish a false correlation with the final answer. Furthermore, the data imbalance mechanism can cause the model to blindly follow a large number of classes, leading to bias in the final answer. Confounding factors, including the data imbalance mechanism, bias due to textual characteristics, and other unknown factors, may also mislead the model and limit its performance.In this study, we propose a new counterfactual-based approach that includes a feature encoder and a counterfactual decoder. The feature encoder utilizes ChatGPT and label resetting techniques to create counterfactual data, compensating for distributional differences in the dataset and alleviating data imbalance issues. Moreover, the sampling prior to label resetting also helps us alleviate the data imbalance issue. Subsequently, label resetting can yield better and more balanced counterfactual data. Additionally, the construction of counterfactual data aids the subsequent counterfactual classifier in better learning causal features. The counterfactual decoder uses counterfactual data compared with real data to optimize the model and help it acquire the causal characteristics that genuinely influence the label to generate the final answer. The proposed method was tested on PubMedQA, a medical dataset, using machine learning and deep learning models. The comprehensive experiments demonstrate that this method achieves state-of-the-art results and effectively reduces the false correlation caused by confounders.