A causality guided loss for imbalanced learning in scene graph generation

Abstract

Listen

Translate article

Unbiased visual relation detection on long-tailed annotations is a critical challenge in scene graph generation (SGG). Imbalanced learning aims to tackle the problem of class distribution that is long-tailed in order to learn unbiased models from imbalanced data. Since long-tailed datasets are inevitable in the real world, obtaining a balanced dataset can be expensive or even impossible. However, training models on such data are easily biased towards head classes and underperform on tail classes. To overcome this challenge, existing methods focus more on utilizing label frequency as prior knowledge, but ignore the research on how imbalanced datasets lead to prediction bias, which is crucial for solving the long-tail problem. Therefore we propose a causal graph for the training process. This causal graph reveals the conventional loss serves as a confounder of the features and predictions during training. Guided by the causal graph, a degree-of-difficulty loss (DDloss) is designed which is a simple yet effective method to alleviate catering to the head. We demonstrate the effectiveness of DDloss through extensive experiments on SGG and test its expansibility on long-tailed image classification.