Empirical study on using adapters for debiased Visual Question Answering

Abstract

In this work, we empirically study debiased Visual Question Answering (VQA) works with Adapters. Most VQA debiasing works sacrifice in-distribution (ID) performance for the sake of out-of-distribution (OOD) performance. Hence, we explore and experiment with the use of adapters to preserve the ID performance by training only a simple adapter network to debias and recreate performance. We conduct an extensive empirical study on recent well-established VQA debiasing works and show that the entirety of the debiasing information from the proposed debiasing methods can be captured and modelled using a single fully connected layer while preserving original network performance by skipping the adapters. Through our exploration, we find that different placements of adapters are required for different debiasing techniques and show the different possibilities of using adapters for debiasing through our experiments. We believe our findings in this work open up more questions to be asked and explored for the VQA community.