Exploring Explainability in Arabic Language Models: An Empirical Analysis of Techniques

Abstract

Transformer-based language models have advanced Natural Language Processing (NLP), achieving state-of-the-art results across various tasks. However, their complex architectures often obscure the decision-making processes, making transparency a critical challenge, especially in sensitive applications. The emerging field of eXplainable Artificial Intelligence (XAI) seeks to address this by enhancing model transparency. Nonetheless, the primary focus of XAI has largely been on high-resource languages, neglecting low-resource ones such as Arabic. In this paper, we first show the importance of studying XAI in the Arabic language. We then detail our methodology, which involves adapting AraBERT and AraGPT models to specific tasks including Arabic sentiment analysis and semantic question similarity. Then, we conduct an empirical study to evaluate various XAI methods, specifically gradient-based and perturbation-based approaches. These methods are assessed using two key metrics: faithfulness and plausibility. Our findings suggest that while gradient-based methods are more faithful, perturbation-based methods align better with human judgment.