Intent Detection in Urdu Queries using Fine-tuned BERT models

Abstract

User's intent detection provides essential cues in query understanding and accurate information retrieval through search engines and task-oriented dialogue systems. Intent detection from user queries is challenging due to short query length and lack of sufficient context. Further, limited prior research in query intent detection has been conducted for Urdu, an under-resourced language. With the recent success of Bidirectional Encoder Representation from Transformers (BERT), that provides pre-trained language models, we propose to develop intent detection model for Urdu by fine-tuning BERT variants for intent detection task. We conduct rigorous experimentation on mono and cross-lingual transfer learning approaches by using pre-trained BERT models i.e. mBERT, ArBERT, and roBERTa-urdu-small and two query datasets. Experimental evaluation reveal that the fine-tuned models of mBERT and roBERTa-urdu-small achieve 96.38% and 93.30% accuracy respectively on datasets I and II outperforming strong statistical and neural network baselines.