Abstract
We propose a fine-tuning methodology and a comprehensive comparison between state-of-the-art pre-trained language models (PLM) when applying to Vietnamese Sentiment Analysis. The fine-tuning architecture includes three main components: (1) pre-processing, (2) a pre- trained language model, and (3) a multi-layer perceptron. The method exploits pre-trained contextual language models in order to represent input sentences. Pre-trained contextual language models are belong to three different kinds: multilingual, cross-lingual and monolingual. We conduct experiments to evaluate trained classifiers fine-tuned using five different contextual language models. The experimental results on two open-access datasets show that the sentiment classifiers trained using the monolingual language model outperform of which cross-lingual and monolingual language models. The results provide an additional evidence about the representation power of monolingual PhoBERT in comparison with multilingual BERT and cross-lingual XLM.
Published Version
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
