BERT BiLSTM-Attention Similarity Model

Abstract

Semantic similarity models are a core part of many of the applications of natural language processing (NLP) that we may be encountering daily, which makes them an important research topic. In particular, Question Answering Systems are one of the important applications that utilize semantic similarity models. This paper aims to propose a new architecture that improves the accuracy of calculating the similarity between questions. We are proposing the BERT BiLSTM-Attention Similarity Model. The model uses BERT as an embedding layer to convert the questions to their respective embeddings, and uses BiLSTM-Attention for feature extraction, giving more weight to important parts in the embeddings. The function of one over the exponential function of the Manhattan distance is used to calculate the semantic similarity score. The model achieves an accuracy of 84.45% in determining whether two questions from the Quora duplicate dataset are similar or not.