Banjarese Chatbot Using Seq2Seq Model

Abstract

The significant development of chatbot technology began with the implementation of the Seq2Seq (Sequence-to-Sequence) model which is based on two RNN (Recurrent Neural Network) architecture. Long-term training with vanilla RNN has the problem of vanishing or exploding gradient, so it is necessary to do research that focuses more on optimization steps for this problem. The optimization carried out on the Seq2Seq model is to apply a LSTM (Long-Short Term Memory) gate mechanism with a stable ability to understand long-term information. A GRU (Gated Recurrent Unit) is faster in training and an attention mechanism that can handle long sequence data. Also, gradient descent can minimize the error rate. A combination of gate types, gradient descent, and attention mechanisms needs to be done to produce a better conversational model. The test results of the conversational model performance were obtained by taking into account the metric scores in accuracy, precision, recall, F1, and loss by using 80% train data and 20% test data from 5000 lines of Banjarese conversation. In the 50th epoch, the best score was from testing the LSTM gate model with metric values of accuracy 64,67%, precision 85,20%, recall 62,49%, F1 72,09%, and loss 3,6073.