Automatic Text Summarization For Conversational Chatbot

Abstract

The communication between humans and machines has benefited from recent developments in speech recognition and natural language understanding. As personal assistants and chatbots are slowly becoming a part of our daily life, we develop a conversational chatbot that can conduct text summarization utilising innovative NLP approaches. Text Summarization is a useful technique for making better use of the vast amount of material that is readily available to us on the Internet and in various archives. So, under the current situation, generating a summary is both necessary and advantageous. The aim of this paper is to review the existing text summarization algorithms do a comparative analysis and integrate them to a conversational chatbot. For generating the summary, we have used six different algorithms: LexRank, TextRank, Latent Semantic Analysis (LSA), T5, BART and PEGASUS. These create useful summaries after reviewing the most recent literature. The performance of these algorithms is compared using the BBC News Dataset, a publicly available dataset, and their results are evaluated using the RecallOriented Understudy for Gisting Evaluation (ROUGE) metric to perform analysis, choose the best algorithm among these, produce the summary, and integrate it to the chatbot. By providing succinct responses to questions in a fraction of the time it would take to read the entire document, this chatbot significantly enhances the user experience.