Sentiment Analysis on Household Audio Conversation with Automated Chatbot for Hospital Visits Prediction

Abstract

AbstractBackgroundPeople living with dementia (PLwD) often experience difficulties describing their physical conditions and seeking medical attention timely, resulting in hospital visits that may cause frustration and exhaustion. Therefore, we aim to develop a machine learning model that helps identify potential hospital visit needs to provide early medical attention for PLwD.MethodPortable smart chatbots are deployed to PLwD households. We collected audio clips between households and chatbots and the hospitalisation records of PLwD. We manually labelled the sentiment scores of each audio clip based on its content and tone to between ‐3 and 3, where ‐3 indicates very negative, 0 indicates neutral, and 3 indicates very positive. We use 10 days of sentiment scores on a minute basis (0 when there’s no interaction) to predict the need for hospital visits in the following days. The positive outputs indicate that hospital visits happened within the 5 following days, while the negative outputs indicate no hospital visits occurred. A gradient boosting model (GBM) is trained to predict if hospital visits will happen in the following 5 days. We evaluated the trained model using the cross‐validation (cv = 5) method and reported Accuracy, Sensitivity, Specificity and macro F1 score.ResultBetween May 13th, 2021, to June 5th 2022, we collected 14 PLwDs’ hospital visit records and 12,278 audio clips between their households and their chatbots. Among all the participating households, we constructed a total of 3065 days as training and testing data (albeit no activities some days), with 37 hospital visit occurrences. This results in 139 days where hospital visits occurred in the following 5 days. The GBM model classified the need for hospital visits in the following 5 days with 0.682 Accuracy while obtaining 0.890 in Sensitivity, 0.669 in Specificity, and 0.522 in F1 score.ConclusionSentiment scores obtained by analysing audio clips between households of PLwD and their chatbots can be used to predict hospital visits in the following 5 days, with 0.890 in Sensitivity. This can help health professionals to provide more timely care to PLwD without frequent home visits. However, further research is needed to improve the Specificity and F1 of the predicting model.