Discovering User Sentiment Patterns in Libraries with a Hybrid Machine Learning and Lexicon-Based Approach

Abstract

The need to enhance library services is the focus of this study, which relies on user feedback for data-driven decision-making. Text data from library user surveys conducted at Politeknik Caltex Riau (PCR) is analyzed to categorize sentiment and identify areas for improvement. The biannual student and lecturer feedback collected from 2018 to 2023 through the institution's official survey system (survey.pcr.ac.id) is utilized, providing a comprehensive and robust picture of user needs across five years. Sentiment analysis is employed using the VADER method to classify user comments into positive or negative categories. Text preprocessing techniques, such as stemming, tokenizing, and filtering, are performed to ensure robust classification. Machine learning algorithms – Naïve Bayes, Support Vector Machine (SVM), and Random Forest – are then utilized to evaluate sentiment classification accuracy. The study offers significant findings. Both SVM and Random Forest achieve an outstanding accuracy of 99%, indicating highly reliable sentiment categorization. Notably, these algorithms also achieve 100% precision, recall, and F1-score, demonstrating their effectiveness in accurately identifying positive and negative user sentiment. While Naïve Bayes shows slightly lower accuracy at 98%, it maintains a high recall rate (100%), ensuring all negative feedback is captured. This research presents a novel approach combining user sentiment analysis with a comprehensive five-year dataset. This enables a deeper understanding of evolving user needs and priorities. The high accuracy and effectiveness of the employed algorithms highlight the potential of this methodology for libraries. Libraries can leverage user feedback for evidence-based service improvement and increased user satisfaction.