Knowledge retrieval and diagnostics in cloud services with large language models

Abstract

Efficient customer support is the foundation for any service provider trying to improve customer relationships. An important measure of successful support is the mean time to resolve issues. The complexity and large scale of modern cloud environments make it unrealistic to reduce the resolution time without deploying intelligent solutions. The latest also provides an exceptional opportunity to leverage cross-customer product usage data for proactive solutions when the troubles of some users can be analyzed in advance to prevent similar issues of other users. We build a recommender system that matches customer support requests to other resolved support requests or knowledge base articles that contain valuable information for problem remediation. This system can be used by customers or support teams to quickly find problem-resolution tips or detect trending issues to warn vulnerable users. We utilize large language models, fine-tune for better performance, and discuss capabilities and possible improvements. During our research, we highlighted several evaluation metrics such as mean time to resolve issues and the accuracy of recommendations. However, estimating accuracy is challenging due to insufficient datasets with precise and comprehensive recommendations. Despite this, our support managers provided some estimates regarding the remediation durations. Typically, identifying and resolving an issue takes several days or weeks. With appropriate recommendations, this time can be significantly reduced to several hours and, in some simple cases, even lead to self-service capabilities.