Large Language Models for Rapid Simplification of Quality Assurance Data Input: Field Trial with Real Data in the Context of Tumour Documentation in Urology

Abstract

Large Language Models (LLMs) such as ChatGPT have rapidly brought the application of artificial intelligence into widespread use. Among many different use cases for text generation and processing, one application is the extraction of data from existing documents and conversations for simplified and automated form-filling. In the field of quality assurance and documentation of cancer diseases, there is currently a significant workload involved in transferring data under various aspects into slightly varying formats and applying interpretations such as the TNM classification of tumours. However, there is a lack of trials with real data to assess the applicability of LLM-supported processes in this area, which would enable an evaluation of efficiency and practicality. This study aims to implement and assess such a trial. A trial was conducted with N=153 privacy-compliant and ethics committee-cleared medical reports from 25 patients. Using the publicly available version of ChatGPT 4.0, an automated script was used to extract the date of initial diagnosis and common tumor classifications. The results were then individually checked for accuracy. Based on this, the utility of a simple system for guided support in tasks related to tumour documentation was assessed. Additionally, the approach was evaluated in terms of operational costs for the model and its applicability. In summary, the study concludes that the use of generative AI in this field is promising and suitable as a tool even in an untrained state. In a simplified calculation, costs of 35 cents are offset by a value creation of 61,54 euros. However, it also becomes clear that AI can only act in a supportive role, and the correct integration with pre-made specific prompts and tools into the workflow is crucial for a relevant performance. The use of generative AI in the context of search, transfer, and interpretation tasks in the creation of tumor documentation is a promising approach. However, its implementation in practical applications must be closely monitored, and the optimal interaction between man and machine should continue to be evaluated and must be accompanied by tools and task-specific prompts.