A Guide to AI in epidemiology: ChatGPT and the STROBE checklist for observational studies

Abstract

Abstract Background Chat-Generative Pre-training Transformer (ChatGPT) is a recently developed Artificial Intelligence (AI) model capable of generating high-quality scientific text and answering complex tasks. This study aims at investigating how AI-based transformers can support public health researchers in designing and conducting epidemiological studies. To accomplish this, we used ChatGPT to reformulate the STrengthening the Reporting of OBservational studies in Epidemiology (STROBE) recommendations for observational studies into a list of questions to be answered by the transformer itself. We then qualitatively evaluated the coherence and relevance of the transformer's outputs. Methods We first chose a study to be used as a basis for the simulation. We then used ChatGPT to transform each STROBE checklist's item into specific prompts. Each answer was evaluated by independent researchers in terms of coherence and relevance. Results The mean scores assigned to each prompt were heterogeneous. On average, for the coherence domain, the overall mean score was 3.6 out of 5.0, and for relevance it was 3.3 out of 5.0. The lowest scores were assigned to items belonging to the Methods section of the checklist. Conclusions ChatGPT can be considered as a valuable support for researchers in conducting an epidemiological study, following internationally recognized guidelines. It is crucial for the users to have knowledge on the subject and a critical mindset when evaluating its outputs. The potential benefits of AI in scientific research are undeniable, but it is crucial to address the risks, and the ethical and legal consequences associated with its use. Key messages • Our analysis showed the ability of AI-based transformers to generate answers and human-like text, which could potentially be used to conduct epidemiological studies or write research articles. • We advocate the necessity for taking a proactive stance towards this evolving context, in order to ensure a careful governance of this inevitable process.