Could generative artificial intelligence replace fieldwork in pain research?

Abstract

Generative artificial intelligence (AI) models offer potential assistance in pain research data acquisition, yet concerns persist regarding data accuracy and reliability. In a comparative study, we evaluated open generative AI models' capacity to acquire data on acute pain in rock climbers comparable to field research. Fifty-two rock climbers (33 m/19 f; age 29.0 [24.0-35.75] years) were asked to report pain location and intensity during a single climbing session. Five generative pretrained transformer models were tasked with responses to the same questions. Climbers identified the back of the forearm (19.2%) and toes (17.3%) as primary pain sites, with reported median pain intensity at 4 [3-5] and median maximum pain intensity at 7 [5-8]. Conversely, AI models yielded divergent findings, indicating fingers, hands, shoulders, legs, and feet as primary pain localizations with average and maximum pain intensity ranging from 3 to 4.4 and 5 to 10, respectively. Only two AI models provided references that were untraceable in PubMed and Google searches. Our findings reveal that, currently, open generative AI models cannot match the quality of field-collected data on acute pain in rock climbers. Moreover, the models generated nonexistent references, raising concerns about their reliability.