Large language models help facilitate the automated synthesis of information on potential pest controllers

Abstract

Abstract The body of ecological literature, which informs much of our knowledge of the global loss of biodiversity, has been experiencing rapid growth in recent decades. The increasing difficulty of synthesising this literature manually has simultaneously resulted in a growing demand for automated text mining methods. Within the domain of deep learning, large language models (LLMs) have been the subject of considerable attention in recent years due to great leaps in progress and a wide range of potential applications; however, quantitative investigation into their potential in ecology has so far been lacking. In this work, we analyse the ability of GPT‐4 to extract information about invertebrate pests and pest controllers from abstracts of articles on biological pest control, using a bespoke, zero‐shot prompt. Our results show that the performance of GPT‐4 is highly competitive with other state‐of‐the‐art tools used for taxonomic named entity recognition and geographic location extraction tasks. On a held‐out test set, we show that species and geographic locations are extracted with F1‐scores of 99.8% and 95.3%, respectively, and highlight that the model can effectively distinguish between ecological roles of interest such as predators, parasitoids and pests. Moreover, we demonstrate the model's ability to effectively extract and predict taxonomic information across various taxonomic ranks. However, we do report a small number of cases of fabricated information (confabulations). Due to a lack of specialised, pre‐trained ecological language models, general‐purpose LLMs may provide a promising way forward in ecology. Combined with tailored prompt engineering, such models can be employed for a wide range of text mining tasks in ecology, with the potential to greatly reduce time spent on manual screening and labelling of the literature.