Abstract
ChatGPT has demonstrated its potential as a surrogate knowledge graph. Trained on extensive data sources, including open-access publications, peer-reviewed research articles, and biomedical websites, ChatGPT extracted information on gene relationships and biological pathways so that it can be used to predict them. However, a major challenge is model hallucination, that is, high false positive rates. To assess and address this challenge, we systematically evaluated ChatGPT’s capacity for predicting gene relationships using GPT-3.5-turbo, GPT-4, and GPT-4o. Benchmarking against the KEGG Pathway Database as the ground truth, we experimented with diverse prompting strategies, targeting gene relationships of activation, inhibition, and phosphorylation. We introduced an innovative iterative prompt refinement technique. By assessing prompt efficacy using metrics such as F-1 score, precision, and recall, GPT-4 suggested improved prompts. A refined prompt, which combines a specialized role with explanatory text, significantly enhanced the performance. Going beyond pairwise gene relationships, we also deciphered complex gene interplays, such as gene interaction chains and pathways pertinent to diseases such as non-small cell lung cancer. Direct prompts showed limited success, but “least-to-most” prompting exhibited significant potentials for such network constructions. The methods in this study may be used for other bioinformatics prediction problems.
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
More From: International Journal of Artificial Intelligence and Robotics Research
Disclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.