Biomedical Natural Language Inference on Clinical trials using the BERT-based Models

Abstract

Clinical trials are crucial in experimental medicine as they assess the safety and efficiency of new treatments. Due to its unstructured and plain language nature, clinical text data often presents challenges in understanding the relationships between various elements like disease, symptoms, diagnosis, and treatment. This task is challenging as the Multi-evidence Natural Language Inference for Clinical Trial Data (NLI4CT) requires intricate reasoning involving textual and numerical elements. It involves integrating information from one or two Clinical Trial Reports (CTRs) to validate hypotheses, demanding a multi-faceted approach. To address these problems, we use BERT-base models’ ability to predict entailment or contradiction labels and compare the use of transformer-based feature extraction and pre-trained models. We utilize seven pre-trained models, including six BERT-based and one T5-based model: BERT-base uncased, BioBERT-base-cased-v1.1-mnli, DeBERTa-v3-base-mnli-fever-anli, DeBERTa-v3-base-mnli-fever-docnli-ling-2c, DeBERTa-large-mnli, BioLinkBERT-base, and Flan-T5-base. We achieve an F1-score of 61% on both DeBERTa-v3-base-mnli-fever-anli and DeBERTa-large-mnli models and 95% faithfulness on the BioLinkBERT-base model.