Abstract

Adverse drug events (ADEs) contribute significantly to morbidity and mortality in the healthcare system. The availability of digitalised hospitals' narrative clinical data offers a potentially rich resource to enhance pharmacovigilance efforts to manage potential safety issues arising from real-world use of drugs. The goal of this paper was to establish a foundation for creating an evaluation corpus by developing a set of annotation guidelines to achieve high inter-annotator agreement (IAA) and to evaluate the performance of basic entity identification tools for drugs, adverse events (AEs) and drug-AE relationships from 100 discharge summaries of a tertiary hospital in Singapore. Two teams of three annotators worked independently on text annotation using Knowtator. Three-way IAA of 86%, 70% and 49% were achieved for drugs, AEs and drug-AE relationships respectively. The performance of the machine algorithm was evaluated against annotations made by at least two annotators, with a recall of 84% and precision of 73% for drugs and a recall of 67% and precision of 53% for AEs. The high recall and precision for drug entity extraction suggests that machine pre-annotation of drugs followed by human annotation of AEs and drug-AE relationships could be a feasible approach in expediting the process of creating a larger evaluation corpus. Non-matches between machine and human annotations were examined to identify ways to further refine the algorithm. When successfully implemented, the identification of ADEs could greatly support pharmacovigilance work in characterising the magnitude and scope of ADEs and prioritising interventions to improve the drug safety.

Talk to us

Join us for a 30 min session where you can share your feedback and ask us any queries you have

Schedule a call

Disclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.