Reliability and validity of a quality tool for assessing clinical forensic medicine legal reports

Abstract

ObjectiveIt is essential that reports written by forensic medicine practitioners undergo appropriate quality control. The aim of this study is to develop and validate a tool for assessing the quality of medico-legal reports in adult and adolescent sexual assault cases. MethodsThe authors developed an audit tool and accompanying guideline aimed at detecting errors, omissions, and inadequacies in medico-legal reports following reported adult or adolescent sexual assault. The authors conducted a benchmarking exercise to reach an agreed audit standard. Subsequently two forensic examiners audited 5 legal reports, first without the tool, and then with the tool following standardised instruction. A further ten forensic examiners audited 20 reports after receiving instruction in use of the tool. Their results were compared to the agreed audit standard. Participants were interviewed about their experiences. ResultsUse of the tool to audit reports significantly increased the sensitivity of error detection compared to usual practice (sensitivity with usual practice 55% vs sensitivity with tool 80%). Study participants using the tool to audit 20 reports detected 73% of items designated as errors by the study authors. The overall accuracy in coding items as errors/not errors was 74%. Interrater reliability was good (Cronbach's alpha = 0.87). Sensitivity, overall accuracy, and interrater reliability results varied by category in the audit: reviewers had lower levels of error detection and lower levels of agreement when auditing opinions in the report as compared to items describing the clinical forensic evaluation. Participants had fundamental disagreements about what constitutes good quality in some aspects of a report, including: the acceptability of including ‘non-relevant’ history; whether to include references and, if so, what constitutes appropriate citations; and how to determine whether the opinions could be understood by a layperson. Study participants reported that using the audit tool and guideline to review medico-legal reports has merit and suggested improvements to increase usability. ConclusionUse of an audit tool supported by a guideline and training is useful for improving error detection and standardising the review process for clinical forensic medicine legal reports. Further research aimed at improving the consensus about opinion standards in adult and adolescent sexual assault cases would be valuable.