Machine learning models outperform manual result review for the identification of wrong blood in tube errors in complete blood count results.

Abstract

Wrong blood in tube (WBIT) errors are a significant patient-safety issue encountered by clinical laboratories. This study assessed the performance of machine learning models for the identification of WBIT errors affecting complete blood count (CBC) results against the benchmark of manual review of results by laboratory staff. De-identified current and previous (within seven days) CBC results were used in the computer simulation of WBIT errors. 101015sets of samples were used to develop machine learning models using artificial neural network, extreme gradient boosting, support vector machine, random forest, logistic regression, decision trees (one complex and one simple) and k-nearest neighbours algorithms. The performance of these models, and of manual review by laboratory staff, was assessed on a separate data set of 1940samples. Volunteers manually reviewing results identified WBIT errors with an accuracy of 85.7%, sensitivity of 80.1% and specificity of 92.1%. All machine learning models exceeded human-level performance (p-values for all metrics were <.001). The artificial neural network model was the most accurate (99.1%), and the simple decision tree was the least accurate (96.8%). Sensitivity for the machine learning models varied from 95.7% to 99.3%, and specificity varied from 96.3% to 98.9%. This study provides preliminary evidence supporting the value of machine learning for detecting WBIT errors affecting CBC results. Although further work addressing practical issues is required, substantial patient-safety benefits await the successful deployment of machine learning models for WBIT error detection.