How Can We Ensure Reproducibility and Clinical Translation of Machine Learning Applications in Laboratory Medicine?

Abstract

Recent studies demonstrating problems with COVID (1) and sepsis (2) prediction models have helped raise awareness (3) about the need for better practices in developing and reporting machine learning (ML; or artificial intelligence) methods in healthcare. This so-called reproducibility (or replication) crisis has been recognized and extends beyond clinical applications (4). In fact, this issue is not specific to ML. Many scientific journals, including Clinical Chemistry, have adopted principles (specified in the article submission guidelines) to facilitate reproducibility, rigor, and transparency in published findings. Although there are common elements that support transparency and rigor in science, each technology has its own set of pitfalls that must be addressed. This is true for the rapidly developing field of ML. The growth and development of open-source software and digitized and publicly available data sources have made the application of ML methods highly accessible. While this has facilitated a surge in interest and publications, it has reduced the requirement for developers to have the necessary foundational or subject matter knowledge needed for quality publications and innovations. When ML methods are published in clinical journals, peer reviewers and editors may lack the expertise to appropriately evaluate technical aspects of submissions. Thus, we need best practices that help educate clinical experts and govern how ML for laboratory medicine should be developed and communicated. This is critical for ensuring practicality and reproducibility of ML applications and, ultimately, their successful translation to clinical practice.