Using Machine Learning to Measure Specialist Wait Times from Family Physicians’ Electronic Medical Records Linked to Ontario Health Administrative Data

Abstract

IntroductionReliable information about the time spent waiting for health care services is a critical metric for measuring health system performance. Wait times are a useful measure of access to various health care sectors. Alongside the increased adoption of electronic medical records (EMR) by Canadian family physicians (FP), is the secondary use of FP EMR data for research. However, using FP EMR data can be challenging in its unstructured, free-text format.
 Objectives and ApproachOur objective was to identify the target specialist physician type from the EMR FP referral note and then calculate wait times from a FP referral to a specialist physician visit. We used FP EMR data and linked to Ontario, Canada health administrative data (called EMRPC). EMRPC collects the entire clinical record from patients including the content of FP referral notes. We used machine learning (ML) methods to identify the type of specialist physician in which the referral was intended. Labels to test the ML methods were created from physicians’ claims data. Wait times were calculated from the FP EMR referral note date to the specialist physician claim date in administrative data.
 ResultsOur ML models’ ability to classify 2016 FP EMR referral notes to selected medical and surgical specialists achieved sensitivity and positive predictive values ranging from the high 70s to low 80s.Compared to earlier analyses from 2008, we observed a similar relative ordering to see specific specialist physicians. Overall, the median wait times have increased by 14 days on average, with a maximum increase of 28 days to see a gastroenterologist.
 Conclusion / ImplicationsThe accuracy of ML on unstructured FP EMR data is high, thereby providing a mechanism to “codifying” information in a timely manner. This information can help inform decision makers and providers about which patients or FP practices are experiencing long wait times in seeing specialist physicians.