P-599 An expected benefit analysis of using an interpretable machine learning model for optimizing the day of trigger during ovarian stimulation

Abstract

Abstract Study question What is the expected benefit of using a machine learning model for predicting the optimal day of trigger during ovarian stimulation? Summary answer Patients who had an optimal day of trigger had improved outcomes compared to propensity matched patients who were triggered late or early. What is known already The timing of the final trigger injection in ovarian stimulation is a subjective decision that varies across clinics and providers, with limited data to support objective criteria. Many studies report that follicles too small or too large on the day of trigger are less likely to yield a mature egg, although it remains unclear on how to apply these findings towards optimizing trigger timing. Clinical decision support tools to optimize the day of trigger have recently been developed, but these existing models rely upon black-box machine learning algorithms that are unable to explain the basis for their recommendations. Study design, size, duration We performed a retrospective analysis of patients undergoing autologous IVF cycles from 2014 - 2020 (n = 30,278) at three different IVF clinics in the United States. Data were split into train (70%), validation (10%), and test (20%) sets. The primary outcomes were the average number of MIIs, 2PNs, and usable blastocysts. Participants/materials, setting, methods Linear regression models were trained to predict MIIs retrieved if triggering today, MIIs if triggering tomorrow, and next-day E2 levels using follicle counts and estradiol levels. A trigger day recommendation algorithm evaluated each patient’s simulation records day-by-day in order to compare MII outcomes if triggering today vs. tomorrow. If the predicted number of MIIs showed an increasing trend, the recommendation was to continue stimulation; for a decreasing trend, the recommendation was to trigger. Main results and the role of chance The linear regression model for predicting MIIs on the day of trigger had a mean absolute error (MAE) of 2.87 oocytes and an R2 of 0.64, and the model for predicting next-day MIIs had an MAE of 3.02 oocytes and an R2 of 0.62. Next-day E2 levels were predicted with a MAE of 274 pg/mL and R2 of 0.88. Our model coefficients indicate that follicles 14-15mm and 16-17mm in diameter were most important for predicting MIIs if triggering today, while small follicles < = 10mm and large follicles >19mm were least important. For predicting next-day MIIs, follicles 11-13mm were most important, while follicles of size >19mm remained the least important. Possible early and late triggers were identified in 48.7% and 13.8% of cycles, respectively, by comparing the actual day of trigger to what the model recommended. After propensity score matching, patients with early triggers had on average 2.3 fewer MIIs, 1.8 fewer 2PNs, and 1.0 fewer blastocysts compared to matched patients with on-time triggers, and patients with late triggers had on average 2.7 fewer MIIs, 2.0 fewer 2PNs, and 0.7 fewer blastocysts compared to matched patients with on-time triggers. Limitations, reasons for caution The primary limitation is the retrospective nature of this study. Further, we did not differentiate between different trigger medications or types of protocols. Some cycles in our dataset had incomplete or missing data, which were excluded from analysis, and could have introduced sampling bias. Wider implications of the findings Our results suggest that an interpretable machine learning model can help optimize the day of trigger for increasing MII outcomes in a significant number of patients. Future work will include continuing to increase the diversity of our dataset and performing validation studies to show improved outcomes with model use. Trial registration number not applicable