e18711 Background: High-quality data is needed to assess the effectiveness and impact of quality improvement collaboratives (QICs). The first Mexico in Alliance with St. Jude Golden Hour Collaborative (MAS Collaborative) ran from May 2019 to November 2020 in 23 hospitals across Mexico and improved the percentage of febrile pediatric hematology-oncology patients (P-HOP) presenting to the emergency department (ED) who receive the first dose of antibiotics in ≤60min from 39% to 78%. This study aimed to evaluate the quality of the data collected during the first MAS Collaborative and inform changes to second, larger-scale MAS Collaborative. Methods: Data quality was determined by data availability throughout the reporting . A complete sequence of was required for inclusion in the analysis. Results and data quality reports were created retrospectively and reviewed by three expert panels to better understand challenges related to data collection. A focus group with MAS Collaborative participants was conducted to review the data collection workflows and identify opportunities for improvement. Results: 2,103 febrile events in P-HOP were reported; 180 (8.6%) events were excluded, 96 (4.6%) informed the baseline, and 1,827 (86.8%) the implementation period. While data availability was excellent for service outcome and process, data availability for other elements of the reporting cascade was lower: 85% for adherence to the institutional guide, 71% for ICU transfer, 70% for sepsis, infections, and blood cultures, 68% for death, and 66% for critical interventions. Experts recommended narrowing the operational definition of critical interventions to those relevant to managing sepsis prior to transfer to the ICU (to assess access challenges), providing more intensive training to teams on the operational definitions for the clinical outcome measures, and more closely monitoring data completeness and quality. The focus group uncovered the need to reduce the number of documented times, differentiate the data collection process for physical or digital patient records, simplify other required variables and operational definitions, and to use a case form for data collection. Additional changes included explicit separation of service and clinical effectiveness measures, using a different software for data reporting and implementing ongoing data validation practices. Conclusions: This study highlights important challenges with the collection of high-quality clinical effectiveness data in the context of QICs in real-word settings. Distinction between service and clinical measures, robust measurement training, and data collection practices that accommodate varied workflows are provided as suggestions to improve data quality and to allow for a more accurate evaluation of the effectiveness and impact of the second MAS Collaborative.