Objective Emergency medical services (EMS) clinicians are expected to provide expert care to all patients, but face obstacles in maintaining skillsets required in the care of critically ill or injured children. The objectives of this study were to describe and assess the effectiveness of a pediatric-focused, simulation-based, procedural training program for EMS clinicians, delivered on-site by a pediatric simulation education team. We also describe a novel, remote, asynchronous performance outcome measurement system using first-person-view video review. Methods This was a prospective study of simulation-based training and procedural outcomes. The study population involved EMS clinicians at three fire-based EMS agencies stratified as urban, suburban, and rural sites. The primary outcome was performance of intraosseous catheterization (IO), bag-valve-mask ventilation (BVM), and supraglottic device placement (SGD), measured across three time points. Secondary outcomes were identification of differences across EMS agencies and participant survey responses. Results We obtained video data from 122 clinicians, totaling 561 videos, with survey response rates of 89.0–91.3%. Pre-intervention scores were high: least-square means (95% confident-intervals) 9.5 (8.9, 10.2) for IO; 9.6 (9.3, 9.9) for BVM; and 11.6 (10.9, 12.2) for SGD. There was significant improvement post-intervention: 11.5 (10.7, 12.3) for IO; 11.0 (10.7, 11.4) for BVM; and 13.6 (12.8, 14.4) for SGD. Improvement was maintained at follow-up after a median of 9.5 months: 10.5 (9.8, 11.2) for IO; 10.2 (9.9, 10.6) for BVM; and 12.4 (11.7, 13.1) for SGD. There were no statistical differences between sites. Of survey respondents, half had not cared for a critically ill or injured child in at least a year, the vast majority had not had hands-on pediatric training in over 6 months, and the majority felt that training should occur at least every 6 months. Conclusions Our pediatric-focused, simulation-based procedural training program was associated with improvement and maintenance of high-baseline procedural performance for EMS clinicians over the study period. Findings were consistent across sites. Remote assessment was feasible. Participant surveys emphasized a desire for more pediatric-focused training and highlighted the low frequency of clinical exposure to procedures potentially needed in the care of critically ill or injured pediatric patients.