BackgroundThe first minutes after birth are critical to reducing neonatal mortality. Helping Babies Breathe (HBB) is a simulation-based neonatal resuscitation program for low resource settings. We studied the impact of initial HBB training followed by refresher training on the knowledge and skills of the birth attendants in facilities.MethodsWe conducted HBB trainings in 71 facilities in the NICHD Global Network research sites (Nagpur and Belgaum, India and Eldoret, Kenya), with a 6:1 ratio of facility trainees to Master Trainers (MT). Because of staff turnover, some birth attendants (BA) were trained as they joined the delivery room staff, after the initial training was completed (catch-up initial training). We compared pass rates for skills and knowledge pre- and post- initial HBB training and following refresher training among active BAs. An Objective Structured Clinical Examination (OSCE) B tested resuscitation skill retention by comparing post-initial training performance with pre-refresher training performance. We identified factors associated with loss of skills in pre-refresher training performance using multivariable logistic regression analysis. Daily bag and mask ventilation practice, equipment checks and supportive supervision were stressed as part of training.ResultsOne hundred five MT (1.6 MT per facility) conducted initial and refresher HBB trainings for 835 BAs; 76% had no prior resuscitation training. Initial training improved knowledge and skills: the pass percentage for knowledge tests improved from 74 to 99% (p < 0.001). Only 5% could ventilate a newborn mannequin correctly before initial training but 97% passed the post-initial ventilation training test (p < 0.0001) and 99% passed the OSCE B resuscitation evaluation. During pre-refresher training evaluation, a mean of 6.7 (SD 2.49) months after the initial training, 99% passed the knowledge test, but the successful completion rate fell to 81% for the OSCE B resuscitation skills test. Characteristics associated with deterioration of resuscitation skills were BAs from tertiary care facilities, no prior resuscitation training, and the timing of training (initial vs. catch-up training).ConclusionsHBB training significantly improved neonatal resuscitation knowledge and skills. However, skills declined more than knowledge over time. Ongoing skills practice and monitoring, more frequent retesting, and refresher trainings are needed to maintain neonatal resuscitation skills.Trial registrationClinicalTrials.gov Identifier: NCT01681017; 04 September 2012, retrospectively registered.Electronic supplementary materialThe online version of this article (doi:10.1186/s12884-016-1141-3) contains supplementary material, which is available to authorized users.