Inter- and intra-observer variability of time-lapse annotations

Abstract

How consistent is the time-lapse annotation of dynamic and static morphologic parameters of embryo development, within and between observers? The assessment of dynamic parameters is characterized by almost perfect agreement within and between observers. The commonly employed method used to assess embryos in IVF treatments is based on static evaluation of morphology in a microscope, but this is limited by substantial intra- and inter-observer variation. Time-lapse imaging has been proposed as a method to refine embryo selection by adding new dynamic predictors of viability to the assessment. Yet, there are no data regarding the consistency of estimates of the time-lapse parameters. Infertile patients were recruited at the Fertility Clinic, Arhus University Hospital from February 2011 to June 2012. All embryos were cultured for 6 days in a time-lapse incubator (EmbryoScope(™)). Automated image recording was performed every 20 min. In total, 158 fertilized embryos from 20 different patients were annotated. Three observers made independent annotations on time-lapse recordings. One observer performed the assessment twice. Twenty-five parameters were annotated and the inter- and intra-observer agreement was assessed by calculating intra-class correlation coefficients (ICCs). Extremely close agreement (ICC 0.99) was found for dynamic parameters including the timing of the following: pronuclei breakdown, completion of blastocyst hatching and the appearance and disappearance of the first nucleus after the first division. Observations of cleavage divisions were strongly correlated (ICC > 0.8), indicating close agreement. Measurements of the static morphologic parameters, i.e. multi-nucleation and evenness of blastomeres at 2-cell stage showed fair-to-moderate agreement (ICC ≤ 0.5). The study was conducted at a single clinic. Only embryos with a good prognosis were included. The influence of training sessions was not measured. Consistency is crucial to the validity of embryo scoring and selection. All of the time-lapse parameters suggested by the literature showed in our study high intra- and inter-observer correlation, thus validating the precision of time-lapse annotations. This provides the basis for further investigation of embryo assessment and selection by time-lapse imaging in prospective trials. Research at the Fertility Clinic was funded by an unrestricted grant from Ferring and MSD. The authors have no competing interests to declare. NCT01139268.