Abstract Study question Can we use focal stacks collected through Hoffman modulation contrast (HMC) microscopy to generate 3D reconstructions of preimplantation embryos? Summary answer A machine learning system was designed to generate 3D meshes that approximate the structures of embryos captured on HMC microscopes up to the 8-cell stage. What is known already The 3D arrangement of cells in preimplantation human embryos is a topic of clinical interest, with significant associations between the cell arrangement and blastulation potential from as early as the 4-cell stage. In basic research, the use of confocal microscopy for generating 3D reconstructions is commonplace. However, the use of confocal microscopy in the IVF clinic is often infeasible due to cost and concerns for embryos’ wellbeing. The assessment of 3D cell arrangement in clinical settings can thus prove difficult and time-consuming as many embryologists rely on focal stacks captured through the HMC microscopes widely integrated into incubators. Study design, size, duration The study was a retrospective analysis of 581 Embryoscope focal stacks of embryos from 4 clinics collected between 2018 and 2020. The number of planes in each stack ranged from 7-11 and cell outlines were annotated along with the depths at which they were most in-focus. A deep learning system was designed to generate 3D reconstructions of the embryos. Two clinics’ data were used for training (N = 551) and the others’ for evaluation (N = 30). Participants/materials, setting, methods The deep learning system consisted of three stages: a super-resolution module, a cell segmentation module and a depth regression module. The super-resolution stage was used to predict missing planes in focal stacks that did not contain 11 focal planes; the segmentation module identified individual cells; the depth regression module identified the focal plane at which each cell was most “in-focus”. Meshes were then generated under the assumption that blastomeres’ dimensions are similar along each axis. Main results and the role of chance The superresolution module was evaluated by calculating the structural similarity index (SSIM; an image similarity measure ranging from 0-1) between predicted and true planes when tasked with predicting missing frames in focal stacks with up to 4 planes artificially removed (by uniform random sampling). The module achieved an SSIM of 0.80. The predictions were also evaluated by 2 embryologists, a clinician and a developmental biologist on a scale of 1-5 (1=very unrealistic; 3=usable; 5=very realistic), achieving a mean score of 4.11. The segmentation module was evaluated on the proportion of cells it managed to identify (91%) as well as the mean overlap between predicted cell segmentations and the ground truth (intersection-over-union of 0.86). The depth module was evaluated on the mean deviation of predictions from the true most “in-focus” plane (0.73 planes). 3D reconstructions generated by the system were evaluated with reference to the original focal stacks by 2 embryologists on a 1-5 scale similar to before, with a mean score of 3.72. The most common issues with the reconstructions identified by the embryologists were missing cells/fragments, incorrect cell shape due to obstruction by the well’s edge and imprecise depth predictions (with the “true” depth being between focal planes). Limitations, reasons for caution As previously mentioned, some reconstructions had inaccuracies. These would likely be ameliorated through modifications to the system modules and more training data. Moreover, the system was not trained or evaluated on morulae/blastocysts. Finally, each focal stack was analysed independently - future work may examine enforcing temporal consistency within timelapses. Wider implications of the findings This work serves as a first step towards unlocking data captured in IVF clinics for research into cell arrangement in preimplantation embryos. Combined with cell tracking, the system may be useful for research into cell fate. Moreover, the work may find clinical relevance in enabling easier assessment of cell arrangement. Trial registration number N/A
Read full abstract