Real Time Three-Dimensional image processing with Application to Automated Cell Surgery Visual Servoing

Abstract

The locations of the target cell, its internal structures and surgical tools are essential to achieve visual servoing control for automated cell surgery. Real time image feedback allows the controller to adjust promptly for location variation due to motion of the cell when in contact with surgery tools. This paper uses Z-stack images of a 2-blastomere mouse embryo cell to develop a novel real time three-dimensional (3D) image processing algorithm. The proposed algorithm computes the centroid of each embryo blastomere and surgical tool tip in 3D image-plane coordinate. 3D Canny edge detector processes the embryo to produce a segmented 3D model. By converting the 3D model into a point cloud, the centroid of each blastomere is then estimated using an unsupervised machine learning technique, k-means clustering. For the surgical tool, 2D Canny edge detector is used in this case to compress computation time. The surgical tool tip location is selected as the furthest point away from the image edge where surgical tool appears. With 6.1μm computation variation and 2.4Hz update frequency, the proposed algorithm is suitable to perform automated cell surgery using visual servoing, especially Image-Based Visual Servoing (IBVS), with the obtained image-plane locations in the 3D image. The proposed algorithm has also shown theoretical potential to be implemented into other embryo development stages.