Abstract Background: Circulating tumor cells (CTCs) are used as a liquid biopsy target in the treatment of cancer patients and are important for personalized medicine. Most CTC detection technologies are based on their surface markers such as EpCAM and cytokeratin. However, those positive selection methodologies appear to count only some CTCs because invasive tumor cells tend to change their surface markers during progression. Because the vast majority of nucleated cells circulating in the bloodstream are white blood cells (WBCs), eliminating WBCs from circulation is an efficient method to purify CTCs by negative selection. In this study, we demonstrate the development of cell recognition by computer vision technologies for pattern recognition based on the morphological features of live cells. Materials and methods: Blood samples were obtained from healthy volunteers. Five cancer cell lines, SW480, DLD-1, HCT116, Panc-1, and HepG2 were used as CTC models. We applied quantitative phase microscopy (QPM) that detects the small delay of the phase and image optical path lengths and provides quantitative morphological information of live cells with high contrast without staining. WBC recognition algorithm was developed using computer vision technologies for pattern recognition based on the histogram-oriented gradient (HOG) features extracted from QPM images (QPIs) for negative selection of CTCs. Results: We observed WBCs and five cell line cells by QPM, extracted certain features from the obtained images as training data for pattern recognition, and created an algorithm to differentiate WBCs from cell lines. The obtained algorithm successfully differentiated WBCs from cell lines (AUC=0.98). To simulate WBCs and cell lines with a small and strong change, respectively, in optical thickness (OT) at the center of the cell, we created a homogeneous hemi-ellipsoid model and a heterogeneous hemi-ellipsoid model, respectively. In these models, OT gradually increased from the edge to the center, but fluctuated around the center to simulate intracellular heterogeneity of a cancer cell. Based on these simulations, the classifier was interpreted to recognize intracellular heterogeneity, especially in the center of the cell. Finally, we applied our image recognition methods to QPIs of the cells flowing in the chamber, and properly differentiate WBCs from cell line cells flowing in the chamber (AUC=0.99). Conclusions: We developed a novel method for label-free image identification of live cells by computer vision technologies for pattern recognition based on the features of QPIs. Our study makes it possible to sort CTCs in a non-cytotoxic manner, thus providing good opportunities for molecular biological approaches to isolate CTCs. Citation Format: Hirotoshi Kikuchi, Yusuke Ozaki, Amane Hirotsu, Hidenao Yamada, Shigetoshi Okazaki, Tomohiro Murakami, Tomohiro Matsumoto, Yoshihiro Hiramatsu, Kinji Kamiya, Takanori Sakaguchi, Hiroyuki Konno, Hiroya Takeuchi. Label-free classification of live cells using quantitative phase microscopy images for negative selection of circulating tumor cells [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2018; 2018 Apr 14-18; Chicago, IL. Philadelphia (PA): AACR; Cancer Res 2018;78(13 Suppl):Abstract nr 2588.