For diseases that lack efficient therapeutic approaches, stem cell therapy is an emerging alternative treatment, wherein cell quality assessment is an essential step before cell transplantation into patients. Cell quality is generally evaluated using approaches, such as immunohistochemistry and flow cytometry, which involve complex laboratory routines. In this milieu, several studies have demonstrated the feasibility of applying image processing and analysis to assist cell quality assessment more efficiently. We developed an image-based analytical tool using cell tracking, cytometric analyses, and gating with time-lapse microscopy to investigate the migratory behavior of human mesenchymal stem cells (hMSCs) cultured on chitosan membranes. Using this tool, systematic studies were carried out to inspect variations in cell quality by extracting two morphological and seven migratory features of cells of interest. We found that chitosan membrane-induced multi-cellular spheroids move at a higher speed and in a more consistent direction than single cells. The hMSCs showed a tendency to aggregate with strong 3-D spheroid formability and increased mobility during maturation; however, both these tendencies decreased with degeneration. Furthermore, we automated the cell quality assessment procedure by utilizing machine learning. We selected 570 hMSCs, 390 adipose-derived adult stem cells (hADSCs) and 180 human placenta-derived multipotent cells (hPDMCs), with complete movement trajectories from 25 videos, at three different passages. The differentiation accuracy between hADSCs and hPDMCs in the mature passage was 96.3%, which was higher than that in the early (82.6%) and late (80.5%) passages, whereas the accuracy of passage identification was 62.8% for hADSCs and 70.6% for hPDMCs. The limited accuracy of passage identification may be owing to the insufficient number of morphological features. This study shows the potential of applying machine learning in monitoring the quality of stem cells during in vitro culture.