With the rapid development of microfluidic platforms in high-throughput single-cell culturing, laborious operation to manipulate massive budding yeast cells (Saccharomyces cerevisiae) in replicative aging studies has been greatly simplified and automated. As a result, large datasets of microscopy images bring challenges to fast and accurately determine yeast replicative lifespan (RLS), which is the most important parameter to study cell aging. Based on our microfluidic diploid yeast long-term culturing (DYLC) chip that features 1100 traps to immobilize single cells and record their proliferation and aging via time-lapse imaging, herein, a dedicated algorithm combined with computer vision and residual neural network (ResNet) was presented to efficiently process tremendous micrographs in a high-throughput and automated manner. The image-processing algorithm includes following pivotal steps: (i) segmenting multi-trap micrographs into time-lapse single-trap sub-images, (ii) labeling 8 yeast budding features and training the 18-layer ResNet, (iii) converting the ResNet predictions in analog values into digital signals, (iv) recognizing cell dynamic events, and (v) determining yeast RLS and budding time interval (BTI) ultimately. The ResNet algorithm achieved high F1 scores (over 92%) demonstrating the effectiveness and accuracy in the recognition of yeast budding events, such as bud appearance, daughter dissection and cell death. Therefore, the results conduct that similar deep learning algorithms could be tailored to analyze high-throughput microscopy images and extract multiple cell behaviors in microfluidic single-cell analysis.