Abstract Study question Can a deep-learning algorithm classify live, stain-free human sperm cells based on the whole cell morphology in real-time? Summary answer Our model predicts sperm morphology with 94% precision and accuracy, maintaining performance despite over six-fold image quality reduction, demonstrating less than a 12% accuracy drop. What is known already Sperm morphology analysis plays a crucial role in infertility diagnosis and treatment. However, current clinical methods either use diagnostic staining, rendering sperm cells unsuitable for treatment or depend on subjective manual examination. While deep learning has presented new opportunities for morphology analysis, previous studies only evaluate sperm head morphology, neglecting other important morphological characteristics of sperm midpiece and tail. Additionally, these algorithms rely on fixed and stained images of sperm, preventing their applicability for the classification of live, unstained sperm cells for use in treatment cycles. Study design, size, duration Donor human semen samples (with a concentration range of 30-80 million/mL) comprising 8 technical replicates sourced from 6 biological replicates, were used to generate a dataset of 2254 stain-free single sperm images. Subsequently, three expert clinicians from two different fertility clinics labelled the images according to their morphological status (assessing head, midpiece, and tail morphology) as the ground truth for training our deep-learning model. Model development, training, testing, and analysis were then performed. Participants/materials, setting, methods Using the YOLOv3 algorithm, 2254 single sperm images were automatically extracted from 1000× magnification microscopy images. Three expert andrology scientists from Monash IVF classified and labelled these images based on the WHO morphological characteristics of the sperm head, midpiece, and tail. Then our optimized ensemble structure was trained using 80% of the images for morphology classification of the whole cell. Testing on the remaining unseen images was conducted to assess accuracy, precision, and processing time. Main results and the role of chance While our clinical labelling results highlighted the subjective nature of current manual morphology analysis practices in clinics, with only 16% agreement between experts, our model classifies human sperm images based on their whole morphology status with an accuracy and precision of 94% in real-time (less than a second per image). Testing our model across multiple image resolutions (800 × 800, 224 × 224, 192 × 192, and 128 × 128 pixels) revealed consistent performance, with less than a 12% variation. This underscores the robustness and compatibility of our model across various clinical imaging settings. Importantly, for more challenging images with 2-out-of-3 experts’ agreement (53% of the images) or images without an agreement between experts (25% of the images), our model consistently classified 96% and 97% of the images with at least 75% confidence, respectively. This demonstrates the performance of the model in classifying sperm images that pose difficulties for the experts to label. Consistency in operation and informing sperm selection with quantitative metrics through this platform offers promising opportunities to reduce day-to-day variability in fertility clinics when analysing or selecting sperm. Limitations, reasons for caution This study is confined to samples and clinics exclusively located in Australia, while a larger number of samples across multiple clinics could enhance the generalizability of findings. Additionally, real-time clinical implications and long-term performance evaluation are still needed. Wider implications of the findings This platform not only provides automated evaluation of live sperm as part of infertility diagnostics but also facilitates selection of the most morphologically normal sperm for treatment cycles, particularly in the context of Intracytoplasmic Sperm Injection (ICSI). Therefore, it can enhance positive long-term treatment outcomes by prioritizing normal sperm selection. Trial registration number 2017370