ABSTRACT Sperm Morphology Analysis (SMA) is an important technique for diagnosing male infertility, but manual analysis is laborious and subjective. Recent deep learning approaches aim to automate SMA, but are limited by scarce sperm image datasets. Generative Adversarial Networks (GANs) can synthesize realistic medical images to augment small datasets. This study applied a GAN-based augmentation technique to expand two sperm image datasets - Modified Human Sperm Morphology Analysis (MHSMA) with 1,540 images, and Human Sperm Head Morphology (HuSHeM) with 216 images. Augmentation doubled both datasets. The expanded datasets were used to train deep learning models to classify sperm abnormalities. The results in MHSMA reached an accuracy of 84.66%, 94.33% and 79.33% in the head, vacuole and acrosome labels, respectively. This result for HuSHeM equalled 95.1%. This improved on state-of-the-art results, demonstrating that GAN augmentation can optimize deep learning for SMA by generating synthetic training images. This allows automated, accurate sperm analysis from scarce datasets. By overcoming data limitations, deep learning with GAN augmentation can be practically implemented for SMA to improve efficiency, throughput and objectivity. This could assist clinicians in faster, more consistent sperm quality assessment and diagnosis of male infertility.