Crack detection has recently gained credence in the field of engineering automation. Engineered cementitious composites (ECC) are a durable, and environmentally friendly construction material and a suitable replacement for conventional concrete. As such, there is a need to automate the process of detecting the unique ductile crack formation and growth in ECC. This research focuses on detecting ECC cracks, accurately mapping the crack morphology, and predicting the crack growth patterns of this advanced material. We design and present a novel generative adversarial algorithm named Residual Pyramidal Generative Adversarial Network (RP-GAN), capable of ECC crack detection. RP-GAN is an enhanced and improved version of the typical Pix2Pix GAN for paired image-to-image translation. It is an extensive network that could be used for macro and micro-segmentation purposes in addition to domain translation tasks. In this research, we first investigate the ability of RP-GAN to detect ECC micro-cracks and map the dimensions of each crack. Secondly, we compared RP-GAN’s crack detection results with those of known state-of-the-art architectures. We then developed an algorithmic pipeline for live crack tracking using an experimental framework. Finally, we evaluate the possibility of detecting ECC crack growth and propagation using sequential images by re-engineering RP-GAN and fusing Convolutional-LSTM into its architecture. The mean intersection over union (IOU) test results of RP-GAN for ECC crack detection equaled 81.28%, and 77.76% for ECC crack propagation prediction.