Quartz fiber-reinforced polymer (QFRP) is a vital non-polar material used in aviation wave-transparent structural components. Automatic characterization of delamination defects in QFRP is critical to aviation structural component safety. Terahertz time-domain spectroscopy (THz-TDS) is one of the new non-destructive testing (NDT) methods with highly accurate characterization of internal defects in non-polar material. Hence, attempts to extract features of THz time-domain signals for automatic characterization have been made by using deep learning algorithms. In this work, a Transformer-based neural network to classify the THz time-domain signals collected from a QFRP curved structure for automatic characterization of pre-embedded delamination defects has been reported. A THz-TDS system combined with a collaborative robot for collecting the THz signals from QFRP curved structure has been built. An automatic characterization method framework is developed. Results show that the precision rates of Transformer-based neural network for 1st delamination to 5th delamination are 1.0, 1.0, 1.0, 0.985, 1.0, and F1 score of it is 0.982. During the process of testing, delamination defects inside the QFRP curved structure were visualized using pixels with different colors. Results indicate that the Transformer-based neural network can characterize all pre-embedded delamination defects while minimizing false identification of non-defective areas, performing outstanding generalization.