Annotated datasets play a significant role in developing advanced Artificial Intelligence (AI) models that can detect bridge structure defects autonomously. Most defect datasets contain visual images of surface defects; however, subsurface defect data such as delamination which are critical for effective bridge deck evaluations are typically rare or limited to laboratory specimens. Three Non-Destructive Evaluation (NDE) methods (Infrared Thermography (IRT), Impact Echo (IE), and Ground Penetrating Radar (GPR)) were used for concrete delamination detection and reinforcement corrosion detection. The authors have developed a unique NDE dataset, Structural Defect Network 2021 (SDNET2021), which consists of IRT, IE, and GPR data collected from five in-service reinforced concrete bridge decks. A delamination survey map locating the areas, extent and classes of delamination served as the ground truth for annotating IRT, IE and GPR field tests’ data in this study. The IRT were processed to create an ortho-mosaic maps for each deck and were aligned with the ground truth maps using image registration, affine transformation, image binarization, morphological operations, connected components and region props techniques to execute a semi-automatic pixel–wise annotation. Conventional methods such as Fast Fourier transform (FFT)/peak frequency and B-Scan were used for preliminary analysis for the IE and GPR signal data respectively. The quality of NDE data was verified using conventional Image Quality Assessment (IQA) techniques. SDNET2021 dataset consists of 557 delaminated and 1379 sound IE signals, 214,943 delaminated and 448,159 sound GPR signals, and about 1,718,083 delaminated and 2,862,597 sound IRT pixels. SDNET2021 addresses one of the major gaps in benchmarking, developing, training, and testing advanced deep learning models for concrete bridge evaluation by providing a publicly available annotated and validated NDE dataset.