There are ongoing research challenges for the addition of the blend of PET and crumb rubber in polymer-modified concretes, which aims to leverage the benefits of both materials. In this study, various percentage combinations of waste aggregates, such as PET and crumb rubber, were used to replace coarse and fine aggregates in natural rubber latex (NRL)-modified concrete. Engineering properties such as compressive and flexural strengths, modulus of elasticity, and toughness obtained from compressive- and flexural stress-strain curves were investigated. Scanning electron microscopy (SEM) analysis was performed to examine the microstructural properties and study the strength development of the studied concretes. The results revealed that the compressive and flexural strengths of NRL-modified concretes with PET and crumb rubber aggregate replacements decreased with increasing replacement ratios. SEM analysis indicated that PET and crumb rubber (hydrophobic and non-polar materials) can affect the microstructure of the studied concrete by creating a weak interface between the aggregate and cement pastes, leading to reduced strength development. With the addition of the NRL additive, the film formation was found to act as a bridge and improve the bond strength of aggregates and hydration products in NRL-modified concrete. Furthermore, the integration of PET and crumb rubber aggregate can enhance the ability of the concrete to absorb energy and improve ductility. It was found that 10% of PET and crumb rubber aggregate replacement can be used for NRL-modified concrete as a rigid pavement, as its mechanical strengths satisfy the requirements set by the Department of Highways (DOH) in Thailand. This research helps repurpose waste materials and reduce the environmental footprint of concrete production.