AbstractIn this paper, carboxyl‐terminated nitrile rubber (CTBN) and Nano cerium oxide (n‐CeO2) were used to toughen epoxy resin (EP), and the effects of toughening agents on the mechanical and thermal properties of EP were studied. Orthogonal experiments determined that the ideal toughener proportions are 20 wt% CTBN and 1 wt% n‐CeO2, under which the resin demonstrated superior mechanical characteristics. The combined application of CTBN and n‐CeO2 significantly enhanced the resin's mechanical properties, surpassing the effects of individual particle toughening. Compared with pure EP, the impact, flexural, and tensile strengths of the resin‐cured product synergistically toughened by CTBN and n‐CeO2 increased by 462%, 136%, and 110%, respectively, and the flexural and Young's moduli increased by 38% and 35%, respectively. Differential scanning calorimetry (DSC), thermogravimetric analysis (TGA), and scanning electron microscopy (SEM) were used to characterize the cured resin. The results indicate that incorporating CTBN and n‐CeO2 reduces the reaction activation energy, thereby accelerating the curing reaction and facilitating its progression. The addition of nanoparticles enhances the maximum weight loss temperature of the cured resin, thereby improving its thermal stability. CTBN and n‐CeO2 particles create a uniform phase separation structure within the resin, enhancing its toughness and strength. Furthermore, the impact fracture mode of the cured resin transitions from brittle to tough fracture, demonstrating a significant toughening effect. This provides an effective method for the toughening modification of EP.Highlights The synergistic use of CTBN and n‐CeO2 enhances the toughness of EP. The incorporation of nanoparticles improves the thermal stability of the cured resin. The addition of CTBN and n‐CeO2 accelerates the curing reaction rate. The inclusion of toughening agents transforms the fracture mode of the cured resin from brittle to tough fracture.