Modified montmorillonite‐containing phytic acid (PA‐MMT) has been prepared by acid treatment and then introduced into unsaturated polyester resin (UPR) with an intumescent flame retardant (IFRs). The flame retardancy and thermal degradation of UPR/IFRs/PA‐MMT were evaluated by a limiting oxygen index (LOI) test, a vertical burning test (UL‐94), a thermogravimetric analysis (TGA), and a cone calorimeter test (CCT). Besides, the mechanical properties were studied by a universal testing machine. The LOI value of UPR/IFRs/PA‐MMT composites was increased to 29.2%. The CCT results indicated that the incorporation of PA‐MMT and IFRs significantly improved the combustion behavior of UPR. The results of the mechanical properties indicated that 1.5 wt% loading of PA‐MMT in UPR/IFRs showed the highest improvement in flexural strength and tensile strength. The flame‐retardant mechanism of PA‐MMT/IFRs was examined and discussed based on the results of combustion behavior and char analysis.