Quartz fiber–reinforced polyimide composites (QFRP) had heterogeneous and anisotropic properties. Because of the gradient change of cutting force at hole exit, some fibers could not be effectively chipping breaking and the defect of delamination burr was often inevitable. In this paper, a cutting force model of fiber fracture stripping considering thermal–mechanical coupling was constructed based on the mechanism of cutting force and thermal action. A series of milling hole tests at different temperatures (from 290 to 80 K) was carried out using liquid nitrogen inner-cooling equipment. Similarly, the hole exit morphology, milling force, and cutting temperature were investigated and the delamination factors were calculated in detail. The results show that the model can explain the reason of hole exit defect, which is attributed to the lack of tangential cutting force and the large gradient change of axial force considering the high cutting temperature. Meanwhile, cryogenic cooling can reduce the effect of cutting temperature for hole exit defect. Although the cutting force is increased, the gradient degree of axial force can be reduced, and the delamination factor can be decreased from 1.08 at low-speed dry cutting to 1.01 of high-speed cryogenic one. At the same time, the increase of tangential force and the decrease of fiber ductility improve the chip breaking ability and the processing efficiency. Conclusion: the spiral hole milling process with cryogenic medium intervention can inhibit the milling hole delamination defects.