In recent years, in order to reduce the damage from the blackout accident, the demand of the power system for its basic resistance ability has gradually increased. Therefore, this paper proposes a double‐layer optimization method for the prevention of cascading failure of the power system that takes into account the transient stability constraint for cascading trips that directly lead to the accident. In terms of models, the paper reconstructs an outer model for optimizing the safety margin based on the inner model that solves the safety margin of cascading failure prevention. In terms of algorithms, the paper uses the Extended Particle Swarm Optimization (EPSO) algorithm with higher precision and accuracy to solve the inner model and innovatively introduced the Practical Dynamic Security Region (PDSR), which uses PSASP software for offline calculation and online application, as a dynamic analysis method to quickly realize transient stability constraint to solve the outer model. Three evaluation indexes of the preventive optimization effect can be obtained by this method; dispatchers can plan more flexible and reliable optimal dispatching schemes through these indexes, to comprehensively improve the safety margin of cascading failure prevention in the current operating state and enhance the risk resistance ability of the power grid. This paper has used MATLAB and PSASP to perform simulation analysis using the IEEE39 node system as a test example to verify the effectiveness and superiority of the proposed model and algorithm.