High-fidelity collapse process and reasonable debris accumulation and distribution after collapse are very critical to carry out post-disaster investigations and show why collapse happens. Moreover, previous researches on progressive collapse is more concerned with structural behavior in resisting collapse and pays little attention to the dynamic response after collapse criterion is reached. Therefore, a hybrid numerical framework is developed through combining the advantages of the finite element method (FEM) and the physics engine (PE), which provides new ideas for studying the full collapse process of structures. In this paper, based on previous work, such hybrid framework is further developed with efforts on element merging technique in the model mapping process from a FE model to a PE model and determination of connection constraints in the PE model. Then, a new Python-based interface program is developed to integrate the FE model using LS-DYNA and the PE model using Blender 3D. The accuracy of FEM-PE framework is verified by a collapse test of a flat plate structure, and an RC frame structure is designed for studying full process of progressive collapse. Results show that FEM-PE simulation can achieve accurate nonlinear structure behavior at small deformation and high-fidelity collapse at large deformation. To obtain reasonable debris accumulation, a single beam, column and slab should be meshed with the element number no more than 3, 2 and 3 × 3 in PE models, respectively. For a seismically designed RC frame structure, a single column removal scenario cannot initiate the collapse, but the simultaneous removal of a corner and a neighouring penultimate column generates the propagated collapse mode.