An aero engine, as the core power equipment of the aircraft, enables safe and stable operation with a very high reliability index, and is an important guarantee in flight. The open rotor turbine engines (contra-rotating propeller) have stood out as a research hotspot for aviation power equipment in recent years due to their outstanding advantages of low fuel consumption, high airspeed, and strong propulsion efficiency. Aiming at the problems of vibration exceeding the standard generated by imbalance during the operation of the dual-rotor system of aircraft development, the difficulty of identifying the coupled vibration under the micro-differential speed condition, and the complexity of the dynamic characteristic law, a kind of numerical simulation of the dynamics based on the finite element technology is proposed, together with an experimental research method for the fast and accurate identification of the coupled vibration of the dual-rotor system. Based on the existing open rotor engine structure design to build a simulation test bed, establish a double rotor finite element simulation digital twin model, and analyze and calculate the typical working conditions of the dynamic characteristics of parameters. The advanced algorithm of double rotor coupling vibration signal identification is utilized to carry out decoupling and dynamic balancing experimental tests, comparing the simulation results with the measured data to verify the accuracy of the technical means. The results of the study show that the vibration suppression rate of the finite element calculation simulation test carried out for the simulated double rotor is 98%, and the average vibration reduction ratio of the actual field test at 850 rpm, 1000 rpm, and 3000 rpm is over 50%, which achieves a good dynamic balancing effect, and has the merit of practical engineering application.