The cantilever dual-rotor system is a typical structure of the blade output end of the open rotor engine and the coaxial output turboshaft engine. The excessive unbalanced vibration of a coupled cantilever dual-rotor system is one of the main factors limiting the application of the above engine type. In order to accurately describe the vibration coupling effect between the dual-rotor-intermediate bearings, the unbalanced response of the cantilever dual-rotor system is analyzed, and the self-sensitivity coefficient is proposed to guide the selection of measuring points and vibration suppression experiments for the dual-rotor system. On this basis, a new online automatic balance actuator applicable to this dual-rotor system is designed, and a feasibility experiment is carried out. The experimental results indicate that: (1) The self-sensitivity coefficient can be used as the basis for the actual vibration measuring point arrangement and unbalanced vibration suppression strategy of the dual-rotor system, and the proposed step-by-step vibration suppression strategy can reduce the vibration of the dual-rotor system by more than 80%. (2) The designed online automatic balance actuator can reduce the unbalanced vibration by 53% in 3.52 s. The proposed method in this study can provide guidance for the vibration suppression of the dual-rotor system.