Abstract This work focuses on in situ residual unbalance estimation of the dual rotor system with the implementation of active magnetic bearing (AMB) as a controller and exciter. The excessive vibration generated due to the presence of residual unbalances limits the operating speed of the system. The compact structure of the dual rotor system provides constraints to the conventional balancing procedure that requires manual addition of the trial unbalances for balancing. In order to overcome the difficulty in balancing of the dual rotor system, an identification algorithm based on the modified influence coefficient method (MICM) is developed for the simultaneous estimation of residual unbalances in both inner and outer rotors with the generation of virtual trial unbalances as magnetic excitation through AMB. The controlling action of AMB attenuates the vibrational response of the system within the required limit and allows the safe operation of the system in the presence of rotor faults and additional excitations. The vibrational responses of the system at the limited locations and the magnitude and phase of the virtual trial unbalances are only required in the MICM for the estimation of unbalances. To numerically illustrate the present methodology, the displacement response is obtained from the developed finite element model of the dual rotor system with discrete disk unbalances and randomly distributed shaft. The robustness of the algorithm in the estimation of residual unbalances is verified with the addition of a different percentage of measurement noises. After balancing, the dual rotor system is found to traverse its critical speed with a less vibrational response.