In this article, a novel inverter nonlinearity compensation method is proposed for open-winding (OW) machines fed by isolated dc-bus dual two-level three-phase inverters with dual switching modes. Inverter nonlinearity of dual inverters is analyzed in both balanced and unbalanced switching modes, and the different voltage distortions are utilized for a closed-loop inverter nonlinearity compensation by alternately applying the two switching modes. The proposed strategy needs no off-line measurement for the parameters of dual inverters and the electrical machine, and can also deal with the variation of inverter parameters in different operation conditions. Furthermore, the proposed strategy can overcome issues in conventional current harmonic-based methods, such as the need for accurate rotor position and preset compensation voltage curves. Experimental results verify the performance of the proposed inverter nonlinearity compensation method.