Online condition monitoring of winding insulation has been proved to be an effective means to improve the reliability of an inverter-fed machine, but little work has been done on the insulation of line-end coil subjected to much higher transient voltage stress with a higher risk of premature insulation break down. In this article, the inverter self-excited high-frequency switching oscillations are utilized for the line-end coil insulation condition monitoring. To improve the condition monitoring sensitivity and its robustness to dynamic operating conditions, a switching oscillation mode decomposition method is proposed. First, the condition-sensitive mode, i.e., high-frequency common-mode (HFCM) current is selected and separated by the variation mode decomposition algorithm among three dominant modes in switching oscillation current. Then, the amplitude-frequency features of HFCM current are extracted for condition monitoring. Finally, the overall insulation condition of three-phase line-end coils and the degree of asymmetric degradation are quantitatively evaluated by the frequency and amplitude features respectively. Experimental work was carried out and the results demonstrate the good performance of high sensitivity, good robustness, and non-contract safety.