The phenomenon of pulse shortening caused by asymmetrical mode competition is observed in the <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\textit{X}$</tex-math> </inline-formula> -band relativistic klystron oscillator (RKO). Based on theoretical and simulation analysis of the electronic conductivity, external quality factor, and resonance characteristics of the TM <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$_{\text{013}}$</tex-math> </inline-formula> mode and asymmetric TM <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$_{\text{512}}$</tex-math> </inline-formula> mode, the reasons for the excitation of the asymmetric mode are obtained. First, the asymmetric TM <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$_{\text{512}}$</tex-math> </inline-formula> mode has negative electronic conductors and great external quality factors in the buncher, which provides a starting condition. Second, asymmetric TM <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$_{\text{512}}$</tex-math> </inline-formula> mode has higher resonance in the extraction structure to provide growth conditions. Therefore, strong positive feedback establishes between the buncher and the extraction structure, which makes the asymmetrical TM <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$_{\text{512}}$</tex-math> </inline-formula> mode effectively excited. Finally, by optimizing the radius of the extraction structure, the resonance characteristics of the TM <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$_{\text{013}}$</tex-math> </inline-formula> mode are enhanced, while the asymmetrical mode is weakened. So the TM <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$_{\text{013}}$</tex-math> </inline-formula> mode establishes stronger positive feedback between the buncher and the extraction structure, effectively suppressing the excitation of the asymmetric mode. The simulation and experimental results show that this method can effectively suppress the excitation of the asymmetrical mode, and solve the problem of pulse shortening.