The phenomena of combustion instabilities beyond limit cycles caused by the nonlinear interaction of various instability modes have recently received much attention. This paper numerically studied the mode competition and cooperation phenomena in the liquid rocket engines when combustion instabilities occur. The liquid rocket engine with the single-element structure is the focus of the present work, and the numerical model is developed employing the Large Eddy Simulation technology, with the complete conservation Navier-Stokes equations taken into account. The results first reveal the relationship between the acoustic mode and the intrinsic vorticity mode: the competition and cooperation between the two will result in the complex dynamic process, such as semi-stable oscillation and intermittency. What's more, a more detailed coupling process between intrinsic vortex shedding and acoustic oscillation is analyzed, indicating existence of four stability regimes in the liquid rocket engines and an important insight into combustion instability in non-premixed dump combustors is provided.