Decades of research in hypersonic science and engineering have shown that scramjet engines are the preferred choice for powering hypersonic vehicles. In these engines, oblique shock combustion and rotating detonation combustion are two common combustion modes. This article primarily focuses on the numerical simulation and theoretical analysis of these two combustion modes. The research indicates that as the combustible mixture injected into the combustion chamber increases, the combustion mode transitions from deflagration combustion to self-sustaining detonation combustion and then to forced detonation combustion to maintain stable combustion in the chamber. Under self-sustaining detonation combustion mode, as the amount of combustible mixture injected into the chamber increases, the number of detonation waves gradually increases from one to multiple and the angle between the detonation waves and the inflow changes from acute to right angles. In the forced detonation combustion mode, the minimum shock wave intensity required to initiate detonation combustion can be obtained by drawing a tangent from the Rayleigh line. The structure generating oblique shock waves in the combustion chamber needs to match the shock wave intensity. In the detonation combustion mode, both the exit total temperature and specific impulse of the combustion chamber increase with the increase in Mach number at the inlet, while the pressure ratio decreases. This article provides a reference for the design of the combustion chamber in supersonic combustion ramjet engines.