A disk-shaped rotating detonation engine with H2/air mixture was tested to identify the impact of combustor outlet geometry on the engine’s operating characteristics. Three combustor outlet diameters and five outlet lengths are employed in the experiments. Results show that with the increase of combustor convergent ratio, the propagation stability of the rotating detonation wave decreases, and the propagation velocity and pressure peak decrease slightly. When the convergent ratio increases to a certain value (1.70 in this study), a “platform zone” with a lower pressure value appears before the sharp rise of the dynamic pressure curve. The propagation mode varies with the increase of mass flow rate at different convergent ratios. As the mass flow rate increases, the wave head number in the combustor increases. But the change rule of propagation mode with mass flow rate is greatly affected by convergent ratio. Increasing the convergent ratio is conducive to the formation of multi-wave modes, and the critical mass flow rate for mode transition drops sharply. When the convergent ratio increases to 1.70, the unstable asymmetric dual-wave mode is obtained. With the increase in the convergent ratio, the engine’s operating range and operating stability decrease significantly. Finally, changing the combustor outlet length has little influence on the engine’s operating characteristics and detonation-wave parameters.