An investigation is performed through experiments of a laboratory-scale optically accessible combustor equipped with a single-element shear coaxial injector. Various optical diagnostics approaches, such as hydroxyl (OH) planar laser-induced fluorescence (OH-PLIF), high-speed imaging and infrared imaging, are simultaneously used to determine the OH radical spatial distribution, flame instantaneous fluctuations, and temperature fields outside the combustor nozzle. This study emphasizes the combustion characteristics and flame stabilizations of two propellant combinations under the same thrust level, including gaseous hydrogen/gaseous oxygen (GH2/GO2) and gaseous methane/gaseous oxygen (GCH4/GO2). Moreover, to validate a pressure-scaling criterion, different pressure conditions are also considered. Based on the results of the high-speed and OH-PLIF images, combustion similarities are evident under different chamber pressures for each propellant couple. The two kinds of flame are seen to be stabilized in the vicinity of the oxidizer injector lip but the anchor region in the GCH4/GH2 cases is found to change dynamically near the lip. The lift-off GCH4/GO2 flame exhibits evident unsteady combustion phenomenon in the combustor. Experimental data indicates that GCH4/GO2 flame stabilization is hard to achieve compared to the GH2/GO2 flame. Furthermore, the GCH4/GO2 flame has a greater heat releasing intensity and gentler reaction zone distribution. This indicates that relative to the GH2/GO2 flame, the GCH4/GO2 flame causes major issues with regard to heat protection of the inner wall in the combustor. Meanwhile, according to the infrared images, the GCH4/GO2 plume has a lower outlet core temperature and larger temperature gradient, which is an advantage for employment in future space engines. This study, identifying preliminary behavior differences between the GH2/GO2 and GCH4/GO2 flames, will be applicable for future engine design.