Hydrogen permeation reduces a material’s properties and increases the risk of brittle fracture, which causes a potential safety hazard. A workpiece’s hydrogen permeation resistance could be improved by improving its surface integrity through surface processing. This paper studies low-alloy steel’s surface integrity and its hydrogen permeation resistance in a hydrogen production reactor, using the electrochemical cathodic hydrogen-charging method to carry out electrochemical hydrogen-charging experiments. After the specimens were pretreated using different surface-grinding methods and shot peening pressure strengthening, they were hydrogen-charged on a self-designed and built electrolytic hydrogen charging platform. Before and after hydrogen charging, the specimens’ section hardness and tensile strength were tested, and the fracture morphology of the specimens was observed. The influence laws of surface roughness and surface residual compressive stress on the distribution of material hardness along the depth, the variation in material hardness, the fracture morphology, and the decline in the tensile properties of the low-alloy steel specimens after 5 h of hydrogen charging were analyzed. The reasons for the influence of surface integrity indexes on the hydrogen permeation resistance of the specimens were also analyzed. Based on the experimental results, a series of mechanical processing parameters were proposed to improve the material’s permeation resistance, which provides a theoretical and practical basis for the processing of materials with high surface integrity and hydrogen permeation resistance. Through the experiments, it was found that the hydrogen permeation resistance of the Ra 0.17 μm surface roughness specimen was the best of all specimens with different surface roughness values, and its hydrogen embrittlement sensitivity index was 20.96%. The specimen had the best hydrogen permeation resistance under 336 MPa surface residual compress stress, and its hydrogen embrittlement sensitivity index was 16.45%.