S136 (STAVAX ESR) mold steel is an advanced material for the optical mold industry. The precisely machined surface quality is essential for guaranteeing the excellent performance of molds. Diamond cutting is a high-efficiency machining technique in optical mold manufacturing. However, the chemical wear of diamond tool is a huge challenge in precisely machining steel mold due to the reaction between atoms C and Fe. In this study, the hybrid machining technique which combined the in-situ laser assisted diamond cutting (LADC) and plasma nitriding was used to machine S136 mold steel. Furthermore, the impact of the various machining conditions on the machined workpiece surface roughness Sa was comprehensively investigated by the Taguchi method (TM) and Response surface methodology (RSM). The experimental results of material characterization indicated that a 90 μm nitride layer was generated on the workpiece surface by plasma nitriding. Analysis of variance (ANOVA) results demonstrated that the feed rate and laser power had an important impact on the machined surface roughness Sa. According to the RSM, a precise mathematical model was developed which is explored by experimental investigations. The results showed that the optimal machining parameter combination by RSM generated the better surface quality of 6.393 nm in in-situ LADC of nitriding steel. This study provides a reference for mold steel ultra-precision machining technique and parameter optimization method for surface integrity.