In Laser Powder Bed Fusion (L-PBF), surface roughness is pivotal for controlling the mechanical and functional performances, as well as the geometrical accuracy of the final product. This study extensively investigated the interactions of hatch and contour processing parameters, along with contour offset distance, on vertical surface roughness for 316L stainless steel in L-PBF. Melt pool morphology and surface arithmetic average roughness (Sa) were quantified using confocal microscopy, while scanning electron microscopy was employed to interpret the detailed microstructure of surface features. Under low volumetric energy density (VED) hatch conditions (e.g., 66.7 J/mm3), varying the contour offset distance has a negatable effect on the surface roughness when the contour VED is lower than 121.6 J/mm3, remaining relatively smooth surfaces dominated by bare melt tracks with sparely attached partially melted particles. Increasing the hatch or contour VED (e.g., 166.7 J/mm3), dross formation, identified by the microstructural differences and explained by the melt pool instability and migration, is inevitable, which dictates the surface roughness with higher Sa values. The larger contour offset distance further promotes the dross occurrence with irregularity and increases Sa. Employing a low contour VED with an appropriate offset distance and adopting the contour-first scan strategy was demonstrated as an effective solution to reduce the dross formation. Through the analysis of melt pool behavior, surface topography, and microstructure, this study elucidates the mechanisms governing dominant surface characteristics under the combined influence of hatch and contour parameters. It lays the foundation for precise control of surface roughness without altering hatching parameters, enabling the tailored manipulation of performance in additively manufactured structures.