The surfaces of objects in the nature can modulate the incident light, and then the scattered light carries the material and microstructure information. Understanding the modulation effect is helpful to extract the target features. In this paper, a large number of randomly rough surfaces were generated to simulate the target surface by frequency domain filtering, and the statistical stability of Mueller matrix elements of rough surface was obtained by Monte Carlo method. The results showed that for the rough surfaces of metal and dielectric, there were 6 elements of Mueller matrix that were completely different. These 6 elements determined the different modulation effects of the two rough surfaces on different types of linearly polarized light. The Stokes vector V components of scattered light on metal and dielectric rough surfaces were significantly different. In the incident plane, V had the maximum value when 45°-polarized light was incident on the metal rough surface. When p-, s-polarized light was incident on the metal rough surface respectively, V had an antisymmetric peak distribution outside of the incident plan. When linearly polarized light with different polarization angles was incident on the dielectric rough surface, V was always 0. By irradiating the rough surface with 45°-linearly polarized light, the rough surfaces of metal and dielectric could be distinguished by whether V had a peak distribution or not. The above conclusions could provide a new basis for metal target recognition in laser detection system.