Surfaces roughness exists inevitably during manufacturing process, while how it affects the performance of the near-field thermophotovoltaic (NF-TPV) system is still unclear. In this work, we investigate the effect of rough surfaces on the performance of NF-TPV systems theoretically. Two typical NF-TPV systems with the same plasmonic emitter (Ga-doped ZnO, GZO) and different TPV cells (GaSb and InAs) are considered. The effective multilayer approach and analytical approximation model are applied to analyze near-field radiative heat transfer and the performance of the NF-TPV system. Near-field radiative heat transfer between the ideal GZO and TPV cell with smooth surfaces is through the TR-SPPs modes (interaction between the evanescent waves generated by total reflection and the surface plasmon polaritons), while the surface roughness excites the red-shift and the coupling of the TR-SPPs modes. The influence of surface roughness on the performance of the NF-TPV system depends on the cooperation between the resonance frequency of the emitter and the bandgap of TPV cell. Generally, roughness increases the power output for a given average emitter-cell distance, while the efficiency can be improved or reduced. This work will help the understanding and evaluation of the performance of NF-TPV systems with rough surfaces.