High voltage electric pulse rock breaking technology is a novel and promising rock breaking technique that is environmentally friendly, efficient, and nearing industrial application. However, the drilling performance of electrode bits under varied drilling muds and the mechanism of high voltage electric pulse rock breaking remain unclear. This study entails real drilling experiments conducted on red sandstone using electrical pulse drilling (EPD) under different initial voltage peaks, electrode bits, and drilling muds to investigate the impact and mechanism of EPD. A laboratory experimental system for high voltage pulse rock breaking is established based on a 0–80 kV EPD pulse generator. Three types of electrode bits are devised, namely, petal-structural, four-point-structural, and circular-ring-structural electrode bits. The EPD experiments on red sandstone are executed under seven different drilling muds at initial peak voltages ranging from 13 to 17 kV. And the efficiency of EPD rock breaking is assessed using five quantitative indexes: rate of penetration, breaking volume, specific energy of rock breaking, and the proportion of electrical fragmentation and hydraulic-electric rock fragmentation. Furthermore, the physical parameters, i.e., density, conductivity, Zeta potential, PH, and apparent viscosity, of the seven drilling muds are measured, shedding light on the rock breaking mechanism of EPD under specific drilling mud conditions. Subsequently, the designed electrode bit is evaluated, and based on experimental phenomenon, structural optimizations of the electrode bit and EPD drilling recommendations under various drilling mud conditions in real drilling scenarios are proposed. This research is poised to offer valuable insights into the design of EPD electrode bits and the selection of drilling mud for enhancing drilling efficiency.