The electrohydraulic effect induced by underwater arc discharge is an efficient way to generate controllable, high-intensity shock waves. However, the development process of underwater arc discharge involves the complex coupling of plasma arc, gas bubble, and liquid medium, of which the evolution mechanism is not well understood. In this paper, the underwater arc discharge process at a millisecond pulse (>50 ms) was investigated by high-speed shadow imaging and colorimetric temperature measurement, and a simulation model of bubble pulsation was proposed to quantitatively estimate the state variation and energy transfer of the gas bubble. The results indicate that the whole arc discharge process can be categorized into three successive stages: short-period oscillation, long-period oscillation, and quasi-steady state. The vapor inside the bubble can reach a supercritical state (827 K and 140 MPa) at the minimum bubble radius. The simulation shows that the light radiation absorption and the heat conduct loss through metal electrodes are the two dominant factors influencing the pulsation of the bubble, and further analysis indicates that the dynamic evolution of the arc determines the bubble pulsation mode. Our findings demonstrate why and how repetitive electrohydraulic shock waves can be generated by a single long pulse underwater arc discharge, providing a low-cost way of shock wave generator based on an AC/DC high-voltage power source.