This paper proposes an improved PGC-Arctan demodulation algorithm for detecting partial discharge signals in transformers. Compared with the traditional PGC-Arctan algorithm, the improved algorithm extracts carrier frequency difference and carrier phase delay by fundamental mixing and using low-pass filtering on the interference signal, and compensates the phase modulation depth by high order frequency, reducing THD and enhancing SNR. The algorithm is theoretically analyzed, simulated and verified through experiments. In the 200 kHz acoustic emission signal detection experiment, the SNR and THD of the improved algorithm are 49.63 dB and 1.34%, respectively, which is 13.59 dB higher than the SNR of the traditional algorithm, and the THD is reduced by 76.61%, and the harmonic distortion is well suppressed. The system is used to detect weak ultrasonic signals generated during partial discharge of transformers and compare them with PZT. The results show that the system has the potential to be used to monitor the partial discharge signal in transformers.