In the continuous-variable quantum key distribution (CV-QKD) system with a local local oscillator (LLO), the center frequency of the sender and the receiver’s source are not exactly the same and a certain frequency drift exists over time, resulting in the frequency of the signal received near the intermediate frequency. Therefore, the LLO system needs an intermediate-frequency balanced homodyne detector (BHD), which needs better symmetry of the arms of the BHD, to obtain the less-common mode noise. Moreover, the traditional intermediate-frequency receiver in classical communication is not available in the CV-QKD system because of the low quantum-to-classical noise ratio. In view of this, in this paper, we construct a broadband intermediate-frequency BHD based on ratio frequency and integrated circuit technology, whose bandwidth can exceed 270 MHz and whose quantum-to-classical noise ratio can reach 14.9 dB. Meanwhile, the BHD has an excellent linear performance with a gain of 22.4 k. By adopting our intermediate-frequency BHD, the secret key rate of the pilot-sequential Gaussian modulated coherent state CV-QKD system with an LLO can reach over 430.8 kbps of 60 km at the standard fiber length, which paves the way to achieve a high-performance LLO CV-QKD system with intermediate-frequency BHD.