To ensure the stability of load voltage, a bi-directional DC-DC converter is usually required in the power buffer application of the ultra-capacitor energy storage system. In this paper, non-isolated Buck-Boost bi-directional DC-DC converter is analyzed to improve the terminal voltage performances. In the light of the requirements of constant terminal voltage when positive and negative bi-directional power flows, the optimal design of power circuit and control parameters increases the operation stability of the whole ultra-capacitor energy storage system. Furthermore, it decreases the fluctuation of load voltage and raises the utilization rate of ultra-capacitors’ energy reserve.