A compact setup for two-way single-photon-level frequency conversion between 852 nm and 1560 nm has been implemented with the same periodically-poled magnesium-oxide-doped lithium niobate (PPMgO:LN) bulk crystals for connecting cesium D2 line (852 nm) to telecom C-band. By single-pass mixing a strong continuous-wave pump laser at 1878 nm and the single-photon-level periodical signal pulses in a 50-mm-long PPMgO:LN bulk crystal, the conversion efficiency of ∼ 1.7% (∼ 1.9%) for 852-nm to 1560-nm down-conversion (1560-nm to 852-nm up-conversion) have been achieved. We analyzed noise photons induced by the strong pump laser beam, including the spontaneous Raman scattering (SRS) and the spontaneous parametric down-conversion (SPDC) photons, and the photons generated in the cascaded nonlinear processes. The signal-to-noise ratio (SNR) has been improved remarkably by using the narrow-band filters and changing polarization of the noise photons in the difference frequency generation (DFG) process. With further improvement of the conversion efficiency by employing PPMgO:LN waveguide, instead of bulk crystal, our study may provide the basics for cyclic photon conversion in quantum network.