Constructing van der Waals materials with spontaneous out-of-plane polarization through interlayer engineering expands the family of two-dimensional ferroelectrics and provides an excellent platform for enhancing the photoelectric conversion efficiency. Here, we reveal the effect of spontaneous polarization on ultrafast carrier dynamics in rhombohedral stacked bilayer WSe2. Using precise stacking techniques, a 3R WSe2-based vertical heterojunction was successfully constructed and confirmed by polarization-resolved second harmonic generation measurements. Through output characteristics and the scanning photocurrent map under zero bias, we reveal a non-zero short-circuit current in the graphene/3R WSe2/graphene heterojunction region, demonstrating the bulk photovoltaic effect. Furthermore, the out-of-plane polarization enables the 3R WSe2 heterojunction region to achieve an ultrafast intrinsic photoresponse time of approximately 3 ps. The ultrafast response time remains consistent across varying detection powers, demonstrating environmental stability and highlighting the potential in optoelectronic applications. Our study presents an effective strategy for enhancing the response time of photodetectors.