To enhance the performance of infill walls and reduce seismic damage, this paper proposes a novel prefabricated reinforced concrete (PRC) energy-dissipating wall, forming a new recoverable energy-dissipating PRC (ED-PRC) structural system. The system features pre-set gaps on both sides and the top of the PRC wall, with flexible materials filling the gaps on the sides. The top of the PRC wall is connected to the beam through several double-conical mild steel dampers to ensure the efficient transfer of horizontal shear forces between the main frame and the PRC wall. A numerical study was employed to investigate the seismic performance and the staged yield capacity. The results show that this design achieves a yielding sequence of dampers → wall → main frame. Furthermore, during the early to mid-phases of the cyclic loading simulations, the double-conical mild steel dampers with the low yield point utilized in the ED-PRC structural system exhibited exceptional energy dissipation capabilities. Notably, the LY100 dampers accounted for up to 61.84% of the total energy dissipation, with the LY160 and LY225 dampers contributing 55.35% and 50.25%, respectively. It indicates that the proposed ED-PRC structural system significantly enhances the ductility and the energy dissipation capacity under seismic loading while substantially reducing damage to the primary structure. The use of prefabricated components facilitates modular construction, allowing for quick dismantling and replacement after an earthquake, thereby rapidly restoring the structural seismic resilience.