Recycled aggregate concrete (RAC) is widely recognized as a promising approach for recycling construction and demolition waste. However, its practical application remains limited. A contributing factor to this limitation is the incomplete understanding of RAC’s durability characteristics, particularly its permeability, which is closely tied to the transport of harmful ions within the concrete matrix. Despite the critical importance of this issue, research in this area is still relatively sparse. This knowledge gap has motivated the current study, which seeks to thoroughly investigate the water permeability properties of RAC. In this comprehensive study, 84 specimens were fabricated for permeability testing. This study explores the influence of various factors, including the sources of coarse and fine recycled aggregates (RAs), the RA replacement ratios, and the water-to-binder ratio. The results indicate that the inclusion of both coarse and fine RAs reduces the impermeability of concrete. However, enhancing the quality of these RAs—specifically by increasing the compressive strength of the source concrete—can mitigate the reduction in impermeability. A statistical relationship is established between the average and maximum water permeation depths measured during permeability testing, leading to the formulation of a correlation between the permeability coefficient and the impermeability grade of RAC. Furthermore, mercury intrusion porosimetry testing was conducted to quantitatively analyze the pore structure of the mortar in RAC, providing a microscopic perspective that explains the macroscopic permeability behavior observed. Finally, a predictive model for estimating the permeability coefficient of RAC is proposed, demonstrating a high level of accuracy.