Intelligent reflecting surfaces (IRSs) enhance the robustness of wireless transmissions against shielding. However, addressing channel-estimation overheads caused by the large number of IRS elements remains a major challenge. In particular, these overheads increase proportionately with an increase in the number of IRS elements, thereby making it difficult to handle multiple IRSs. Therefore, we investigate a passive beamforming method with partial channel state information (CSI) in distributed IRS communication systems called selective reflection control (SRC). Specifically, the proposed method determines the associations between the IRS and user equipment (UE) to obtain the CSI of the associated IRS-UE links. Simulations were conducted, and the obtained results confirm that the proposed method based on SRC outperforms the benchmark method in terms of the user sum rate. Furthermore, the proposed method realizes robust transmissions against shielding owing to appropriate IRS–UE associations. Therefore, the proposed SRC achieves better communication performance with less channel estimation overheads, which is expected to accelerate large-scale IRS deployment.