This study addresses the escalating demand for clean water resources driven by population growth and water quality deterioration. The research focuses on evaluating the efficacy of a nanocomposite material, incorporating zero valent iron nanoparticles into a chelating cation exchange resin matrix, for selectively removing heavy metals from polluted aquatic environments. The selected resin, featuring iminodiacetic acid functional groups, demonstrates notable selectivity for heavy metal cations over alkali earth metals. Column experiments were conducted to assess the nanocomposite’s performance, utilizing a feed solution spiked with heavy metals at concentrations ten times higher than Greek legislation limits for wastewater effluent recycling. The nanocomposite exhibited significant effectiveness for Cu, Cr(VI), and Pb, consistently maintaining Cu levels below detection limits and demonstrating limited breakthrough of Cr(VI) and Pb depending on experimental conditions. However, the removal efficiency was lower for Ni and insufficient for Cd, Zn, and As in this complex multicomponent solution. This research contributes valuable insights into the potential application of the developed nanocomposite for targeted removal of specific heavy metals in contaminated water sources, providing a foundation for further exploration and application in water remediation technologies.