Heat exchangers play a crucial role in industrial processes by facilitating the efficient transfer of thermal energy. The thermophysical properties of nanofluids, which consist of nanoparticles dispersed in conventional heat transfer fluids, are enhanced. Their remarkable thermal conductivity and convective heat transfer characteristics have the potential to significantly improve heat transfer processes. In the present investigation, a numerical study has been carried out to investigate the effect of different nanofluids on heat transfer from outer surface of a spiral coil based on the first and second laws of thermodynamics. The numerical solutions are checked against previous experimental and numerical research in the field. Nusselt number, Euler number, exergy efficiency, irreversibility, and equipment cost are studied with respect to concentration, mass flow ratio (Rm), and shell and tube inlet temperature for a variety of nanofluids, including Al2O3/H2O, CuO/H2O, Fe2O3/H2O, and TiO2/H2O. Compared to water, the exergy efficiency is lower while using nanofluids, as shown by the results, especially in higher nanofluid concentrations, and among the different nanofluids, Al2O3–H2O nanofluid was found to have the highest thermodynamics second law efficiency. The exergy destruction rate experiences linear growth by increasing the nanofluid concentration. A higher pressure drop based on the Euler number is expected for nanofluids. In addition, the consumption of nanofluids on the shell side results in a considerable reduction of the purchased equipment cost (PEC) value compared to water. As opposed to the tube-side input temperature, variations in the shell-side inlet temperature found to have a greater impact on the heat exchanger's exergetic performance.