High-efficiency thermal energy systems are very important in meeting the growing demand for thermal energy. As a result, several heat transfer improvements have been proposed. Some promising methods include flow heat transfer in shell and spiral tube exchangers.In a shell-and-coil heat exchanger, utilizing a meticulously designed coil instead of a basic one significantly boosts heat transfer and overall thermal efficiency. This is due to the enhanced fluid dynamics and increased turbulence facilitated by the advanced coil design, making it ideal for space-constrained applications. Moreover, the helical configuration helps minimize fouling and maintenance, and may also provide self-cleaning benefits. Consequently, helical coils are highly regarded in industrial contexts for their superior performance, maintenance ease, and design adaptability.This study conducts a numerical evaluation of the heat transfer and fluid flow properties of two distinct shell-and-coil heat exchangers with specialized designs. The fluids analyzed include water-based hybrid nanofluids, specifically Water/MgO-TiO2and Ag-HEG/water, with results compared to those obtained using pure water. The investigation spans Reynolds numbers from 500 to 2000 and is divided into two segments.The first segment examines the influence of spiral coil geometry and fluid type on the heat exchanger's endothermic performance, utilizing nanoparticle volume concentrations of φ1 = φ2 = 0.3. In the second segment, the optimal geometric and fluid model is chosen based on the findings from the first part. Following this, the impact of various hybrid nanofluids on thermal performance is assessed, comparing fluids with volume concentrations of φ1 = φ2 = 0.3 to pure water (φ1 = φ2 = 0).The findings reveal that Case [A], featuring a unique geometry with Water/Ag_HEG, achieves the highest thermal performance across all examined Reynolds numbers. At the lowest Reynolds number, the thermal efficiency improvements for Case [A], Case [B], and Case [C] were 137 %, 113 %, and 56 %, respectively, compared to the baseline. Additionally, the second part of the study demonstrates that at the lowest Reynolds number, the thermal efficiencies of Water/MgO-TiO2 and Water/Ag_HEG nanohybrid fluids increased by 76 % and 49 %, respectively.