Heat exchangers are integral parts of important industrial units such as petrochemicals, medicine and power plants. Due to the importance of systems energy consumption, different modifications have been applied on heat exchangers in terms of size and structure. In this study, a novel heat exchanger with helically grooved annulus shell and helically coiled tube was investigated by numerical simulation. Helically grooves with the same pitch of the helical coil tube and different depth are created on the inner and outer wall of annulus shell to improve the thermal performance of heat exchanger. In the first section, thermal performance of the shell and coil heat exchanger with the helical grooves on its outer shell wall was compared with same but without helical grooves. At the second section, helically grooves created on both outer and inner wall of the annulus shell with different groove depths. The results showed that the heat exchanger with grooves on both inner and outer shell wall has better thermal performance up to 20% compared to the heat exchanger with grooves on only outer shell wall. The highest thermal performance achieves at lower flow rates and higher groove depths whereas the pressure drop did not increase significantly.