A novel double shell-passes multi-layer helically coiled tubes heat exchanger (DSMHCTHE) is proposed to improve the flow and heat transfer performance of the shell side. It was studied by using conjugate heat transfer numerical model. By changing coil diameter and pitch, the influence of coil torsion on the performance of the heat exchangers was investigated in the range of 0.0637–0.3183, and compared with the conventional multi-layer helically coiled tubes heat exchangers (MHCTHE). Two kinds of heat exchangers were optimized by using multi-objective optimization differential evolution algorithm. The results indicate that the coil pitch has a greater effect on the overall heat transfer coefficient, and the effects of coil pitch variations in the inner and outer helically coiled tubes follow different patterns. Compared with MHCTHE, the heat transfer rate of DSMHCTHE are increased by 13.9 %–19.1 %, the comprehensive performance is increased by 13.0 %–18.0 %, and the heat transfer entropy generation number and friction entropy generation number are lower. It shows that DSMHCTHE have better application potential as heat exchangers.