This study aimed to investigate the separation performance of a cyclone separator after reshaping its cylindrical body by installing the helical triangular fins. A numerical simulation based on Fluent was adopted to perform an orthogonal test to optimise the structure of the cyclone separator with helical triangular fins. Three structural parameters of the helical triangular fins were selected as optimisation variables: base width, fin size, and fin pitch, and their influences on the evaluation indices of the cut-off diameter were investigated. The optimal combination scheme was determined by range analysis, and the cyclone separator performances before and after optimisation were compared and analysed. The significant influence of the structural parameters on the cut-off diameter was in descending order as the fin pitch, fin size, and base width. For particles with diameter of 0.1, 0.5, 1, 2, and 3 μm, the separation efficiency of the cyclone separator with optimized helical triangular fins increased by 7.4 %, 15.9 %, 20.1 %, 10.9 % and 14.8 % respectively. Moreover, the cut-off diameter of the finned cyclone separator is reduced by 30.7 %, while the pressure drop is only increased by 6.6 %. The short circuit flow and back-mixing were alleviated, thereby considerably enhancing the stability of the flow field. Therefore, the finned cyclone separator was found to play a critical role in increasing the separation of fine particulate matter.