This study investigated combustion characteristics of composite fuel grains designed based on a modular fuel unit strategy. The modular fuel unit comprised a periodical helical structure with nine acrylonitrile–butadiene–styrene helical blades. A paraffin-based fuel was embedded between adjacent blades. Two modifications of the helical structure framework were researched. One mirrored the helical blades, and the other periodically extended the helical blades by perforation. A laboratory-scale hybrid rocket engine was used to investigate combustion characteristics of the fuel grains at an oxygen mass flux of 2.1–6.0 g/(s·cm2). Compared with the composite fuel grain with periodically extended helical blades, the modified composite fuel grains exhibited higher regression rates and a faster rise of regression rates as the oxygen mass flux increased. At an oxygen mass flux of 6.0 g/(s·cm2), the regression rate of the composite fuel grains with perforation and mirrored helical blades increased by 8.0% and 14.1%, respectively. The oxygen-to-fuel distribution of the composite fuel grain with mirrored helical blades was more concentrated, and its combustion efficiency was stable. Flame structure characteristics in the combustion chamber were visualized using a radiation imaging technique. A rapid increase in flame thickness of the composite fuel grains based on the modular unit was observed, which was consistent with their high regression rates. A simplified numerical simulation was carried out to elucidate the mechanism of the modified modular units on performance enhancement of the composite hybrid rocket grains.