Helical milling is a new hole-making technique which has lower burr formations and fiber delamination compared to the conventional drilling. In the helical milling process, the cutter follows the circular and vertical movements simultaneously, leading to two cutting zones at the peripheral part and the bottom part of the cutter, respectively. In this paper, the total cutting forces generated by both the side flutes and the bottom flutes are analytically predicted in the presence of cutter runout. Based on the kinematics analysis of the helical milling process, the bottom cutter-workpiece engagements (CWEs) are found to be the whole bottom part of the cutter due to its continuous plunge cut, while the peripheral CWEs are extracted by determining the feasible contact region and the axial depth of cut at any immersion angle. Next, an analytical method is proposed to calculate the instantaneous uncut chip thickness (IUCT) with high precision considering the cutter runout effect, which is defined as the distance from the cutting point to the previous flute’s circular trajectory. Combining the calibrated cutter runout parameters and cutting force coefficients of both side and bottom flutes, the dual mechanism force model is adopted to predict cutting forces. The helical milling tests have been conducted to validate the proposed mechanical model. The results show that the predicted cutting forces using this method match the measured data more closely compared to the existing methods which do not take into account the cutter runout effect.