Rotary ultrasonic machining is a hybrid machining process that combines diamond grinding and ultrasonic machining. The mathematical predictive material removal rate models have been developed in rotary ultrasonic machining with a constant pressure. However, there is no report on mathematical predictive cutting force model in rotary ultrasonic drilling at a constant feedrate presently. Since cutting force can not only reflect the processing state, but also affect the machined surface quality, it is necessary to develop a mathematical model for predicting cutting force which can forecast the machining results. This paper presents a mathematical model to predict the cutting force in rotary ultrasonic machining. On the basis of this model, the relations between cutting force and controllable machining parameters are researched by numerical computation method. This paper also researches the influences of spindle speed and feedrate on cutting force by experiments. The results observed through the experiments agree well with the relations generated from the mathematical model, which verify the developed model.