In the present study, rotary ultrasonic machining (RUM) was adopted to perform drilling of carbon fiber reinforced plastics (CFRP) in a cryogenic environment. An L27 orthogonal array was selected to conduct experiments by varying the spindle speed (denoted as N), feed rate (denoted as f), and ultrasonic power (denoted as P). The thrust force (denoted as Fz), exit burr area, and surface roughness (denoted as Ra) were measured to evaluate the machining performance. The influence of process parameters and the regression model were derived for each output quality response. Additionally, multi-objective optimization was performed using desirability analysis, and the predicted levels were used for confirmation. The results indicated that the feed rate (f) contributed more to the thrust force (Fz) by 45.85% and a maximum thrust force was recorded at 0.1 mm/rev. A decrease in spindle speed (N) was associated with an increase in feed rate (f) and ultrasonic power (P), and it resulted in minimum exit burr area. The influence of ultrasonic power (P) was highly significant in reducing burrs with a contribution of 52.45%. Conversely, the surface roughness (Ra) of the drill holes decreased at 3000 rpm, and this was attributed to the brittle fracture of the fibers at a lower temperature. Both N (30.88%) and f (30.83%) had an equal influence on producing a better surface finish in the drill holes. Furthermore, the predicted optimal settings were used to validate the results and were found to be within 95% confidence and prediction interval. Finally, the microscopic images of tool wear, burr formation, and drill hole surface morphology were analyzed and examined.