Carbon fiber–reinforced plastic composites have many superior properties, including low density, high strength-to-weight ratio, and good durability, which make them attractive in many industries. However, due to anisotropic properties, high stiffness, and high abrasiveness of carbon fibers in carbon fiber–reinforced plastic, high cutting force, high tool wear, and high surface roughness are always caused in conventional machining processes. This article reports an investigation using rotary ultrasonic machining in surface grinding of carbon fiber–reinforced plastic composites in order to develop an effective and high-quality surface grinding process. In rotary ultrasonic machining surface grinding of carbon fiber–reinforced plastic composites, tool selection is of great importance since tool variables will significantly affect output variables. In this work, the effects of tool variables, including abrasive size, abrasive concentration, number of slots, and tool end geometry, on machining performances, including the cutting force, torque, and surface roughness, are experimentally studied. The results show that lower cutting forces and torque are generated by the tool with higher abrasive size, lower abrasive concentration, and two slots. Lower surface roughness is generated by the tool with smaller abrasive size, smaller abrasive concentration, two slots, and convex end geometry. This investigation will provide guides for tool selections during rotary ultrasonic machining surface grinding of carbon fiber–reinforced plastic composites.