Carbon fiber reinforced polyetheretherketone (CF/PEEK) is widely applied in aerospace and medical fields due to its excellent strength-to-weight ratio, fatigue resistance and biocompatibility. However, traditional dry machining significantly limits the improvement of machining efficiency and surface quality. Addressing this issue, this paper proposes a process optimization method of high machining efficiency and low surface defects for high-speed dry (HSD) milling UD-CF/PEEK, based on analyzing the thermal effect of cutting temperature. Mixed-level orthogonal experiments are designed for dry milling unidirectional (UD) CF/PEEK laminates with the fiber orientation of 0° and 90°. Three-dimensional surface roughness Sq, cutting temperature Tc and material removal rate MRR are extracted to characterize experiment results. The results show that the cutting temperature is mainly affected by cutting speed and fiber orientation, and machined surface defect is mainly caused by the thermal effect of high cutting temperature. Then, a process optimization model of high machining efficiency and low surface defects is established based on the genetic algorithm optimized BP (GA-BP) neural network, non-dominated sorting genetic algorithm (NSGA-II) and technique for order preference by similarity to an ideal solution (TOPSIS). The optimization objectives are Sq, MRR and Tc, and Tc also serves as a constraint of the thermal effect. Verification experiments are carried out, and results reveal that the surface defects are significantly reduced by limited thermal effect, and the model can be used to effectively improve machining efficiency and surface quality for HSD milling UD-CF/PEEK. Besides, the material removal rate generated by the HSD milling operation can easily reach 2–3 times higher than the traditional low-speed dry milling operation. The optimized cutting speed in HSD milling UD-CF/PEEK is recommended as 1300–1600 m/min. This study provides significant technical guidance for the machinability improvement of HSD milling UD-CF/PEEK in practice.