AbstractS2‐glass fiber reinforced plastics (S2‐GFRP) and basalt fiber reinforced plastics (BFRP) have emerged as crucial materials due to their exceptional mechanical properties, and milling of composite materials plays an important role in achieving desired properties. However, they have proven challenges due to relative inhomogeneity compared with metals, resulting unpredictability in quality of milling operations. The objective of this work is to investigate the effect of cutting parameters, tool geometry and tool surface materials on the surface quality of composites using burrs as a metric. S2‐GFRP and BFRP composites were produced by the vacuum infusion method. Helical and straight flute end mills were manufactured from high‐speed steel (HSS) and carbide rounds, and half of them were coated with titanium nitride using reactive magnetron sputtering technique. Taguchi L18 orthogonal array is used to determine the effect of tool material, tool angle, coating, cutting direction, spindle speed, and feed rate on the machining quality of S2‐GFRPs and BFRPs with respect to burr formations. Milling experiments were conducted under dry conditions and then the burrs were imaged to calculate the total area and length. Statistical analysis was also performed to optimize the machining parameters and tool type for ensuring the structural integrity and performance of the final composite parts. The results showed that the selection of tool material has the most significant impact on the burr area and length of the machined surface. The novel image analysis allows to analyze the extent of the burr size with a desirable operation speed for industrial applications.Highlights Aerospace grade S2‐Glass (S2‐GFRP) and basalt fiber reinforced plastics (BFRP) were manufactured. Carbide and HSS end mills were fabricated and coated with titanium nitride protective layer. FRPs were machined at various process parameters designed by Taguchi method. Distinctive image processing was firstly used to compute milling induced Burr area and length. Statistical analysis was performed to quantify the contribution of parameters and optimize milling.