Machined surface defects influence functional properties of the component, such as reducing carrying capacity and shortening fatigue life. This study collected the cutting force and the acoustic emission (AE) signals, analyzed the effect of shear and ploughing effect on the frequency of the AE signals in machining, built the relationship between the first-order mode components (IMF1) and the second-order mode components (IMF2) of AE signals and the shearing and ploughing process respectively through variational mode decomposition (VMD). A new parameter Er = (RMS2/RMS1)2 was proposed as the signal feature to monitor machined surface defects, which represented the ratio of specific ploughing energy and specific shear energy. The results showed that when Er was in the range of 0.03–0.05, the machined surface quality was good, once it was higher than 0.05, the ploughing effect between the flank face and the machined surface was severe, thereby causing machining-induced surface defects.