This paper presents a systematic methodology focused on herringbone gear microgeometry modifications toward vibration reduction. The dynamic model considering the unique characteristics of aviation herringbone gear is developed to study the vibration behavior. The optimal ease-off shape can be defined as the outcome of a multi-objective optimization process, the objective functions are loaded transmission error, meshing impact excitation and root mean square (RMS) of vibration acceleration. With special attention given to computational efficiency, a novel fitness predicted genetic algorithm is developed. An application to herringbone gear are presented, the results show the proposed method can obtain optimal modifications that significantly improve the gear performance over a wide range of operating conditions. Furthermore, the reduction of the vibration also leads to a reduction of bending stresses. Finally, a test on herringbone gear is executed under various combinations of torque and speed to demonstrate the accuracy of the proposed model.