The quantification of mental fatigue (MF) in aviation remains a long-lasting problem which has a bearing on flight safety. In this study, we have developed a complexity-based MF index based on multiscale entropy (MSE) analysis of instantaneous frequency variation (IFV) in auditory steady-state response (ASSR). Subjects were asked to undertake a flight simulation task, before and after which, electroencephalograms (EEGs) were recorded with no auditory stimulus, and with Don chirp sound to elicit ASSR. In the anti-interference tests, our proposed MF index showed higher performance in terms of anti-interference from electrooculogram (EOG) and environment. In the flight simulation task, fatigue evaluations based on ASSR have achieved a better fatigue detection effect comparing with the conventional methods: 95% in the fatigue detection rate and −13.885% in the MF index. In conclusion, the proposed MF index has high sensitivity and high anti-interference capacities, therefore has potential applications in a variety of fields that require robust monitoring of MF outside a shielded chamber.