With the increasing penetration of renewables, the power system is facing unprecedented challenges of low-inertia levels. The inherent ability of the system to defense disturbance and power imbalance through inertia response is degraded, and thus, system operators need to make faster and more efficient scheduling operations. As one of the most promising solutions, machine learning (ML) methods have been investigated and employed to realize effective inertia forecasting with considerable accuracy. Nevertheless, it is yet to understand its vulnerability with the growing threat of cyberattacks. To this end, this paper proposes a methodological framework to explore the vulnerability of ML-based inertia forecasting models, with a special focus on data integrity attacks. In particular, a cost-oriented false data injection attack is proposed, for the first time, with the primary objective to significantly increase the system operation cost while retaining the stealthiness of the attack via minimizing the differences between the pre-perturbed and after-perturbed inertia forecasts. Moreover, we propose four vulnerability assessment metrics for the ML-based inertia forecasting models. Case studies on the GB power system demonstrate the vulnerability and impact of the ML-based inertia forecasting models, as well as the stealthiness and transferability of the proposed cost-oriented data integrity attacks.