With the increasing penetration level of renewable energy resources with less system inertia, accurate estimation of the power system inertia has become a critical issue for maintaining the frequency stability of the entire power system. To tackle out this challenging task, a two-stage data-driven method is proposed in this paper for estimating the system inertia from disturbed PMU measurements. First, by integrating its low-order system frequency response model with the first-order turbine model, analytical expressions of the frequency response under the steady-state and these transient oscillatory components are derived. Then, based on this parametric model, a two-stage estimation algorithm will be developed. At the first stage, system parameters of oscillatory components can be extracted from PMU measurements by Estimation of Signal Parameters via Rotational Invariance Techniques (ESPRIT). By utilizing frequency measurement data from PMUs and those estimated parameters from the ESPRIT algorithm, a weighted nonlinear least square approach is applied at the second stage to estimate physical parameters such as system inertia, damping coefficient, turbine time constant, and regulation coefficient. To validate the effectiveness of the proposed method, simulation studies of IEEE 39- bus system is investigated. Historical PMU measurements from various contingencies of Taiwan Power System will also be explored. All of these results demonstrated that the proposed method provides satisfactory performances with the mean relative error less than 10%. Comparisons with other existing methods are performed to demonstrate the advantage of the proposed method.