A vibration energy harvester can collect vibration energy from the environment to supply low-power devices, such as wireless sensor nodes. Reducing the size and power consumption of these devices is a challenging problem. A dual-function device, which includes energy harvesting and vibration measurement, may solve this problem. Therefore, studying inversion methods for identifying the external excitation of a harvester is meaningful. In this study, a modified inversion method that combines an extended Rauch–Tung–Striebel smoother (ERTSS) with the particle swarm optimization (PSO) algorithm, i.e., the ERTSS-PSO method, is proposed to identify the time domain signal of external excitation through the voltage response of a piezoelectric vibration energy harvester. A modified ERTSS approach with sliding windows is developed. The sliding windows are partially overlapped to eliminate identification errors between the two contiguous windows. Near real-time identification is achieved for the small sliding window. PSO is utilized to estimate state noise covariance and measurement noise covariance, improving identification accuracy compared with the L-curve approach. Under this proposed framework, prior knowledge regarding the statistical data of unknown external excitations is unnecessary, enabling a more comprehensive range of applications. To assess the performance of the proposed ERTSS-PSO method, numerical simulations, including two piezoelectric vibration energy harvesting systems with third-order and fifth-order nonlinear stiffness, are conducted to verify the effectiveness of the proposed method under harmonic, multifrequency, and random excitations. The method proposed in this study is also validated by a circular laminated piezoelectric plate harvester under harmonic and random excitations in an experiment. Results from the experimental studies demonstrate that the proposed method improves identification accuracy by at least 9% compared with the extended Kalman filter, at least 7% compared with the augmented Kalman filter (AKF) with Rauch–Tung–Striebel smoother (ARTSS), and at least 40% compared with AKF under random excitation. In addition, the proposed method is unaffected by initial conditions, and it is more stable and accurate than AKF and ARTSS. The proposed method lays the theoretical foundation for identifying the external excitation of a harvester. It is also a possible solution for the miniaturization and compactness of wireless monitoring devices.