Toroidal Alfvén eigenmodes excited by energetic particles (EPs) and their transition to energetic particle modes are investigated using a three-dimensional nonlinear kinetic-magnetohydrodynamic hybrid simulation code. Both the symmetric chirping inside the Alfvén continuum gap for upward and downward directions and the asymmetric chirping affected by the Alfvén continuum are found in the simulations depending on the EP drive and the mode damping. For weak drive and damping, symmetric frequency chirping is observed with a chirping rate consistent with the Berk–Breizman theory (Berk et al 1997 Phys. Lett. A 234 213). For moderate drive and damping, upward chirping is dominant because downward chirping is interrupted by interaction with the Alfvén continuum. For strong drive and damping, two branches of frequency chirping are found for different poloidal harmonics. The dominant chirping is downward into the Alfvén continuum and the other is an upward chirping inside the Alfvén continuum gap. The creation and the propagation of holes and clumps are found in EP phase space, which is qualitatively consistent with the Berk–Breizman theory.