The evolution of vortex structure and mass transport process during dynamic stall of the two-dimensional pitching NACA0012 airfoil are studied in detail, using Lagrangian coherent structures (LCSs), and the nature of dynamic stall is given from viewpoint of nonlinear dynamics. First, the numerical method with SST k−ω turbulence model is used to simulate the flow field around the pitching oscillation airfoil. Then, the Lagrangian-averaged vorticity deviation (LAVD) method is used to analyze the formation and evolution of vortex structure during dynamic stall. Further, the circulation development process of the primary leading edge vortex (LEV) is studied, with comparison between LAVD method with Q criterion. Finally, the dynamic behaviors, such as the mass transport and evolution of flow structure during the process of dynamic stall, are analyzed in depth by using LCSs, and the fluctuation of lift coefficient and the dynamic stall are investigated from the viewpoint of nonlinear dynamics. The results show that the evolution of vortex structure and mass transport are closely related to the fluctuation of lift coefficient during the dynamic stall, and the low-pressure region formed by vortices and the complex mass transport and mixing process on the suction surface of the airfoil are the main reasons for the high lift during the dynamic stall process. In particular, the primary LEV, which can be accurately identified and tracked by LAVD method, plays a very important role in lift enhancement, and the primary LEV and the secondary LEV form and grow in turn by the intermittent feeding process of shear layer. Additionally, the fluid particles in front of the leading edge of the airfoil enter the LEV through a special mass transport channel, which is textured by LCSs and opened and closed intermittently. More importantly, the saddle point of LCSs is an important component of vortex boundary, and its dynamic behaviors represent the behaviors of vortex. The first separation of saddle point from the shear layer at leading edge indicate the separation of primary LEV from the suction surface of the airfoil, and the occurrence of dynamic stall. The second separation of saddle point from the shear layer at leading edge indicate that primary LEV begins to leave the upper region of the airfoil, corresponding to a sharp drop in lift coefficient. Generally, compared to the traditional visualization techniques, the Lagrangian analysis based on LCSs and LAVD can provide a deeper insight into the dynamics of the vortex in flow field around pitching oscillation airfoil and the dynamic mechanism of dynamic stall.