Abstract This paper focuses on the management of physical movement sports disciplines and combines relevant algorithms to design an artificial intelligence-based physical education management system framework. In terms of related algorithms, the first is for the research of character recognition algorithms. Considering the intra-class differences between individual movements, the LR-based feature extraction algorithm is proposed. Secondly, for the research of the action evaluation algorithm, the statistical value, DTW parameter action time series difference and correlation coefficient based on the statistical value and DTW parameter are used as the features, combined with SVM-based feature extraction algorithm for perceiving the action differences. Finally, with the help of deep neural networks for movement analysis, we can improve students’ movement techniques during training. In the course schedule supported by the system, the proportion of moderate-intensity exercise time is high, basically, more than 40%, which meets the WHO requirements for high-intensity exercise time in physical education classes. The post-course RPE feedback from students was spread out between 12 and 15, and the highest post-course RPE value was 14.82<unk>1.38, which was evaluated within a reasonable range of intervals. The teaching management system in this paper can meet the requirements of physical training and teaching, and provide quantifiable basis and management tools for the informatization and precise teaching of physical movement subjects.