Motion and aggressive behaviors in pigs provide important information for the study of social hierarchies in pigs and can be used as a selection indicator for pig health and aggression parameters. However, relying only on visual observation or surveillance video to record the number of aggressive acts is time-consuming, labor-intensive, and lasts for only a short period of time. Manual observation is too short compared to the growth cycle of pigs, and complete recording is impractical in large farms. In addition, due to the complex process of assessing the intensity of pig aggression, manual recording is highly influenced by human subjective vision. In order to efficiently record pig motion and aggressive behaviors as parameters for breeding selection and behavioral studies, the videos and pictures were collected from typical commercial farms, with each unit including 8~20 pigs in 7~25 m2 space; they were bred in stable social groups and a video was set up to record the whole day's activities. We proposed a deep learning-based recognition method for detecting and recognizing the movement and aggressive behaviors of pigs by recording and annotating head-to-head tapping, head-to-body tapping, neck biting, body biting, and ear biting during fighting. The method uses an improved EMA-YOLOv8 model and a target tracking algorithm to assign a unique digital identity code to each pig, while efficiently recognizing and recording pig motion and aggressive behaviors and tracking them, thus providing statistics on the speed and duration of pig motion. On the test dataset, the average precision of the model was 96.4%, indicating that the model has high accuracy in detecting a pig's identity and its fighting behaviors. The model detection results were highly correlated with the manual recording results (R2 of 0.9804 and 0.9856, respectively), indicating that the method has high accuracy and effectiveness. In summary, the method realized the detection and identification of motion duration and aggressive behavior of pigs under natural conditions, and provided reliable data and technical support for the study of the social hierarchy of pigs and the selection of pig health and aggression phenotypes.