Determining the psycho-emotional state of the observed based on the analysis of video observations

Abstract

This paper develops a system for determining the psycho-emotional state of the observed people based on the analysis of video surveillance with the application of artificial intelligence technology using hardware and software tools such as PoseNet, PyTorch, SQLite, FastAPI and Flask. In many areas of human endeavor, there is an urgent need for a surveillance system that can reliably function and detect suspicious activities. To solve this problem, this paper proposes a novel framework for a real-time surveillance system that automatically detects abnormal human activities. The system has been tested and validated in real environments. The results of testing artificial intelligence program models showed the best results (f1 score with values of 0.98–0.99). The weighted average value of the f1-score metric was 0.96, which is quite a high value. The use of PoseNet implemented with PyTorch allowed to accurately determine the pose of the person in the video and extract information about the position of different body parts. The peculiarity of this work lies in the development of artificial intelligence models for automatic detection of possible physical aggression in videos, in the methods of forming an optimal set of features for the development of AI models that identify the aggressor and the victim of bullying. The developed system has the potential to be a useful tool in various fields such as psychology, medicine, security and others where it is important to analyze the emotional state of people based on their physical manifestations. The obtained applied results can be used in educational institutions and in spheres where video analysis is necessary