스토리 영상 콘텐츠 분석을 위한 장면 그래프 생성 모델 기반의 자동 온톨로지 구축 프레임워크

Abstract

In this paper, we propose an extended ontology automatic construction framework based on multiple deep learning models as part of an effective analysis of story video content. Semi-automatic techniques using image processing techniques have been the mainstream for the existing ontology construction methods for moving pictures, but there is a problem that human intervention is required and the accuracy is low due to the limitations of image processing techniques. To overcome this, in this paper, we propose an auto mated method based on the deep learning scene graph generation technique. In particular, in the case of video content with a story, the relationship between characters is a very important factor in understanding the scene, so deep learning-based object relationship creation model, character identification model, and important area caption generation model are applied to extract objects and recognize their relationships. And design a framework that automatically builds a domain ontology dependent on the story through procedural fusion between each model and module function. In addition, the proposed framework suggests a method for efficiently processing system requirements and system resources through meta control in the condition that requires simultaneous operation of multiple deep learning models to analyze story video content. Through this, the proposed framework effectively identifies critical region captions and object relationships in a scene in story-telling video content, and executes three types of models simultaneously. Finally, we conduct an experiment to automatically build an ontology by applying the proposed framework to specific video content, and check the effectiveness of the proposed framework.