Realtime Semantic Enrichment of Video Streams in the Age of Big Data

Abstract

This paper describes AgileRAI, a framework for searching, organizing, and accessing multimedia data in a fast and semantic-driven way. AgileRAI supports realtime ingestion of video streams on which different machine-learning techniques (such as global and local visual features extraction and matching) are applied in a parallel and scalable way. Extracted features are matched to a reference database of visual patterns (e.g., faces, logos, and monuments) in order to produce a set of metatags describing the ingested contents. Furthermore, these tags are semantically enriched using open semantic data repositories. The system is designed with a scale-out pattern architecture based on Apache Spark, ensuring high performance in Big Data management environments.