A Flexible Framework for Audio Semantic Content Detection

Abstract

Audio semantic analysis is a crucial issue in multimedia applications. In this paper, a hierarchical framework is proposed for high-level semantic content detection for a continuous audio stream. In the proposed framework, basic audio events are modeled with hidden Markov models. Based on the obtained key audio event sequence, a neural network-based approach is proposed to discover the high-level semantic content of the audio context. With the neural network-based approach, human knowledge and machine learning are effectively combined in the semantic inference. We select some audio streams to evaluate the performance of the proposed framework, and the experiment results demonstrate the framework can achieve satisfying results.