Cooperating experts for soundtrack analysis of MPEG movies

Abstract

Abstract In this paper, we propose a multi-expert classification system (MES) for the audio classification of MPEG movies. The system has been designed according to an hybrid architecture which is made of three cascaded stages and constitutes an ensemble of different classifiers, each one implemented by means of a multi-expert architecture. Classification of the audio tracks exploits four pure classes (music, speech, silence and noise) plus three hybrid classes associated to complex patterns resulting from the overlap of different components (e.g., speech overlapped with music or noise). The soundtracks of 30 movies selected from various genres have been used for building a wide database of samples and for the successive assessment of system performance. A significant amount of experimental results obtained by the proposed MES, by other classification systems using a single classifier, and by another MES using a parallel fusion scheme, are reported in the paper together with comments and comparative analyses. In addition, the paper demonstrates the application of the knowledge arising from an analysis of intermediate classification results in order to obtain indications about the definition of the MES architecture. The results achieved by using our system are extremely encouraging when compared with those obtained by the other MES.