Research of cross-media information retrieval model based on multimodal fusion and temporal-spatial context semantic

Abstract

The solution of "semantic gap" between the low-level features describing and the high-level semantic knowledge has become the key in problems of the Cross-Media Retrieval(CMR),a CMR model based on multimodal fusion and temporal-spatial context semantic was designed.The Independent Component Analysis(ICA) and Principal Component Analysis(PCA) were applied to dimension reduction of multimodal fusion features.The classifier of Support Vector Machine(SVM) and Hidden Markov Model(HMM) was designed to map semantic relationship in the model;meanwhile,methods of temporal-spatial fuzzy cluster and relevance feedback were used to improve the effect of CMR system.A prototype based on the model had been developed,and validated the correctness of the new model,which can provide enlightenment to the designers who work at CMR system.