Abstract
Information on the World Wide Web appears in diverse forms, including text, image, audio, and video. Presented with a wide range of information, an information user often takes great effort to correlate and track online information related to specific topics of interest. Fusion of multimedia information in a unified framework is thus needed for efficiently understanding and further analyzing the semantically related information. This thesis addresses the problem of multimedia web information fusion and analysis by presenting an approach for modelling multimedia information in a unified semantic framework, based on which cross-media information analysis and mining is realized. As multimedia data are heterogeneous in their contents and formats, we employ a strategy for multimedia information fusion based on semantics of the data. Specifically, we develop two methods, one using a statistical vague transformation technique and the other employing a self-organizing neural network, to associate web images with related surrounding texts, based on which the semantics of the media objects can be extracted. Our experiments show that the proposed methods can identify associated image and text pairs with good accuracy and outperform a state-of-the-art method for image annotation using a statistical relevance model. To support cross-media analysis, this thesis develops a semantic representation schema, that combines MPEG-7 multimedia description, RDF language specification, and conceptual graph based knowledge representation techniques for modelling multimedia information. In addition, we develop a semantic metadata extraction algorithm utilizing a myriad of natural language processing (NLP) techniques to automatically extract concepts and relations from text contents. The extracted concepts are formally represented as bags of WordNet senses, based on which an incremental clustering approach i ATTENTION: The Singapore Copyright Act applies to the use of this document. Nanyang Technological University Library
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
