Multimodal Analysis of User-Generated Content in Support of Social Media Applications

Abstract

The number of user-generated multimedia content (UGC) online has increased rapidly in recent years due to the ubiquitous availability of smartphones, cameras, and affordable network infrastructures. Thus, it attracts companies to provide diverse multimedia-related services such as preference-aware multimedia recommendations, multimedia-based e--learning, and event summarization from a large collection of multimedia content. However, a real-world UGC is complex and extracting semantics from only multimedia content is difficult because suitable concepts may be exhibited in different representations. Modern devices capture contextual information in conjunction with a multimedia content, which greatly facilitates in the semantics understanding of the multimedia content. Thus, it is beneficial to analyse UGC from multiple modalities such as multimedia content and contextual information (eg., spatial and temporal information). This doctoral research studies the multimodal analysis of UGC in support of above-mentioned social media problems. We present our proposed approaches, results, and works in progress on these problems.