Cross-media Analysis Research Articles

How Deep Neural Networks Understand Motion? Toward Interpretable Motion Modeling by Leveraging the Relative Change in Position

Motion understanding plays an important role in video-based cross-media analysis and multiple knowledge representation learning. This paper discusses physical motion recognition and prediction by deep neural networks (DNNs), such as convolutional neural networks and recurrent neural networks. In physics, motion is the relative change in position with respect to time. To ablate the moving object and the background where the motion happens, we focus on an ideal scenario where a point moves in a plane. As the first contribution, we evaluate a few popular DNN architectures from video research on the relative position change modeling. Experiment results and conclusions can be insightful in action recognition and video prediction. As the second contribution, we propose a vector network (VecNet) to model the relative change in position. VecNet considers the motion in a short interval as a vector. Meanwhile, VecNet can move a point to the corresponding position given a vector representation. To obtain the representation of the motion for a long time, we use a long short-term memory (LSTM) to aggregate or predict vector representations over time. The resulting VecNet+LSTM approach is able to effectively support both recognition and prediction, proving that modeling relative position change is necessary for motion recognition and makes motion prediction easier.

Convolutional Neural Network-Based Cross-Media Semantic Matching and User Adaptive Satisfaction Analysis Model.

In this paper, an in-depth study of cross-media semantic matching and user adaptive satisfaction analysis model is carried out based on the convolutional neural network. Based on the existing convolutional neural network, this paper uses rich information. The spatial correlation of cross-media semantic matching further improves the classification accuracy of hyperspectral images and reduces the classification time under user adaptive satisfaction complexity. Aiming at the problem that it is difficult for the current hyperspectral image classification method based on convolutional neural network to capture the spatial pose characteristics of objects, the problem is that principal component analysis ignores some vital information when retaining a few components. This paper proposes a polymorphism based on extension Attribute Profile Feature (EMAP) Stereo Capsule Network Model for Hyperspectral Image Classification. To ensure the model has good generalization performance, a new remote sensing image Pan sharpening algorithm based on convolutional neural network is proposed, which increases the model's width to extract the feature information of the image and uses dilated instead of traditional convolution. The experimental results show that the algorithm has good generalization while ensuring self-adaptive satisfaction.

Celebrity memes, audioshop, and participatory fan culture: a case study on Keanu Reeves memes

ABSTRACT Contemporary meme culture keeps Keanu Reeves’ image at the forefront of our imagination. Numerous Keanu-related memes circulate on the internet, featuring images of him from on and off screen. In this article, we argue that the rise of user-generated content, such as memes and social media, requires us to reconsider how audiences and fans consume, adapt, and circulate Keanu’s stardom. Specifically, we contend that memes and their attending participatory culture make Keanu familiar by placing him in knowable spaces and perpetuating his extraordinary ordinariness. To do so, we examine the 2019 ‘Keanu Reeves Walking in Slow Motion’ video clip from Always by My Maybe and the memes generated by users. First, we explore the purpose of celebrity memes and participatory culture, examining the ways in which Keanu’s image and persona is defined within this culture. Second, using a cross-media analysis, we define the recent meme genre ‘audioshop’, and explore how ‘Keanu Reeves Walking in Slow Motion’ enables audiences to use their prior knowledge of Keanu’s persona to more fully engage with the meme and its possible interpretations. Finally, we discuss how meme culture objectifies and commodifies celebrity, arguing that celebritisation through meme production amplifies Keanu’s persona and cross-media presence.

A Creature Without a Cave

This article analyses prominent examples of the Wendigo myth in modern North American horror media and the implications of misappropriation by non-Indigenous creators for non-Indigenous audiences. This article’s cross-media analysis covers television, film and game media; Teen Wolf (2011–17), Supernatural (2005–20), Bruce Wemple’s The Retreat (2020) and SuperMassive Games’ Until Dawn (2015). This analysis will trace the process of these media, made by non-Indigenous white creators, removing the Wendigo’s indigeneity and placing it within fictional settings as an antagonist. I have named this observation of the Wendigo the ‘Caveless Creature’ phenomenon. The paper concludes that employing the Wendigo as a caveless creature is a common practice within horror as it easily creates a villain for white protagonists to defeat repeatedly. This construction is problematic in the horror genre as it presents an Indigenous antagonist that poses a threat to white culture for its otherness and indigeneity – while at the same time, misappropriating, discarding and demonising the Indigenous culture the myth comes from, at whim. Although this article is specifically observing the Wendigo, I argue that it is one of many caveless creatures, and the treatment of them by creators of non-Indigenous horror genre should be analysed in the future.

The Role of Political Communication in Shaping Public Opinion: A Comparative Analysis of Traditional and Digital Media

The purpose of this cross-media analysis was to better understand how public opinion is influenced by political communication, and how it can be regulated and held accountable to better reflect political discourse in the media. A content analysis of news articles and social media posts, as well as surveys and a review of the pertinent literature, were used to collect data. Traditional media continue to play a significant role in influencing public opinion, but digital media have become a formidable tool for political communication, particularly among newer generations. Concerns regarding the propagation of false information and the impact of social media algorithms impede open and objective political discourse. There was unanimity that regulatory and accountability measures, such as media literacy programs and disclosure requirements for political advertisements, were required to ensure an honest and balanced media portrayal of political debate. These findings highlight the need for ongoing research and policymaking to address the dynamic character of political communication in the digital age.

Formal reporting style (Journalistic Reporting Styles)

Formal reporting style (Journalistic Reporting Styles)

Introduction to the Special Issue on the Cross-Media Analysis for Visual Question Answering

Introduction to the Special Issue on the Cross-Media Analysis for Visual Question Answering

A feature selection framework for video semantic recognition via integrated cross-media analysis and embedded learning

Video data are usually represented by high dimensional features. The performance of video semantic recognition, however, may be deteriorated due to the irrelevant and redundant components included into the high dimensional representations. To improve the performance of video semantic recognition, we propose a new feature selection framework in this paper and validate it through applications of video semantic recognition. Two issues are considered in our framework. First, while those labeled videos are precious, their relevant labeled images are abundant and available in the WEB. Therefore, a supervised transfer learning is proposed to achieve the cross-media analysis, in which the discriminative features are selected by evaluating feature’s correlation with the classes of videos and relevant images. Second, the labeled videos are normally rare in real-world applications. In our framework, therefore, an unsupervised subspace learning is added to retain the most valuable information and eliminate the feature redundancies by leveraging both labeled and unlabeled videos. The cross-media analysis and embedded learning are simultaneously learned in a joint framework, which enables our algorithm to utilize the common knowledge of cross-media analysis and embedded learning as supplementary information to facilitate decision making. An efficient iterative algorithm is proposed to optimize the proposed learning-based feature selection, in which convergence is guaranteed. Experiments on different databases have demonstrated the effectiveness of the proposed algorithm.

The media-oriented cross domain recommendation method

With the rapid development of modern high-tech, such as big data and artificial intelligence, the demand for cross-media services is also greatly improved. Since different modes of cross-media data apply different dimensions and different attributes of the underlying features to present data, many tasks need to work collaboratively to handle multiform of information (including text, audio, video, image, etc.), so as to build cross-media analysis and reasoning. Through the cross-media, the method could express the same semantic information from their own side and could reflect the specific information more fully than a single media object and its specific modal. The same information is cross spread and integrated across different kinds of media objects. Only conducting fusion analysis to these multi-modal media, can ones fully and correctly understand the content information contained in the cross-media complex, which also adds difficulty in cross-media information recommendation process. At the same time, data sparsity, cold start and scalability issues of traditional recommendation system have long been unsolved, and as such it cannot adapt to the personalized service needs in cross-media applications. Focusing on the field of media information recommendation and taking the media data, user behavior data and project attribute information as the information source, this paper aims at researching the cross-domain recommendation algorithm. With the help of the label data, matrix decomposition, the author constructs a media-oriented cross-domain recommendation model in order to improve the recommendation accuracy and solve the data sparsity, cold start problems of media information recommendation technology, exploring a high-accurate media-oriented cross-domain recommendation method.

Cross-media analysis and reasoning: advances and directions

Cross-media analysis and reasoning is an active research area in computer science, and a promising direction for artificial intelligence. However, to the best of our knowledge, no existing work has summarized the state-of-the-art methods for cross-media analysis and reasoning or presented advances, challenges, and future directions for the field. To address these issues, we provide an overview as follows: (1) theory and model for cross-media uniform representation; (2) cross-media correlation understanding and deep mining; (3) cross-media knowledge graph construction and learning methodologies; (4) cross-media knowledge evolution and reasoning; (5) cross-media description and generation; (6) cross-media intelligent engines; and (7) cross-media intelligent applications. By presenting approaches, advances, and future directions in cross-media analysis and reasoning, our goal is not only to draw more attention to the state-of-the-art advances in the field, but also to provide technical insights by discussing the challenges and research directions in these areas.

Learning robust uniform features for cross-media social data by using cross autoencoders

Cross-media analysis exploits social data with different modalities from multiple sources simultaneously and synergistically to discover knowledge and better understand the world. There are two levels of cross-media social data. One is the element, which is made up of text, images, voice, or any combinations of modalities. Elements from the same data source can have different modalities. The other level of cross-media social data is the new notion of aggregative subject (AS)— a collection of time-series social elements sharing the same semantics (i.e., a collection of tweets, photos, blogs, and news of emergency events). While traditional feature learning methods focus on dealing with single modality data or data fused across multiple modalities, in this study, we systematically analyze the problem of feature learning for cross-media social data at the previously mentioned two levels. The general purpose is to obtain a robust and uniform representation from the social data in time-series and across different modalities. We propose a novel unsupervised method for cross-modality element-level feature learning called cross autoencoder (CAE). CAE can capture the cross-modality correlations in element samples. Furthermore, we extend it to the AS using the convolutional neural network (CNN), namely convolutional cross autoencoder (CCAE). We use CAEs as filters in the CCAE to handle cross-modality elements and the CNN framework to handle the time sequence and reduce the impact of outliers in AS. We finally apply the proposed method to classification tasks to evaluate the quality of the generated representations against several real-world social media datasets. In terms of accuracy, CAE gets 7.33% and 14.31% overall incremental rates on two element-level datasets. CCAE gets 11.2% and 60.5% overall incremental rates on two AS-level datasets. Experimental results show that the proposed CAE and CCAE work well with all tested classifiers and perform better than several other baseline feature learning methods.

When News Goes Online. A Cross-Media Analysis of Editorial Logics and Consumers’ Feedbacks in the Printed, Online and Facebook Versions of the Italian Newspaper la Repubblica

<p>As a consequence of the advent and diffusion of new media, one of the most accredited hypotheses in the realm of mediatization theory has been that the essential prerequisites of mediatization would have slowly started to disappear. On the contrary, we hypothesize that the unprecedented knowledge about users’ preferences given to media companies would be reflected in the logics of news production, which would shift from being guided by internal logics and standards of newsworthiness to be driven by an audience-oriented commercial logic. Therefore, we expect storytelling techniques to prevail in online news production, with <em>soft news </em>becoming progressively prevalent moving from traditional to new media. We address our hypothesis performing a cross-media analysis of the Italian newspaper <em>la Repubblica</em>, investigating both the different editorial logics underlying the selection and framing of contents as well as the relationship between the general news frame and the level of readers’ engagement. In our findings, <em>soft news </em>prevails regardless of the platform, also following a positive trend as we move towards Facebook. Moreover, <em>soft news </em>seems to be able to foster a higher level of users’ engagement as expressed in terms of likes and shares, while <em>hard news </em>prevails in commenting activities.</p>

Editorial of the special issue on cross-media analysis

Today, there are lots of heterogeneous and homogeneous media data from multiple sources, such as news media websites, microblog, mobile phone, social networking websites, and photo/video sharing websites. Integrated together, these media data represent different aspects of the real-world and help document the evolution of the world. Consequently, it is impossible to correctly conceive and to appropriately understand the world without exploiting the data available on these different sources of rich multimedia content simultaneously and synergistically. Cross-media analysis is a research area in the general field of multimedia content analysis that focuses on the exploitation of the data with different modalities frommultiple sources simultaneously and synergistically

Large-scale cross-media analysis and mining from socially curated contents

Large-scale cross-media analysis and mining from socially curated contents

Augmented reality in education: a meta-review and cross-media analysis

Augmented reality (AR) is an educational medium increasingly accessible to young users such as elementary school and high school students. Although previous research has shown that AR systems have the potential to improve student learning, the educational community remains unclear regarding the educational usefulness of AR and regarding contexts in which this technology is more effective than other educational mediums. This paper addresses these topics by analyzing 26 publications that have previously compared student learning in AR versus non-AR applications. It identifies a list of positive and negative impacts of AR experiences on student learning and highlights factors that are potentially underlying these effects. This set of factors is argued to cause differences in educational effectiveness between AR and other media. Furthermore, based on the analysis, the paper presents a heuristic questionnaire generated for judging the educational potential of AR experiences.

When standard RANSAC is not enough: cross-media visual matching with hypothesis relevancy

The same scene can be depicted by multiple visual media. For example, the same event can be captured by a comic image or a movie frame; the same object can be represented by a photograph or by a 3D computer graphics model. In order to extract the visual analogies that are at the heart of cross-media analysis, spatial matching is required. This matching is commonly achieved by extracting key points and scoring multiple, randomly generated mapping hypotheses. The more consensus a hypothesis can draw, the higher its score. In this paper, we go beyond the conventional set-size measure for the quality of a match and present a more general hypothesis score that attempts to reflect how likely is each hypothesized transformation to be the correct one for the matching task at hand. This is achieved by considering additional, contextual cues for the relevance of a hypothesized transformation. This context changes from one matching task to another and reflects different properties of the match, beyond the size of a consensus set. We demonstrate that by learning how to correctly score each hypothesis based on these features we are able to deal much more robustly with the challenges required to allow cross-media analysis, leading to correct matches where conventional methods fail.

Serial Narrative, Intertextuality, and the Role of Audiences in the Creation of a Franchise: An Analysis of the Indiana Jones Saga from a Cross-Media Perspective

Rather than the simple adaptation of a story across different media, cross-media narrative should be defined as that which is transformed by virtue of use a medium and its different languages (film, comics, video games, etc.). This article extends and updates the dimensions of previous cross-media analysis models by underlining narrative aspects of the different products discussed and examining the influence of generic conventions along the way. To test this methodology, the article focuses on a product from the “adventure” genre— the Indiana Jones franchise—which it argues should be seen as a benchmark in the history of this kind of narrative. Special attention is given to the treatment of the character of Indiana Jones, who is the true focal point of the narrative, and to other underlying thematic features.

Framing Transparency in the U.S. – Cross-media Analysis of the Debate on WikiLeaks

Some scholars have argued that the novelty of WikiLeaks for transparency is its usage of new technologies, acting as an example for the start of a new technological information era. Cull (2011), for instance, has argued that WikiLeaks exemplifies a “shift in power” made possible by “the technological revolution” that “has given one individual the communication power that was the monopoly of the nation state in the previous century” (pp. 2-3). As Bunz (2011) further outlines, Wikileaks shows how information from one can be send to many through the “digitalisation of knowledge” (pp. 139-140), whereby it has become easy to transport a great amount of information using minimal space. Furthermore, it demonstrates that the autopsy of data has become easy as programs help us order and analyse information. It is also an example of how the Internet has made it possible for anyone to publish and access information at any time (ibid.). In this chapter, we suggest that technology alone is insufficient to create different transparency standards and change the way politics are conducted. In line with Florini (2002), we believe that “transparency is a choice, encouraged by changing attitudes about what constitutes appropriate behavior” (p. 13). Thus, new technology must be accompanied by a change in attitudes, as “without a norm of transparency, technology will continue to protect private information as well as ferret it out” (Florini, 2002, p. 15). Studying the debate triggered by WikiLeaks presents the opportunity to examine whether its revelations have strengthened transparency in public perception, or if WikiLeaks is no more than the example of new technological means without any real impact on the discursive boundary between publicity and secrecy.

A Bayesian network modeling approach for cross media analysis

Existing methods for the semantic analysis of multimedia, although effective for single-medium scenarios, are inherently flawed in cases where knowledge is spread over different media types. In this work we implement a cross media analysis scheme that takes advantage of both visual and textual information for detecting high-level concepts. The novel aspect of this scheme is the definition and use of a conceptual space where information originating from heterogeneous media types can be meaningfully combined and facilitate analysis decisions. More specifically, our contribution is on proposing a modeling approach for Bayesian Networks that defines this conceptual space and allows evidence originating from the domain knowledge, the application context and different content modalities to support or disproof a certain hypothesis. Using this scheme we have performed experiments on a set of 162 compound documents taken from the domain of car manufacturing industry and 118 581 video shots taken from the TRECVID2010 competition. The obtained results have shown that the proposed modeling approach exploits the complementary effect of evidence extracted across different media and delivers performance improvements compared to the single-medium cases. Moreover, by comparing the performance of the proposed approach with an approach using Support Vector Machines (SVM), we have verified that in a cross media setting the use of generative rather than discriminative models are more suited, mainly due to their ability to smoothly incorporate explicit knowledge and learn from a few examples.

Cross Media Analysis and Retrieval on the Web: Theory and Algorithm

Cross Media Analysis and Retrieval on the Web: Theory and Algorithm