Video Hyperlinking Research Articles

Japanese Perception of Organ Donation and Implications for New Medical Technologies: Quantitative and Qualitative Social Media Analyses.

The Rapid Autopsy Program (RAP) is a valuable procedure for studying human biology and diseases such as cancer. However, implementing the RAP in Japan necessitates a thorough understanding of concepts such as good death and the integration of sociocultural aspects. By revising perceptions of organ donation on social media, we bring attention to the challenges associated with implementing new medical research procedures such as the RAP. This study aims to examine YouTube and Twitter/X to identify stakeholders, evaluate the quality of organ donation communication, and analyze sociocultural aspects associated with organ donation. Based on our findings, we propose recommendations for the implementation of new medical research procedures. Using the term "" (organ donation), we collected data from YouTube and Twitter/X, categorizing them into 5 dimensions: time, individuality, place, activity, and relationships. We utilized a scale to evaluate the quality of organ donation information and categorized YouTube videos into 3 groups to analyze their differences using statistical methods. Additionally, we conducted a text-based analysis to explore narratives associated with organ donation. Most YouTube videos were uploaded in 2021 (189/638, 29.6%) and 2022 (165/638, 25.9%), while tweets about organ donation peaked between 2019 and 2022. Citizens (184/770, 23.9%), media (170/770, 22.0%), and unknown actors (121/770, 15.7%) were the primary uploaders of videos on organ donation. In a sample of average retweeted and liked tweets, citizens accounted for the majority of identified users (64/91, 70%, and 65/95, 68%, respectively). Regarding Japanese regions, there were numerous information videos about organ donation in Hokkaido (F2.46,147.74=-5.28, P=.005) and Kyushu and Okinawa (F2.46,147.74=-5.28, P=.005). On Twitter/X, Japan and China were the most frequently mentioned countries in relation to organ donation discussions. Information videos often focused on themes such as borrowed life and calls to register as donors, whereas videos categorized as no information and misinformation frequently included accusations of organ trafficking, often propagated by Chinese-American media. Tweets primarily centered around statements of donation intention and discussions about family consent. The majority of video hyperlinks directed users to YouTube and Twitter/X platforms, while Twitter/X hyperlinks predominantly led to news reports from Japanese media outlets. There is significant potential to implement new medical research procedures such as the RAP in Japan. Recommendations include conceptualizing research data as borrowed data, implementing horizontally diversified management of donation programs, and addressing issues related to science misinformation and popular culture trends.

Learning to Match Anchor-Target Video Pairs With Dual Attentional Holographic Networks.

Video hyperlinking is the task of linking two video fragments/clips based on their multi-modal contents. Specifically, given an anchor video as a query, machine techniques automatically generate links between the anchor and target videos by modeling and comparing their content aboutness. The term "aboutness" specifically refers to contextually relevant multimedia content, i.e., a fragment is on or of something. Since video contents are multi-modal (e.g., audio and vision), the content aboutness may be reflected across different modalities. Existing approaches regard hyperlinking as a retrieval task, by embedding multi-modal video contents into one or multiple common video representation space(s) for cross-modal comparison. As a result, the aboutness between videos is scored by computing the vector-distance based similarity in the learnt common feature space. However, these methods suffer from two main limitations: (1) the video modality descriptors/features are treated equally in representation learning, which hinders the effective modeling of their respective capabilities in linking; and (2) directly using the vector-distance based similarity to measure aboutness bears the risk of returning more duplicates. This paper focuses on addressing these two problems. Specifically, we firstly build attentional neural networks to learn a compact fragment-level representation, assigning different importance weights to different descriptor/feature contents by an attention mechanism. We believe that the potentially interesting content(s) should be highlighted in the representation. Furthermore, instead of directly computing the similarity of two representation embeddings, we secondly build a holographic composition network to model the aboutness for link establishment, with the core use of circular correlation. The two networks string together to form the final hyperlinking matching system. The entire model is trained in an end-to-end fashion. We examine its effectiveness by creating four train/validate/test partitioning schemes on the Blip10000 dataset and employing two video fragmentation methods.

Neighbourhood Structure Preserving Cross-Modal Embedding for Video Hyperlinking

Video hyperlinking is a task aiming to enhance the accessibility of large archives, by establishing links between fragments of videos. The links model the aboutness between fragments for efficient traversal of video content. This paper addresses the problem of link construction from the perspective of cross-modal embedding. To this end, a generalized multi-modal auto-encoder is proposed. The encoder learns two embeddings from visual and speech modalities, respectively, whereas each of the embeddings performs self-modal and cross-modal translation of modalities. Furthermore, to preserve the neighbourhood structure of fragments, which is important for video hyperlinking, the auto-encoder is devised to model data distribution of fragments in a dataset. Experiments are conducted on Blip10000 dataset using the anchor fragments provided by TRECVid Video Hyperlinking (LNK) task over the years of 2016 and 2017. This paper shares the empirical insights on a number of issues in cross-modal learning, including the preservation of neighbourhood structure in embedding, model fine-tuning and issue of missing modality, for video hyperlinking.

Effective video hyperlinking by means of enriched feature sets and monomodal query combinations

Video content has been increasing at an unprecedented rate in recent years, bringing the need for improved tools providing efficient access to specific contents of interest. Within the management of video content, hyperlinking aims at determining related video segments from a collection with respect to an input video anchor. This paper describes the system we designed to address feature selection for the video hyperlinking challenge, as defined by TRECVID, one of the top worldwide venues for multimedia benchmarking. The proposed solution is based on different combinations of textual and visual features, enriched to capture the various facets of the videos: automatically generated transcripts, visual concepts, video metadata, named-entity recognition, and concept-mapping techniques. The different combinations of monomodal queries are experimentally evaluated, and the impact of both parameters and single features are discussed to identify their contributions. The best performing approach at the TRECVID 2017 video hyperlinking challenge was the ensemble feature selection, which includes three different monomodal queries based on enriched feature sets.

A Crossmodal Approach to Multimodal Fusion in Video Hyperlinking

With the recent resurgence of neural networks and the proliferation of massive amounts of unlabeled multimodal data, recommendation systems and multimodal retrieval systems based on continuous representation spaces and deep learning methods are becoming of great interest. Multimodal representations are typically obtained with autoencoders that reconstruct multimodal data. In this article, we describe an alternative method to perform high-level multimodal fusion that leverages crossmodal translation by means of symmetrical encoders cast into a bidirectional deep neural network (BiDNN). Using the lessons learned from multimodal retrieval, we present a BiDNN-based system that performs video hyperlinking and recommends interesting video segments to a viewer. Results established using TRECVIDs 2016 video hyperlinking benchmarking initiative show that our method obtained the best score, thus defining the state of the art.

VIREO-VH

"Video Hyperlinking" refers to the creation of links connecting videos that share near-duplicate segments. Like hyperlinks in HTML documents, the video links help user navigating videos of similar content, and facilitate the mining of iconic clips (or visual memes) spread among videos. Figure 1 shows some example of iconic clips, which can be leveraged for linking videos and the results are potentially useful for multimedia tasks such as video search, mining and analytics. VIREO-VH [1] is an open source software developed by the VIREO research team. The software provides end-to-end support for the creation of hyperlinks, including libraries and tools for threading and visualizing videos in a large collection. The major software components are: near-duplicate keyframe retrieval, partial nearduplicate localization with time alignment, and galaxy visualization. These functionalities are mostly implemented based on state-of-the-art technologies, and each of them is developed as an independent tool taking into consideration flexibility, such that users can substitute any of the components with their own implementation. The earlier versions of the software are LIP-VIREO and SOTU, which have been downloaded more than 3,500 times. VIREO-VH has been internally used by VIREO since 2007, and evolved over the years based on the experiences of developing various multimedia applications, such as news events evolution analysis, novelty reranking, multimedia-based question-answering [2], cross media hyperlinking [3], and social video monitoring.

Spatial and temporal content-based access to hypervideo databases

Providing content-based video query, retrieval and browsing is the most important goal of a video database management system (VDBMS). Video data is unique not only in terms of its spatial and temporal characteristics, but also in the semantic associations manifested by the entities present in the video. This paper introduces a novel video data model called Logical Hypervideo Data Model. In addition to multilevel video abstractions, the model is capable of representing video entities that users are interested in (defined as hot objects) and their semantic associations with other logical video abstractions, including hot objects themselves. The semantic associations are modeled as video hyperlinks and video data with such property are called hypervideo. Video hyperlinks provide a flexible and effective way of browsing video data. Based on the proposed model, video queries can be specified with both temporal and spatial constraints, as well as with semantic descriptions of the video data. The characteristics of hot objects' spatial and temporal relations and efficient evaluation of them are also discussed. Some query examples are given to demonstrate the expressiveness of the video data model and query language. Finally, we describe a modular video database system architecture that our web-based prototype is based on.