Video Description Research Articles

Video Description Generation Method Based on Contrastive Language–Image Pre-Training Combined Retrieval-Augmented and Multi-Scale Semantic Guidance

In view of the insufficiency of the text encoder using CLIP and the insufficiency of the interaction between the two towers using CLIP, a CLIP-based video description model RAMSG is proposed, which combines retrieval augmentation with multi-scale semantic guidance. Firstly, RAMSG uses the visual and text encoder module of CLIP to achieve cross-modal retrieval and extract relevant text as a supervisory signal. Then, the intrinsic order of semantic words is obtained by a semantic detector and ranker. Finally, the global and local semantic guidance of the multi-scale semantic guidance module is used to improve the video description generation effect of the decoder module. Experimental results on the video description datasets MSR-VTT and VATEX show that the RAMSG model has a significant improvement in several performance indicators compared with other work, and the additional text semantics obtained through videotext matching tasks greatly improve the performance of the model.

A Commentary on "Full-Endoscopic Resection of a Lumbar Intradural Tumor (Schwannoma): Video Case Report and Description of the Surgical Technique".

A Commentary on "Full-Endoscopic Resection of a Lumbar Intradural Tumor (Schwannoma): Video Case Report and Description of the Surgical Technique".

Clinical Experience and Video Description of Minimally Invasive Surgery for Benign Prostatic Obstruction using the Schelin Catheter®.

Although designed for in-office use, water vapor thermal therapy (RezumTM) remains a painful procedure, and oral ± intravenous sedation is often required. Schelin Catheter® (ProstaLund AB, Lund, Sweden) is an innovative bladder catheter that delivers local anesthesia to the prostate in a sterile transurethral way. We aimed to evaluate the safety and feasibility of RezumTM for male lower urinary tract symptoms due to benign prostatic obstruction (LUTS/BPO) with local anesthesia delivered with Schelin® catheter, and to report its first step-by-step video description. A total of 15 patients were enrolled, and 14 analyzed. Median pain numeric rating scale at catheter insertion, anesthetic injection, RezumTM insertion, RezumTM treatment, and at 2-h post-operatively were 3(2-5), 3(2-5), 1(0-3), 3.5(2-6), and 0(0), respectively. In 1 patient local anesthesia protocol failed and intravenous sedation was used. All RezumTM therapies were performed successfully. Local anesthesia with Schelin® catheter opens up new possibilities for ultra-minimally invasive surgery for LUTS/BPO.

Religious Commodification in Social Experiments on Zavilda TV's YouTube Channel

Social media, including YouTube, has emerged as a significant aspect of contemporary lifestyles in society. Youtubers need to be able to present interesting content ideas in order to attract more viewers and advertisers. Zavilda TV YouTube Channel turns Islam into a content idea and commodifies it to attract more viewers. Religion and spirituality are no longer something exclusive, but have become a commodity and a public broadcast, used as an object and capital in the political economy of the media. Thus, this study aims to uncover realities that reflect the practice of commodification of religion in the Islamic social experiment targeting women. This research uses a qualitative approach with a literature review. Dialogue manuscripts, titles, video descriptions and thumbnails for Zavilda TV are research objects that are studied using observation and documentation methods. The study of the political economy of media and critical discourse analysis are used to uncover and explain commodification in a more critical and in-depth way. The results found in this study are forms of religious commodification in the form of religious symbols and Islamic songs, as well as religious reflections that are given to the target to make them willing to wear the headscarf. The issue of religious tolerance is also included in the content by targeting women of religions other than Islam to try wearing the headscarf. There is a relationship that is lamely built through the clothing of talent and targets like preachers and mad'u. Zavilda TV attempts to disseminate its understanding of religion. The channel asserts that its interpretation of religious doctrine is accurate and obligatory for all Muslim women, even if some of the remarks given tend to discriminate.

Science communicators, flat-eathers, or fitness coaches: who is citing scientific publications in youtube video descriptions?

In this study, we undertake an extensive analysis of YouTube channels that reference research publications in their video descriptions, offering a unique insight into the intersection of digital media and academia. Our investigation focuses on identifying the actors behind YouTube channels that reference scholarly publications, and analyzing the types of content these actors share. Specifically we are interested in three principal aspects: the background of YouTube channel actors, their thematic focus, and the nature of their operational dynamics, specifically addressing whether they work individually or in groups. Our results highlight a strong emphasis on content related to science and engineering, as well as health, particularly in channels managed by individual researchers and academic institutions. However, there is a notable variation in the popularity of these channels, with professional YouTubers and commercial media entities often outperforming in terms of viewer engagement metrics like likes, comments, and views. This underscores the challenge academic channels face in attracting a wider audience. Further, we explore the role of academic actors on YouTube, scrutinizing their impact in disseminating research and the types of publications they reference. Despite a general inclination towards professional academic topics, these channels displayed a varied effectiveness in spotlighting highly cited research. Often, they referenced a wide array of publications, indicating a diverse but not necessarily impact-focused approach to content selection.

I-DINO: High-Quality Object Detection for Indoor Scenes

Object Detection in Complex Indoor Scenes is designed to identify and categorize objects in indoor settings, with applications in areas such as smart homes, security surveillance, and home service robots. It forms the basis for advanced visual tasks including visual question answering, video description generation, and instance segmentation. Nonetheless, the task faces substantial hurdles due to background clutter, overlapping objects, and significant size differences. To tackle these challenges, this study introduces an indoor object detection approach utilizing an enhanced DINO framework. To cater to the needs of indoor object detection, an Indoor-COCO dataset was developed from the COCO object detection dataset. The model incorporates an advanced Res2net as the backbone feature extraction network, complemented by a deformable attention mechanism to better capture detailed object features. An upgraded Bi-FPN module is employed to replace the conventional feature fusion module, and SIoU loss is utilized to expedite convergence. The experimental outcomes indicate that the refined model attains an mAP of 62.3%, marking a 5.2% improvement over the baseline model. These findings illustrate that the DINO-based indoor object detection model exhibits robust generalization abilities and practical utility for multi-scale object detection in complex environments.

A Riemannian multimodal representation to classify parkinsonism-related patterns from noninvasive observations of gait and eye movements.

Parkinson's disease is a neurodegenerative disorder principally manifested as motor disabilities. In clinical practice, diagnostic rating scales are available for broadly measuring, classifying, and characterizing the disease progression. Nonetheless, these scales depend on the specialist's expertise, introducing a high degree of subjectivity. Thus, diagnosis and motor stage identification may be affected by misinterpretation, leading to incorrect or misguided treatments. This work addresses how to learn multimodal representations based on compact gait and eye motion descriptors whose fusion improves disease diagnosis prediction. This work introduces a noninvasive multimodal strategy that combines gait and ocular pursuit motion modalities into a geometrical Riemannian Neural Network for PD quantification and diagnostic support. Markerless gait and ocular pursuit videos were first recorded as Parkinson's observations, which are represented at each frame by a set of frame convolutional deep features. Then, Riemannian means are computed per modality using frame-level covariances coded from convolutional deep features. Thus, a geometrical learning representation is adjusted by Riemannian means, following early, intermediate, and late fusion alternatives. The adjusted Riemannian manifold combines input modalities to obtain PD prediction. The geometrical multimodal approach was validated in a study involving 13 control subjects and 19 PD patients, achieving a mean accuracy of 96% for early and intermediate fusion and 92% for late fusion, increasing the unimodal accuracy results obtained in the gait and eye movement modalities by 6 and 8%, respectively. The proposed method was able to discriminate Parkinson's patients from healthy subjects using multimodal geometrical configurations based on covariances descriptors. The covariance representation of video descriptors is highly compact (with an input size of 625 and an output size of 256 (1 BiRe)), facilitating efficient learning with a small number of samples, a crucial aspect in medical applications.

North Korean propaganda and counterpropaganda on YouTube

Abstract Drawn from attribute agenda setting theory and propaganda theory, this study examined an attribute agenda structure of YouTube videos on North Korean propaganda and its relationship with social media network indicators. A topic modeling analysis with 2,538 video descriptions clustered bulk documents into four attribute agendas: proximity, prominence, conflicts, and common interest. Proximity related North Korean propaganda to its main stakeholders such as China. Prominence was concerned with the relationship between North Korea and business. Conflict showcased North Korea’s military power as a confronting mode against South Korea. Common interest covered daily attractions in North Korea, including music and tourism. A positive relationship between conflict and all network strength indicators implies that YouTube descriptions of North Korea are related to military confrontations, showing belligerence as a notable attribute. Four topics and their linkage with network ties suggest that, despite North Korea’s efforts to build a soft image via YouTube, influencers set conflicts as key agenda attributes.

Extremely compact video representation for efficient near-duplicates detection

Extremely compact video representation for efficient near-duplicates detection

Video Multimodal Entity Linking via Multi-Perspective Enhanced Subgraph Contrastive Network

Video Multimodal Entity Linking (VMEL) is a task to link entities mentioned in videos to entities in multimodal knowledge bases. However, current entity linking methods primarily focus on text and image modalities, neglecting the significance of video modality. To address this challenge, we propose a novel framework called the multi-perspective enhanced Subgraph Contrastive Network (SCMEL) and construct a VMEL dataset named SceneMEL, based on tourism domain. We first integrate textual, auditory and visual modal contexts of videos to generate a comprehensive high-recall candidate entity set. Furthermore, a semantic-enhanced video description subgraph generation module is utilized to convert videos into a multimodal feature graph structure and perform subgraph sampling on the domain-specific knowledge graph. Lastly, we conduct contrastive learning on local perspectives (text, audio, visual) within the video subgraphs and the knowledge graph subgraphs, as well as global perspectives, to capture fine-grained semantic information about videos and entities. A series of experimental results on SceneMel demonstrate the effectiveness of the proposed approach.

Prehistoric Recycling Explained in a Playful Way: The Pfahlbauten Wimmelbild—An Interactive Digital Mediation Tool Designed by Young People

With the “Talents Internship program” established by the Austrian Research Promotion Agency (FFG, Österreichische Forschungsförderungsgesellschaft), 14- to 17-year-old students from various school types visited the Natural History Museum and the Kuratorium Pfahlbauten in summer 2022 to gain practical experience in research. The internship focused on a sustainability approach, discussing recycling methods, the sustainable use of resources and the circular economy in prehistory. The UNESCO World Heritage “Prehistoric Pile Dwellings around the Alps” was used as a research area for the project. The project also aimed to make the content developed by the students available to the public as a digital media tool. The pupils brought an illustration of a prehistoric lake shore settlement to life and created an interactive image available at the website of Kuratorium Pfahlbauten. Various scenes of the illustration have been augmented with animations created by the students of HTL Spengergasse in Vienna. Students from federal secondary schools from Vienna (Stubenbastei) and Upper Austria (Traun) researched the information about the objects and wrote texts that, as a description of the animated videos, introduce the users to the prehistoric artifact and explain the recycling process behind it. The students worked independently using the scientific literature, 140-year-old inventory books and 6000-year-old objects from the collection of the Natural History Museum Vienna. The activities and the supporting program within the internship were recorded by the students in blog posts, available at the Pfahlbauten-Blog. The co-creative approach of the FFG Talent Internship made it possible to introduce a group of school students to the process of scientific work and the communication of results. It was honored with the Creative App Award at CHNT 2023.

Help-Seeking Situations Related to Visual Interactions on Mobile Platforms and Recommended Designs for Blind and Visually Impaired Users.

While it is common for blind and visually impaired (BVI) users to use mobile devices to search for information, little research has explored the accessibility issues they encounter in their interactions with information retrieval systems, in particular digital libraries (DLs). This study represents one of the most comprehensive research projects, investigating accessibility issues, especially help-seeking situations BVI users face in their DL search processes. One hundred and twenty BVI users were recruited to search for information in six DLs on four types of mobile devices (iPhone, iPad, Android phone, and Android tablet), and multiple data collection methods were employed: questionnaires, think-aloud protocols, transaction logs, and interviews. This paper reports part of a large-scale study, including the categories of help-seeking situations BVI users face in their interactions with DLs, focusing on seven types of help-seeking situations related to visual interactions on mobile platforms: difficulty finding a toggle-based search feature, difficulty understanding a video feature, difficulty navigating items on paginated sections, difficulty distinguishing collection labels from thumbnails, difficulty recognizing the content of images, difficulty recognizing the content of graphs, and difficulty interacting with multilayered windows. Moreover, corresponding design recommendations are also proposed: placing meaningful labels for icon-based features in an easy-to-access location, offering intuitive and informative video descriptions for video players, providing structure information about a paginated section, separating collection/item titles from thumbnail descriptions, incorporating artificial intelligence image/graph recognition mechanisms, and limiting screen reader interactions to active windows. Additionally, the limitations of the study and future research are discussed.

Evaluating the Adequacy of YouTube Videos for Learning Laparoscopic Right Hemicolectomy With Complete Mesocolic Excision.

Complete mesocolic excision (CME) is becoming an oncological surgical principle day by day for right hemicolectomy. However, the procedure is technically difficult and carries a higher risk of complications than open surgery. In this study, the adequacy of YouTube videos that facilitate education for laparoscopic right hemicolectomy with complete mesocolic excision (LRHCME) was investigated. In July 2024, in the search bar of the YouTube platform, the term "laparoscopic right hemicolectomy complete mesocolic excision" was searched. The first 100 videos in each search were evaluated. Animations,advertisements, lectures, non-surgical videos (pre-surgery, post-surgery vlog, etc.), and non-English videoswere excluded from the study. Steps identified in the Delphi consensus were used to determine the reliability of the videos. The quality of the videos was measured using the Global Quality Scale (GQS) and the modified DISCERN score. Seventy videos were included in the evaluation. While 28 (40%) of these videos were classified as reliable, 42 (60%) were not found reliable. In reliable videos, video description, HD resolution, GQS, modified DISCERN, and duration were significantly higher(p-value <0.001, 0.012, <0.001,<0.001, 0.041 respectively). Reliable videos had a better rank than unreliable videos (p=0.046). When evaluated according to Delphi consensus, the most of LRHCME videos on the YouTube platform were unreliable. We conclude that YouTube alone is insufficient for learning LRHCME without a professional instructor.

Volleyball training video classification description using the BiLSTM fusion attention mechanism

This study aims to explore methods for classifying and describing volleyball training videos using deep learning techniques. By developing an innovative model that integrates Bi-directional Long Short-Term Memory (BiLSTM) and attention mechanisms, referred to BiLSTM-Multimodal Attention Fusion Temporal Classification (BiLSTM-MAFTC), the study enhances the accuracy and efficiency of volleyball video content analysis. Initially, the model encodes features from various modalities into feature vectors, capturing different types of information such as positional and modal data. The BiLSTM network is then used to model multi-modal temporal information, while spatial and channel attention mechanisms are incorporated to form a dual-attention module. This module establishes correlations between different modality features, extracting valuable information from each modality and uncovering complementary information across modalities. Extensive experiments validate the method's effectiveness and state-of-the-art performance. Compared to conventional recurrent neural network algorithms, the model achieves recognition accuracies exceeding 95% under Top-1 and Top-5 metrics for action recognition, with a recognition speed of 0.04s per video. The study demonstrates that the model can effectively process and analyze multimodal temporal information, including athlete movements, positional relationships on the court, and ball trajectories. Consequently, precise classification and description of volleyball training videos are achieved. This advancement significantly enhances the efficiency of coaches and athletes in volleyball training and provides valuable insights for broader sports video analysis research.

Building an Open-Vocabulary Video CLIP Model With Better Architectures, Optimization and Data.

Despite significant results achieved by Contrastive Language-Image Pretraining (CLIP) in zero-shot image recognition, limited effort has been made exploring its potential for zero-shot video recognition. This paper presents Open-VCLIP++, a simple yet effective framework that adapts CLIP to a strong zero-shot video classifier, capable of identifying novel actions and events during testing. Open-VCLIP++ minimally modifies CLIP to capture spatial-temporal relationships in videos, thereby creating a specialized video classifier while striving for generalization. We formally demonstrate that training Open-VCLIP++ is tantamount to continual learning with zero historical data. To address this problem, we introduce Interpolated Weight Optimization, a technique that leverages the advantages of weight interpolation during both training and testing. Furthermore, we build upon large language models to produce fine-grained video descriptions. These detailed descriptions are further aligned with video features, facilitating a better transfer of CLIP to the video domain. Our approach is evaluated on three widely used action recognition datasets, following a variety of zero-shot evaluation protocols. The results demonstrate that our method surpasses existing state-of-the-art techniques by significant margins. Specifically, we achieve zero-shot accuracy scores of 88.1%, 58.7%, and 81.2% on UCF, HMDB, and Kinetics-600 datasets respectively, outpacing the best-performing alternative methods by 8.5%, 8.2%, and 12.3%. We also evaluate our approach on the MSR-VTT video-text retrieval dataset, where it delivers competitive video-to-text and text-to-video retrieval performance, while utilizing substantially less fine-tuning data compared to other methods.

Performing the Garcia Score for Accuracy and Reliability

Reliable, sensitive, and accurate tests are needed to assess animal models of brain injury. The Garcia score is a neuro behavioral measure that has been used in many murine studies. However, despite its widespread use, there are no detailed video descriptions of the steps to properly perform and grade the Garcia test on rats. Consequently, there has been significant variation in its performance and reliability, calling for greater standardization. The Garcia score is comprised of six measures: spontaneous activity, symmetry in the four limbs, fore paw outstretching, climbing, body sensation, and response to vibrissae touch. Each component is scored with a minimum of zero or one and a maximum of three, with the highest total score of 18. This report systematically and clearly describes how each component of the Garcia score is performed and graded with an accompanying video illustration. The purpose of this report is to assist researchers in implementing the Garcia test and to help standardize the use of this measure across studies.

Evaluation of Sector 6 Citarum Harum's Social Media for Waste Management

The study aims to analyze the utilization of information media platforms by the Citarum Harum Task Force Sector 6 in waste management endeavors within the Baleendah District, Bandung Regency. The platforms under scrutiny include Instagram (@guardian_sektor6), YouTube (guardian_citarum), and TikTok (guardian_citarum). Employing a qualitative approach, the study integrates interviews, direct observations, and document analysis. Findings underscore the imperative of developing an official website to enhance information dissemination and organization. Notably, while the Instagram account @guardian_sektor6 demonstrates consistent posting and visually appealing content, shortcomings include inconsistent hashtag usage and limited content diversity. Similarly, the YouTube channel guardian_citarum maintains posting consistency but lacks optimization in SEO utilization and incomplete video descriptions. Meanwhile, TikTok account guardian_citarum, though active and visually engaging, underutilizes features such as hashtags and User-Generated Content. Consequently, the research output comprises a wireframe layout design for the official website of the Citarum Harum Task Force Sector 6. It is envisaged that this wireframe design will amplify the effectiveness of information delivery and waste management socialization efforts in the areas overseen by the Citarum Harum Task Force Sector 6.

Language for Specific Purposes and Audio Description Tasks: A Case Study

In the Foreign Language (FL) classroom, video description and dubbing have recently been introduced for the development of communicative skills in various types of didactic methods (Vermeulen and Ibáñez 2014, 2017; Talaván 2020; Ávila-Cabrera and Rodríguez-Arancón 2021). This paper offers an innovative didactic proposal of Audio Description (AD) in the English for Tourism (ET) classroom. It supports the hypothesis that authentic tasks enhance communicative skills in LSP (Language for Specific Purposes) education (Melnichuk et al. 2017). It presents a quasi-experimental study of student performance in two different productive tasks (writing and speaking). The study follows a Task-Based Learning (TBL) approach and addresses a group of 119 students within an undergraduate program of ET (B2 level) at a Spanish distance-education university (UNED). As a post-task, a twofold rubric, designed in a previous teaching project, scored and analyzed student writing and speaking responses to an AD script of a mute video about a touristic place of their choice. Results show that implementing an AD task in the LSP classroom had a positive effect on students’ marks in their final test. In addition, the number of students who completed the writing task was around 50% greater than those who completed the speaking task. Introducing ICT tools in the LSP classroom seems to be positive, and it will be argued that developing digital technologies, such as mobile apps, may be a necessary step to foster future professionals’ language performance and competences.

Is YouTube Reliable to Teach Laparoscopic Gastric Bypass?

The increasing prevalence of obesity has led to the popularity of bariatric surgery. Laparoscopic Roux-en-Y gastric bypass (LRYGB) is one of the most complex methods in bariatric surgery. The main steps of LRYGB were determined in the Delphi Consensus. This study investigated the instructiveness and reliability of YouTube videos about LRYGB based on the Delphi Consensus. In February 2024, three different searches were done in the search bar of the YouTube platform with the terms "laparoscopic gastric bypass" "laparoscopic Roux-en-Y gastric bypass" and "laparoscopic RYGB". The first 50 videos in each search were evaluated. Animations, lectures, advertisements, non-English videos, and non-surgical videos (pre-surgery, post-surgery vlog, etc.) were excluded from the study. Delphi consensus steps were used to determine the reliability of the videos. The quality of the videos was measured using the Global Quality Scale (GQS) and modified DISCERN test. Forty-five videos were included in the evaluation. While 14 (31.1%) of these videos were classified as reliable, 31 (68.8%) were not found reliable. In reliable videos, video description, high definition (HD) resolution, GQS, and modified DISCERNwere significantly higher(p-value 0.023, 0.004, 0.017, and 0.025 respectively). The rate of unreliable videos was higher on the YouTube platform. We conclude that YouTube alone is insufficient to learn LRYGB.

Differentiation in social perception: Why later-encountered individuals are described more negatively.

According to the cognitive-ecological model of social perception, biases toward individuals can arise as by-products of cognitive principles that interact with the information ecology. The present work tested whether negatively biased person descriptions occur as by-products of cognitive differentiation. Later-encountered persons are described by their distinct attributes that differentiate them from earlier-encountered persons. Because distinct attributes tend to be negative, serial person descriptions should become increasingly negative. We found our predictions confirmed in six studies. In Study 1, descriptions of representatively sampled persons became increasingly distinct and negative with increasing serial positions of the target person. Study 2 eliminated this pattern of results by instructing perceivers to assimilate rather than differentiate a series of targets. Study 3 generalized the pattern from one-word descriptions of still photos of targets to multisentence descriptions of videos of targets. In line with the cognitive-ecological model, Studies 4-5b found that the relation between serial position and negativity was amplified among targets with similar positive attributes, zero among targets with distinct positive or negative attributes, and reversed among similar negative targets. Study 6 returned to representatively sampled targets and generalized the serial position-negativity effect from descriptions of the targets to overall evaluations of them. In sum, the present research provides strong evidence for the explanatory power of the cognitive-ecological model of social perception. We discuss theoretical and practical implications. It may pay off to appear early in an evaluation sequence. (PsycInfo Database Record (c) 2024 APA, all rights reserved).