Audiovisual Video Research Articles

Optimized deep learning enabled lecture audio video summarization

Video summarization plays an important role in multiple applications by compressing lengthy video content into compressed representation. The purpose is to present a fine-tuned deep model for lecture audio video summarization. Initially, the input lecture audio-visual video is taken from the dataset. Then, the video shot segmentation (slide segmentation) is done using the YCbCr space colour model. From each video shot, the audio and video within the video shot are segmented using the Honey Badger-based Bald Eagle Algorithm (HBBEA). The HBBEA is obtained by combining the Bald Eagle Search (BES) and Honey Badger Algorithm (HBA). The DRN training is executed by HBBEA to select the finest DRN weights. The relevant video frames are merged with the audio. The proposed HBBEA-based DRN outperformed with a better F1-Score of 91.9 %, Negative predictive value (NPV) of 89.6 %, Positive predictive value (PPV) of 90.7 %, Accuracy of 91.8 %, precision of 91 %, and recall of 92.8 %.

Toward Long Form Audio-Visual Video Understanding

We live in a world filled with never-ending streams of multimodal information. As a more natural recording of the real scenario, long form audio-visual videos (LFAVs) are expected as an important bridge for better exploring and understanding the world. In this article, we propose the multisensory temporal event localization task in long form videos and strive to tackle the associated challenges. To facilitate this study, we first collect a large-scale LFAV dataset with 5,175 videos and an average video length of 210 seconds. Each collected video is elaborately annotated with diversified modality-aware events, in a long-range temporal sequence. We then propose an event-centric framework for localizing multisensory events as well as understanding their relations in long form videos. It includes three phases in different levels: snippet prediction phase to learn snippet features, event extraction phase to extract event-level features, and event interaction phase to study event relations. Experiments demonstrate that the proposed method, utilizing the new LFAV dataset, exhibits considerable effectiveness in localizing multiple modality-aware events within long form videos. We hope that our newly collected dataset and novel approach serve as a cornerstone for furthering research in the realm of LFAV understanding. Project page: https://gewu-lab.github.io/LFAV/ .

Developing Mothers' Knowledge about Weaning Food in Infants Aged 6-24 Months through Video

Background: During this period, infants and children receive food and drink with an appropriate nutritional intake for optimal growth and development. Media such as audiovisual videos make it easier to increase maternal knowledge. This study was conducted to determine the effect of video on rising mothers' knowledge about weaning food in infants 6-24 months at Posyandu (Integrated Service Center). Methods: The type of research was quantitative, which is experimental with a one-group pre-post-test design approach. This study's population was mothers with children aged 6-24 months, using the total sampling method. Data collection used a questionnaire as an instrument that contained information on complementary foods. Data analysis used univariate and bivariate analysis with the Wilcoxon test. Results: the results showed a significant influence between weaning food education through video about maternal knowledge obtained with p-value=0.001 <a=0.05. Conclusion: There was a significant difference in maternal knowledge before and after being given an educational video intervention on complementary foods, which means that educational video affects increasing maternal knowledge about complementary foods in infants aged 6-26 months in the village.

A short simulated nature experience as an effective way to promote restoration from work-related stress.

Spending time in nature, and even watching images or videos of nature, has positive effects on one's mental state. However, cognitively stressful work is often performed indoors, in offices that lack easy access to nature during breaks. In this study, we investigated whether watching a 5-min audiovisual video that describes a first-person perspective walk on a forest path could help to restore one's mental state after cognitive stress. Participants were asked to perform cognitive stressor tasks, after which they were shown either a nature walk video or a control video. Subjective restoration was measured using self-reports before and after the videos, while electrodermal activity (EDA) and electroencephalography (EEG) were measured during the video-watching session. The results showed that experiencing the nature walk video enhanced subjective restoration more than watching the control video. Arousal of the autonomic nervous system, measured using EDA, decreased more during the nature walk video than during the control video. Additionally, activity in the EEG's upper theta band (6-8 Hz) and lower alpha band (8-10 Hz) increased during the nature walk video, suggesting that it induced a relaxed state of mind. Interestingly, the participants' connection with nature moderated the effects of the nature video. The subjective and physiological measures both suggest that watching a short, simulated nature walk may be beneficial in relaxing the mind and restoring one's mental state after cognitive stress.

Audio-visual video based circuit training model development design in futsal sports

This study aimed to develop a circuit training model using audio-visual video media in futsal. The researcher used the R&D method. The subject possessed the characteristics of being male, aged between 20 and 30 years, with a Body Mass Index between 18 and 34, and was a futsal athlete. Before the study began, the researcher explained the research procedure, and the subjects filled out informed consent forms. This technique was used to analyze the data obtained from distributing questionnaires. Questionnaires were given to futsal experts, media experts, and coaching experts. The reliability test was carried out by 12 subjects from the small group test and 24 subjects from the large group test. The results of this development are in the form of videos; in these videos, there are 4 training models consisting of 4 posts each. The 4 posts are arranged in 1 round, with each post consisting of 3 athletes. In this training model, the athletes exercise at each post for 20 seconds; after completing one exercise model, they move to the next post. Exercises are done for 3 sets, with 1-minute recovery between sets. Based on the expert validity test, the evaluation results of futsal experts averaged 85.5%, media experts averaged 82.0%, and coaching experts averaged 83.3%. Furthermore, the results of the reliability test for the small group obtained a percentage of 88%, and for the large group, it was 87.87%. The development of a circuit training model in futsal games can help athletes become more enthusiastic and can also enable them to do additional exercises outside the schedule by using this video product as a training guide, providing athletes with many variations in their exercises.

Audiovisual Moments in Time: A large-scale annotated dataset of audiovisual actions.

We present Audiovisual Moments in Time (AVMIT), a large-scale dataset of audiovisual action events. In an extensive annotation task 11 participants labelled a subset of 3-second audiovisual videos from the Moments in Time dataset (MIT). For each trial, participants assessed whether the labelled audiovisual action event was present and whether it was the most prominent feature of the video. The dataset includes the annotation of 57,177 audiovisual videos, each independently evaluated by 3 of 11 trained participants. From this initial collection, we created a curated test set of 16 distinct action classes, with 60 videos each (960 videos). We also offer 2 sets of pre-computed audiovisual feature embeddings, using VGGish/YamNet for audio data and VGG16/EfficientNetB0 for visual data, thereby lowering the barrier to entry for audiovisual DNN research. We explored the advantages of AVMIT annotations and feature embeddings to improve performance on audiovisual event recognition. A series of 6 Recurrent Neural Networks (RNNs) were trained on either AVMIT-filtered audiovisual events or modality-agnostic events from MIT, and then tested on our audiovisual test set. In all RNNs, top 1 accuracy was increased by 2.71-5.94% by training exclusively on audiovisual events, even outweighing a three-fold increase in training data. Additionally, we introduce the Supervised Audiovisual Correspondence (SAVC) task whereby a classifier must discern whether audio and visual streams correspond to the same action label. We trained 6 RNNs on the SAVC task, with or without AVMIT-filtering, to explore whether AVMIT is helpful for cross-modal learning. In all RNNs, accuracy improved by 2.09-19.16% with AVMIT-filtered data. We anticipate that the newly annotated AVMIT dataset will serve as a valuable resource for research and comparative experiments involving computational models and human participants, specifically when addressing research questions where audiovisual correspondence is of critical importance.

Greenfluencing Through the Power of Emotions? Impact of Message Frames and Emotionally Matching Background Music on the Effectiveness of Influencers’ Environmental Communication

Social media influencers have become increasingly important in persuading people to become environmentally sensitive. As “greenfluencers” have been studied primarily in the context of advertising, it is crucial to investigate the mechanisms and effects of their non-product–related environmental messages. In two studies, we tested different message frames, a well-known persuasion strategy, in the context of Instagram reels (short audiovisual videos), where user-added background music is a prominent feature. In a 2 × 2 between-subjects online experiment ( N = 240), we manipulated the message frame (gain vs. loss) and the background music (positive vs. negative emotional valence) and found that loss frames reduced behavioral intention via negative emotions. A second 2 × 2 between-subjects online experiment ( N = 207) replicated these findings with an older sample and provided additional evidence that gain frames should be used in environmental communication. Contrary to our assumptions, we found no effect of the emotional valence of music in either study.

The Effect of Health Education Using Audiovisuals on The Knowledge of Adolescent Girls about Breast Self-Examination (BSE)

Cancer is a non-communicable disease with the second-largest death rate in the world. Breast cancer is one of the main diseases treated by palliative care services. Breast cancer is the most common cancer among women in the world. Early detection is a very important first step to find out early the presence of tumors or lumps in the breast to reduce the death rate due to cancer. The advantage of early detection of breast cancer is beneficial to increase the chances of life expectancy in women with breast cancer. Almost 85% of disorders or lumps are found by the patient through proper examination. In addition, BSE is the easiest, fastest, cheapest, and simplest method that can detect breast cancer early. Knowing the Effect of Health Education Using Audio Visual Media on Adolescent Girls' Knowledge about BSE at SMAN 10 Tangerang Regency Banten 2023. The research design used was quasi-experimental research with a one-group pretest-posttest design approach. The population used in this study was class X adolescent girls at SMAN 10 Tangerang Regency totaling 155 people. The number of samples taken was 30 samples obtained through the Proportionate Stratified Random Sampling technique. Data were collected using a questionnaire. Data analysis was done by Paired T Test. The results of univariate analysis before conducting health education using audio-visual video exposure were 10.67, while the average knowledge of adolescent girls increased after health education to 15.60. The results of bivariate analysis with paired sample t-test P value = 0.001 means that there is an effect of counseling with audiovisual media on increasing the knowledge of adolescent girls about breast self-examination. Using audiovisual media to increase knowledge about breast self-examination (BSE) for early detection of cervical cancer.

Hierarchical Augmentation and Distillation for Class Incremental Audio-Visual Video Recognition.

Audio-visual video recognition (AVVR) integrates audio and visual cues to accurately categorize videos. While current methods using provided datasets achieve satisfactory results, they face challenges in retaining historical class knowledge when new classes appear in real-world situations. There are no dedicated methods to address this issue, prompting this paper to explore Class Incremental Audio-Visual Video Recognition (CIAVVR). CIAVVR aims to preserve historical knowledge contained in stored data and learned models to prevent catastrophic forgetting. Audio-visual data and models inherently have hierarchical structures, where the model contains both low-level and high-level semantic information, and data includes snippet-level, video-level, and distribution-level spatial information. It is crucial to fully exploit these hierarchical structures for data knowledge preservation and model knowledge preservation. However, existing image class incremental learning methods do not explicitly consider these hierarchical structures. Therefore, we introduce Hierarchical Augmentation and Distillation (HAD), which includes the Hierarchical Augmentation Module (HAM) and Hierarchical Distillation Module (HDM). These modules efficiently utilize the hierarchical structure of data and models. Specifically, HAM uses a novel augmentation strategy, segmental feature augmentation, to preserve hierarchical model knowledge. Simultaneously, HDM employs newly designed hierarchical logical distillation (video-distribution) and hierarchical correlative distillation (snippet-video) to maintain intra-sample and inter-sample hierarchical knowledge. Evaluations on four benchmarks (AVE, AVK-100, AVK-200, and AVK-400) show that HAD effectively captures hierarchical information, enhancing the preservation of historical class knowledge and performance. We also provide a theoretical analysis to support the segmental feature augmentation strategy.

Edukasi Pemenuhan Nutrisi sebagai Upaya Pencegahan Stunting menggunakan Media Audio Visual

Stunting is an irreversible growth disorder caused by a lack of nutritional intake during pregnancy (during 1,000 HPK), therefore, babies aged 0 to 2 years must receive balanced and optimal nutritional intake, because this period is the golden period of growth and development. child. Stunting is caused by a lack of understanding about how to overcome health problems (nutrition intake), therefore it is necessary to prevent stunting as an effort to increase health knowledge through education. Community service aims to find out how education about fulfilling nutrition in efforts to prevent stunting has an impact on increasing knowledge. The method of implementing counseling is carried out through face-to-face meetings and discussions which are considered more effective, where the activity begins with giving a pre-test questionnaire to measure the level of knowledge of mothers of toddlers, then the presentation of the material that takes place allows the audience to pay attention and focus on audio-visual videos regarding nutritional needs. To prevent stunting, evaluation of the results of the activity is carried out by administering a post-test questionnaire in the form of questions regarding the material that has been presented. This community service was carried out on September 18 2023 in Salekoe sub-district, East Wara sub-district, Palopo City, attended by 40 participants, all of whom were mothers of toddlers. The results showed that there was a significant influence on increasing respondents' knowledge before and after being given counseling with a p-value = 0.00 (p<0.05), where there were 20% of respondents who had good knowledge during the pre-test and this increased to 55% of respondents at the time of the post-test, 25% of respondents had sufficient knowledge about stunting at the time of the pre-test and this increased to 32.5% of respondents at the time of the post-test, 55% of respondents had insufficient knowledge at the time of the pre-test and decreased to 12.5% of respondents at the time post-test. The next follow-up is that health workers are expected to be able to use audio-visual videos in efforts to prevent stunting to increase knowledge among mothers of toddlers so that they can reduce the prevalence of stunting.

Neuronal Population Encoding of Identity in Primate Prefrontal Cortex.

The ventrolateral prefrontal cortex (VLPFC) shows robust activation during the perception of faces and voices. However, little is known about what categorical features of social stimuli drive neural activity in this region. Since perception of identity and expression are critical social functions, we examined whether neural responses to naturalistic stimuli were driven by these two categorical features in the prefrontal cortex. We recorded single neurons in the VLPFC, while two male rhesus macaques (Macaca mulatta) viewed short audiovisual videos of unfamiliar conspecifics making expressions of aggressive, affiliative, and neutral valence. Of the 285 neurons responsive to the audiovisual stimuli, 111 neurons had a main effect (two-way ANOVA) of identity, expression, or their interaction in their stimulus-related firing rates; however, decoding of expression and identity using single-unit firing rates rendered poor accuracy. Interestingly, when decoding from pseudo-populations of recorded neurons, the accuracy for both expression and identity increased with population size, suggesting that the population transmitted information relevant to both variables. Principal components analysis of mean population activity across time revealed that population responses to the same identity followed similar trajectories in the response space, facilitating segregation from other identities. Our results suggest that identity is a critical feature of social stimuli that dictates the structure of population activity in the VLPFC, during the perception of vocalizations and their corresponding facial expressions. These findings enhance our understanding of the role of the VLPFC in social behavior.

THE INFLUENCE OF AUDIO-VISUAL VIDEO MEDIA ON KNOWLEDGE IN EFFORTS TO PREVENT SEXUAL VIOLENCE AGAINST ADOLESCENTS WITH DISABILITIES AT SLB SHANTI KOSALA NGANJUK

Background: In Indonesia, teenagers account for one-quarter of the population. The focus of national development is on teenagers. Teenagers with disabilities are of particular concern due to their numerous restrictions, particularly in reproductive health education. They often experience health problems, especially sexual and reproductive health. Reproductive health education is the right solution for people with disabilities so that they can increase their knowledge in sexual violance so that they can protect themselves from a person. The aim of this research is to determine the effect of audio-visual video media on knowledge in efforts to prevent sexual violence among disabled teenagers. Method: We included all of the student that attend the school on the day of data collection. We used questionnaire to collect the data and the analytical test used was the Wilxocon Rank Test. Result: there was an increase in knowledge before implementing health education using audio-visual media, primarily 13.6%, with the high knowledge group comprising 54.5%. Meanwhile, the low knowledge group dropped from 86.4% to 45.5%. The Wilxocon Rank Test value is p= 0.035, indicating that audio-visual media has a significant influence on knowledge. Conclussion: There is an influence of audio-visual media with information in attempts to prevent sexual violence against disabled youth at SLB. Shanti Kosala Nganjuk.

Hate Speech against the Bima Tribe: A Semantic Study in Perspective for Forensic Linguistics

Language is one of the legal evidence tools that requires the field of linguistics to dissect disputed languages. The disputed language is inseparable from its ambiguous nature. Therefore, it is important that the language in question be analysed linguistically to help provide evidence in the judicial process. This paper aims to analyse speech as legal evidence. The study uses the methods of listening, recording, and using semantic approaches and forensic linguistics. The use of the listen-and-record method is necessary because the data is sourced from Saifudin Ibrahim's audio-visual video on Facebook and YouTube. The results of this study show that speech that has lexical meaning, such as people, humans, nations, or Bima, is speech that is directly intended by SI for its interlocutors. While the interlocutor referred to by SI is mentioned with words that are included in the referential meaning, such as the regent, his inferiority, Bima people, and brain, he is unable to express words or phrases referred to by SI as objects of discussion about his interlocutors. First, SI speech falls into several categories, primarily speech that attacks the honour of the Bima community by giving a negative assessment of the Bima community leader. Second, spreading fake news related to Bima society in the context of family and social psychology. Three utterances contain insults to the Bima tribe by degrading the ability or competence of the Bima people.

Create a Model to Detect Audiovisual Videos by Breaking Down Superscribing Tensor and Using Less Frequency and a Lower Ranking

Create a Model to Detect Audiovisual Videos by Breaking Down Superscribing Tensor and Using Less Frequency and a Lower Ranking

Pelatihan Pembuatan Video Audio Visual Sebagai Inovasi Pembelajaran, Media Ekspresi dan Penunjang Siswa pada Mata Pelajaran Seni Budaya Di SMP Negeri 1 Jatirogo

Practically, the subject of Culture and Arts at schools is closely related to the advancement of information technology. However, the utilization remains low, causing the instructional process to be less innovative. Knowledge of art and culture in the subject should be an expressive, interactive, and dynamic lesson if technology is used as one of the teaching-learning media. One example is the creation of audiovisual videos about art and culture. A training was created for selected junior high school students at SMP Negeri 1 Jatirogo to make audiovisual short films. The purpose is to teach knowledge on using one of the latest advancements in information technology, particularly audiovisual video, and a participative learning experience that differs from past techniques. The approach employed is participatory, with students serving as learning resources to improve soft skills and abilities in processing and applying concepts in audiovisual form and training to be autonomous and responsible in the learning process. The outcomes of this program provide new ideas for instructors, particularly Cultural and Arts teachers, to create further learning innovations for more engaging, expressive, and apprehensible class meetings through audiovisuals. Another outcome is that students gain diverse learning experiences, including an understanding of short filmmaking, which is another approach to incorporating arts into learning, particularly in Culture and Arts.

The effect of audio-visual video with korotkoff sounds on anxiety levels and blood pressure measurement skills of nursing students: A randomized controlled study

AimThis study was aimed to determine the effect of the audio-visual video created with korotkoff sounds on the anxiety levels and blood pressure measurement skills of nursing students. BackgroundBlood pressure measurement is the basis of nursing care and clinical decision. For this reason, it is stated that nursing students should specialize in this skill. DesignThis research was conducted as a randomized controlled experimental study. MethodThis research was conducted with 130 (intervention group: 67, control group: 63) nursing students in the nursing department of a university in March 2022. Before the skill practice, the students in the intervention group listened to the korotkoff sounds using an audio-visual video containing the korotkoff sounds and then the practice was made. Data of the study were collected using the short version of the State-Trait Anxiety Inventory, the student introduction form and the blood pressure measurement evaluation form. ResultsThe mean age of the students was 19.41 (SD 1.75) in the intervention group and 19.20 (SD 1.04) in the control group. The State-Trait Anxiety Inventory mean scores of both groups were similar before the skill practice. At the end of the skill practice, the state anxiety score of the intervention group 0.56 (SD 1.03) was lower than the control group 1.30 (SD 1.81) and the difference was statistically significant (p < 0.05). When the blood pressure measurements of the students in both groups were compared, the rate of hearing korotkoff sounds and measuring blood pressure correctly was higher in the students in the intervention group and the difference was statistically significant (p < 0.05). ConclusionThe korotkoff sounds presented with audio-visual video increased the skill levels of the students and reduced the anxiety level. In this direction; it is recommended to use audio-visual video containing korotkoff sounds for students to gain blood pressure measurement skills.

Effectiveness of Media-Based Health Education on Stunting Prevention in Adolescents: A Systematic Review

Efforts to promote stunting prevention from an early age can be carried out by media that is effective at targeting youth, where youth activities cannot be separated from the digital world. This systematic review uses a scoping review approach and aims to identify media as an intervention in efforts to prevent stunting in adolescents. Article searches were conducted through Google Scholar, ScienceDirect, and PubMed published in 2018 to 2022 with criteria of full text, open access and keywords such as ‘media AND prevention AND stunting AND effective AND teen’ found 16,508 articles which were then obtained based on the inclusion criteria 8 selected papers. This research describes 5 media interventions for adolescent stunting prevention, including audiovisual; Android applications, social media, leaflets, and booklets. Social media is the most influential media in preventing stunting in adolescents with an increase in the pre-test and post-test difference by TikTok (36.75%), Instagram (22.29%), Twitter (20.25%), Youtube (16.66%), and Facebook (12.61%). Audiovisual videos accompanied by explanations are the second medium that is effectively used, marked by an increase in knowledge from 50.9% to 76.6% compared to the use of videos without reasons. The media interventions have proven effective in increasing adolescents understanding of stunting prevention. Information dissemination to the public, especially youth, is expected to utilize digital media to increase adolescents’ knowledge of stunting prevention which is more effective and efficient.

Increasing Interest, Results and Ability to Listen to Stories Using Audio Visual Media in Elementary Schools

This study aims to increase students' interest and learning outcomes using audio-visual media on the listening skills of class III SD Negeri Grantung students. This study uses Classroom Action Research (CAR), which consists of two cycles. Each cycle consists of two meetings. The subjects in the research were Grade III students at SDN Grantung, totaling 22 students. Data collection methods used are interviews, observation and documentation with quantitative analysis techniques. The media used in this research is audio-visual. The stages are 1) The teacher plays an audio-visual video 2) students listen to the story 3) the teacher evaluates. Based on the results of the research that has been done about students' interest in learning increasing, it can be seen based on the results of the application of the method of listening to stories using audio-visual in the pre-cycle to get an average of 72.75%, then in cycle I get an average of 75.775% and cycle II get average 88.25% included in the high criteria. Furthermore, student learning outcomes increased from pre-cycle obtaining an average of 50%, then in the first cycle stage there was an increase with an average of 70.5% entering the medium category but in cycle II experiencing a fairly good increase to 87.5% entering in very good criteria.

The Sabancı University Dynamic Face Database (SUDFace): Development and validation of an audiovisual stimulus set of recited and free speeches with neutral facial expressions.

Faces convey a wide range of information, including one's identity, and emotional and mental states. Face perception is a major research topic in many research fields, such as cognitive science, social psychology, and neuroscience. Frequently, stimuli are selected from a range of available face databases. However, even though faces are highly dynamic, most databases consist of static face stimuli. Here, we introduce the Sabancı University Dynamic Face (SUDFace) database. The SUDFace database consists of 150 high-resolution audiovisual videos acquired in a controlled lab environment and stored with a resolution of 1920 × 1080 pixels at a frame rate of 60 Hz. The multimodal database consists of three videos of each human model in frontal view in three different conditions: vocalizing two scripted texts (conditions 1 and 2) and one Free Speech (condition 3). The main focus of the SUDFace database is to provide a large set of dynamic faces with neutral facial expressions and natural speech articulation. Variables such as face orientation, illumination, and accessories (piercings, earrings, facial hair, etc.) were kept constant across all stimuli. We provide detailed stimulus information, including facial features (pixel-wise calculations of face length, eye width, etc.) and speeches (e.g., duration of speech and repetitions). In two validation experiments, a total number of 227 participants rated each video on several psychological dimensions (e.g., neutralness and naturalness of expressions, valence, and the perceived mental states of the models) using Likert scales. The database is freely accessible for research purposes.

Toward a perceptive pretraining framework for Audio-Visual Video Parsing

Audio-Visual Video Parsing (AVVP) is a new multi-modal weakly supervised task which aims to detect and localize events leveraging the partial alignment of audio and visual streams and weak labels. We identified two significant challenges in the AVVP: Cross-mode semantic misalignment and Contextual audio-visual dataset bias. For challenge 1, the existing methods tend to leverage the temporal similarity of the features. However, it is inappropriate for our AVVP task because multi-modal features with the same label do not always have the same semantics. Thus, we propose an instance-adaptive multi-modal time series max-margin loss (MTSM) which uses the temporal information to align features adaptively. Furthermore, to restrict the inescapable noise introduced during the feature fusion, we reuse the expression of MTSM in the single-mode. For the second challenge, we argue that bias mitigation should seek help from model generalization. Thus, we propose collocating pre-trained models: either” traverse” or based on domain-adaptation. First, we prove a hypothesis and then propose a method based on the Alternating Direction Method of Multipliers(ADMM) to decouple the optimal pre-trained model collocation solution, which reduces the time consumption. Experiments show that our method outperforms the contrastive methods.