Audiovisual Modeling Research Articles

Audio–Visual Modeling and its Effect on Behavior of Pediatric Patient: An Observational Study

ABSTRACT Background: The prognosis and patients’ cooperation are interdependent on each other. In case of children, this aspect becomes all the more highlighted because of heightened response to pain and anxiety. Material and Methodology: Hundred (4-to-10 years old) children who were rated as negative on Frankl Behavior Rating Scale were taken for the study. The children were shown a video of a positive child model, and also various procedures of the dental treatment were shown. Results: Sixty-one children out of 100 demonstrated positive behavior and 3 showed definitely positive behavior during preventive treatment procedures. On the other hand, 73 children out of 100 demonstrated positive behavior and 3 showed definitely positive behavior during main treatment procedures. Significant change was observed in behavior of children in before and after the video was shown to them (<0.001). Conclusion: Audio–visual modeling has been proved to be a very efficient method in developing positive behavior of children toward dental treatment.

Improving Audio-Visual Segmentation with Bidirectional Generation

The aim of audio-visual segmentation (AVS) is to precisely differentiate audible objects within videos down to the pixel level. Traditional approaches often tackle this challenge by combining information from various modalities, where the contribution of each modality is implicitly or explicitly modeled. Nevertheless, the interconnections between different modalities tend to be overlooked in audio-visual modeling. In this paper, inspired by the human ability to mentally simulate the sound of an object and its visual appearance, we introduce a bidirectional generation framework. This framework establishes robust correlations between an object's visual characteristics and its associated sound, thereby enhancing the performance of AVS. To achieve this, we employ a visual-to-audio projection component that reconstructs audio features from object segmentation masks and minimizes reconstruction errors. Moreover, recognizing that many sounds are linked to object movements, we introduce an implicit volumetric motion estimation module to handle temporal dynamics that may be challenging to capture using conventional optical flow methods. To showcase the effectiveness of our approach, we conduct comprehensive experiments and analyses on the widely recognized AVSBench benchmark. As a result, we establish a new state-of-the-art performance level in the AVS benchmark, particularly excelling in the challenging MS3 subset which involves segmenting multiple sound sources. Code is released in: https://github.com/OpenNLPLab/AVS-bidirectional.

Effectiveness of Audio-visual Education through Digital Media Platforms Regarding Quality of Life Amongst Breast Cancer Person

Background: Improving the education system of patient-centered health information is particularly important for vulnerable populations. The magnitude of the attributable breast cancer burden is very essential to be a concern to developing a good management strategy. Improving the patient’s understanding of their conditions may enhance the quality of life following breast cancer. The study aimed to review the effectiveness of audio-visual education through digital media platforms to improve the quality of life among people with breast cancer.Method: These 1643 articles from eight databases i.e., Scopus, PubMed, MEDLINE via EBSCO, the Cochran Library, Springer Link, Wiley Online Library, BioMed Central, and Sage Journals were searched by a combination of medical subject heading (MESH) term, and unique references were examined. All studies evaluating audio-visual education of women with breast cancer delivered by digital media platforms were included.Results: The search yielded 14 articles with various digital media platforms, such as web-based, applications, e-mail, videoconference, YouTube, and WhatsApp. All studies reported improving quality of life, except one study using WhatsApp as the platform. Audio-visual interventions were variably effective in enhancing the quality of life of patients depending on their characteristics. Audio-visual interventions appear to be effective in improving quality of life amongst breast cancer person. Nevertheless, its effectiveness depends on the frequency and intensity of audio-visual delivery. Meanwhile, the contents of education materials must be adjusted to the administration method. Audio-visual modeling via digital platforms may facilitate improving quality of life and can be an important consideration in future health-education interventions.

Conscious Multisensory Integration: Introducing a Universal Contextual Field in Biological and Deep Artificial Neural Networks.

Conscious awareness plays a major role in human cognition and adaptive behavior, though its function in multisensory integration is not yet fully understood, hence, questions remain: How does the brain integrate the incoming multisensory signals with respect to different external environments? How are the roles of these multisensory signals defined to adhere to the anticipated behavioral-constraint of the environment? This work seeks to articulate a novel theory on conscious multisensory integration (CMI) that addresses the aforementioned research challenges. Specifically, the well-established contextual field (CF) in pyramidal cells and coherent infomax theory (Kay et al., 1998; Kay and Phillips, 2011) is split into two functionally distinctive integrated input fields: local contextual field (LCF) and universal contextual field (UCF). LCF defines the modulatory sensory signal coming from some other parts of the brain (in principle from anywhere in space-time) and UCF defines the outside environment and anticipated behavior (based on past learning and reasoning). Both LCF and UCF are integrated with the receptive field (RF) to develop a new class of contextually-adaptive neuron (CAN), which adapts to changing environments. The proposed theory is evaluated using human contextual audio-visual (AV) speech modeling. Simulation results provide new insights into contextual modulation and selective multisensory information amplification/suppression. The central hypothesis reviewed here suggests that the pyramidal cell, in addition to the classical excitatory and inhibitory signals, receives LCF and UCF inputs. The UCF (as a steering force or tuner) plays a decisive role in precisely selecting whether to amplify/suppress the transmission of relevant/irrelevant feedforward signals, without changing the content e.g., which information is worth paying more attention to? This, as opposed to, unconditional excitatory and inhibitory activity in existing deep neural networks (DNNs), is called conditional amplification/suppression.

Audiovisual modeling: An efficient, time-saving, radiation-specified method of reducing dental anxiety in children undergoing panoramic radiographic imaging and IOPA radiographic imaging

Background: Children undergoing panoramic radiographic imaging (PRI) and intraoral periapical (IOPA) radiographic imaging show anxiety due to the presence of a revolving PRI machine and IOPA X-ray unit and a different environment which can pose behavioral concerns. Staying still for 15–20 s is not possible for young children. This could lead to distortions and require repetitions, leading to more radiation exposure. Aim: The aim of the study was to reduce the dental anxiety and reduce the increased radiation exposure and saving time in PRI and IOPA radiographic imaging by audiovisual modeling. Methodology: Forty children age 2–12 years indicated for PRI and IOPA radiographic imaging were subjected and observed before and after audiovisual modeling. The patients were modeled by audiovisual modeling before imaging in the oral radiology department. The anxious and disruptive behavior was assessed using the anxious and disruptive behavior code by a single blinded observer. Statistical Analysis Used: Statistical Software SPSS 13.0. Results: There was significant difference in the occurrence of body movement and complaining and restraint before and after audiovisual modeling in the same patients. There were statistically significant differences in the overall events before and after audiovisual modeling. Conclusion: The basic behavior management technique and use of audiovisual modeling can reduce the anxiety and disruptive behavior in children undergoing PRI and IOPA radiographic imaging and also reduces increased radiation exposure, saving time for the process thus reducing the cost of repeated imaging.

Effect of Audiovisual Modeling on 5–10-year-old Children\'s Anxiety and Cooperation Behavior in First and Second Dental Visits

Effect of Audiovisual Modeling on 5–10-year-old Children\'s Anxiety and Cooperation Behavior in First and Second Dental Visits

Joint Audiovisual Hidden Semi-Markov Model-Based Speech Synthesis

This paper investigates joint speaker-dependent audiovisual Hidden Semi-Markov Models (HSMM) where the visual models produce a sequence of 3D motion tracking data that is used to animate a talking head and the acoustic models are used for speech synthesis. Different acoustic, visual, and joint audiovisual models for four different Austrian German speakers were trained and we show that the joint models perform better compared to other approaches in terms of synchronization quality of the synthesized visual speech. In addition, a detailed analysis of the acoustic and visual alignment is provided for the different models. Importantly, the joint audiovisual modeling does not decrease the acoustic synthetic speech quality compared to acoustic-only modeling so that there is a clear advantage in the common duration model of the joint audiovisual modeling approach that is used for synchronizing acoustic and visual parameter sequences. Finally, it provides a model that integrates the visual and acoustic speech dynamics.

Effectiveness of audiovisual modeling on the behavioral change toward oral and dental care in children with autism

Background/aimThe purpose of this study was to evaluate the effectiveness of audiovisual modeling on behavioral change of Autistic children toward oral and dental care. MethodThis prospective study was performed on 36 autistic children aged from 6 to 12 years old attending The “Egyptian Autistic Society (EAS).” Children participated in tooth brushing intervention program. They were randomly divided in to 3 different groups according to modeling methods of tooth brushing. Children in the first group were subjected to demonstration of tooth brushing steps using model representing upper and lower jaws, children in the second group were subjected to specific colored pictures explaining tooth brushing steps (pictorial Activity schedule (PAS)) and children in the third group were subjected to video modeling (Peer model brushing his teeth). Behavior of the children toward tooth brushing before and after running the program and the dental health knowledge and attitudes of parents toward dental care were assessed using a special questionnaire completed by parents and teachers of the children. Final evaluation was done after one month. ResultsThe result of the study showed that: no statistically significant change in behavior of the children after the program in group A there was a statistically significant change in behavior of the children after program in Group B (PAS model) and Group C (video model). ConclusionVisual pedagogy and Technology-based intervention is useful tool for helping Autistic children to improve their oral hygiene.

Child′s dental fear: Cause related factors and the influence of audiovisual modeling

Delivery of effective dental treatment to a child patient requires thorough knowledge to recognize dental fear and its management by the application of behavioral management techniques. Children's Fear Survey Schedule - Dental Subscale (CFSS-DS) helps in identification of specific stimuli which provoke fear in children with regard to dental situation. Audiovisual modeling can be successfully used in pediatric dental practice. To assess the degree of fear provoked by various stimuli in the dental office and to evaluate the effect of audiovisual modeling on dental fear of children using CFSS-DS. Ninety children were divided equally into experimental (group I) and control (group II) groups and were assessed in two visits for their degree of fear and the effect of audiovisual modeling, with the help of CFSS-DS. The most fear-provoking stimulus for children was injection and the least was to open the mouth and having somebody look at them. There was no statistically significant difference in the overall mean CFSS-DS scores between the two groups during the initial session (P > 0.05). However, in the final session, a statistically significant difference was observed in the overall mean fear scores between the groups (P < 0.01). Significant improvement was seen in group I, while no significant change was noted in case of group II. Audiovisual modeling resulted in a significant reduction of overall fear as well as specific fear in relation to most of the items. A significant reduction of fear toward dentists, doctors in general, injections, being looked at, the sight, sounds, and act of the dentist drilling, and having the nurse clean their teeth was observed.

A Fuzzy-Logic Mapper for Audiovisual Media

Recent technological developments have enabled us to synthesize images and sounds concurrently within single computers, even in real time, giving birth to novel and genuinely integrated audiovisual art forms (Hunt et al. 1998). But how should we organize and compose such works? Given certain soundscape, what would form an appropriate sequence of images to that soundscape? Given certain sequence of images, what soundscape is appropriate to it? If the image sequence and the soundscape are being created concurrently, how should we compose them? Authors have proposed different approaches to these questions (Whitney 1980; Hunt et al. 1998; Lokki et al. 1998; Rudi 1998; Kim and Lipscomb 2003; Gerhard and Hepting 2004; Yeo et al. 2004). These approaches differ significantly, and they are based on diverse principles, such as correspondence of aural to visual harmony, audiovisual modeling of mathematical principles, audiovisual rendering, data sonification, algorithmic control, and parameter space exploration. It is important to notice that there is no easy or correct solution, because the problem we must deal with lies in combining two entirely different media in time (Hunt et al. 1998). A fuzzy-logic approach to the challenge of composing both sound and moving image within coherent framework is proposed here as an alternative solution. This approach is based on fuzzy-logic model that enables flexible mapping of either aural or visual information onto the other, and it is able to generate complex audiovisual relationships by very simple means. This mapping strategy is inspired by two fundamental ideas: isomorphism and synaesthesia. Isomorphism applies when two complex structures can be mapped onto each other based on the fact that changes in one modality consistently cause changes in another modality (Hofstadter 1999). The word synaesthesia comes directly from the Greek syn (together) and aisthesis (perceive; Van Campen 1999), thus meaning a union of the senses. Synaesthesia occurs when stimulation in one sensory modality automatically triggers perception in second modality in the absence of any direct stimulation to this second modality (Harrison and Baron-Cohen 1997). This article is structured as follows. First, the motivations for this work are presented and discussed, including discussions of audiovisual domains, synaesthesia, and isomorphisms. Second, fuzzy logic is introduced, including its main features. Third, details of the proposed fuzzy-logic mapper are presented. Fourth, ID-FUSIONES (2001) and TIME EXPOSURE (2005), computer music-video works that use the proposed model, are discussed as actual implementations of the approach described in this article. Finally, conclusions and directions for future work are addressed.

The Effects of Audiovisual Modeling and Reading on Baccalaureate Students' Patient Teaching

The Effects of Audiovisual Modeling and Reading on Baccalaureate Students' Patient Teaching