Audiovisual Network Research Articles

Russian-Language Media of YouTube: Trends of the “Fifth Power” in 2021

Purpose. Compared to 800 million users in 2012, global YouTube reached over 2 billion monthly active users in 2021. Just over a quarter of the world’s population visits YouTube every month. Worldwide, users watch over 1 billion hours of content every day. Russia is in the top five countries in 2021 in terms of the total estimated number of YouTube users – 58 million. According to Why Video, over 65 % of viewers perceive YouTube content as real life. Daily statistics show the scale of YouTube and it becomes clear that this is not just social media and video hosting, but a full-fledged “fifth power”. Results. Based on the analysis of 127 Russian-language media YouTube channels conducted in the fall-winter of 2021, as well as on expert interviews and monitoring of sociological research, the authors are trying to determine the vectors of development of the enormously popular platform.Conclusion. YouTube and audiovisual networks are becoming not only a means of procrastinating and entertaining viewers, but also an informational and educational source. Social media, and in particular YouTube, have established their own full-fledged media space with their own laws, trends, culture, fashion, etc. Most YouTubers create a completely competitive product without large-scale professional, especially television, production facilities, while their audience is many times greater than the television one. They re-invent journalistic genres that seem outdated on television and radio, raising hype about them. They earn money with the help of not only the YouTube platform but also advertising integrations. YouTubers grew into powerful media, developing a personal brand, choosing the most comfortable social media platforms for themselves, and successfully mastering new ones.

Class-Aware Sounding Objects Localization via Audiovisual Correspondence.

Audiovisual scenes are pervasive in our daily life. It is commonplace for humans to discriminatively localize different sounding objects but quite challenging for machines to achieve class-aware sounding objects localization without category annotations, i.e., localizing the sounding object and recognizing its category. To address this problem, we propose a two-stage step-by-step learning framework to localize and recognize sounding objects in complex audiovisual scenarios using only the correspondence between audio and vision. First, we propose to determine the sounding area via coarse-grained audiovisual correspondence in the single source cases. Then visual features in the sounding area are leveraged as candidate object representations to establish a category-representation object dictionary for expressive visual character extraction. We generate class-aware object localization maps in cocktail-party scenarios and use audiovisual correspondence to suppress silent areas by referring to this dictionary. Finally, we employ category-level audiovisual consistency as the supervision to achieve fine-grained audio and sounding object distribution alignment. Experiments on both realistic and synthesized videos show that our model is superior in localizing and recognizing objects as well as filtering out silent ones. We also transfer the learned audiovisual network into the unsupervised object detection task, obtaining reasonable performance.

Le leggi protettivi dei mercati cinematografici nazionali contemporanei in Italia e Brasile e il ruolo televisivo nello sviluppo delle loro produzioni

This article provides a general overview of two cinematographic markets, Italian and Brazilian, and discusses how in both cases a a legal protection system has been established for their regional audiovisual network. The local broadcasters are obliged to finance local productions or even buy local contents and insert them properly in their transmissions. A protective system has defended regional media, by means of law local producers, and thus increased the number of films available for audience, expanding their media structure while maintaining a quality in their content. This would not be possible without the support of television networks and streaming services. Between a combative stance and a profile based on lamentation there are many possibilities to be chosen by artists and producers that can be a prolific path for both whether for experimental and avantgarde filmmakers or for those truly focused on commercial pieces. The development of the laws regarding cultural production especially cinema and TV can put them in a complementary self-fostering field.

Seamless Multicast : an SDN-based architecture for continuous audiovisual transport

For audiovisual network operators, end-users satisfaction is a major issue. This is the case for TDF who operates a nationwide network in France whose main purpose is to carry Digital Terrestrial Television streams. Such audiovisual content is forwarded through multicast real-time streams which require continuity of service. Therefore, the main goal of this work is to define a new architecture to prevent impact during network healing time. The proposed architecture aims to use a pair of redundant multicast trees, and ensure their seamless resiliency. This architecture called “Seamless Multicast” takes advantage of the network-end equipment’s ability to receive and combine two identical streams, complete or not. The main contribution of this paper is the development and evaluation of an algorithm for the computation of a pair of multicast trees and the associated hitless deployment scheme. Implementation requires an Software-Defined Networking architecture, in which performance knowledge and bandwidth management are centralized in a controller. A proof of concept controller has been used for validation of the architecture’s global behaviour using a virtualized environment in multiple scenarios.

Binaural Audio-Visual Localization

Localizing sound sources in a visual scene has many important applications and quite a few traditional or learning-based methods have been proposed for this task. Humans have the ability to roughly localize sound sources within or beyond the range of the vision using their binaural system. However most existing methods use monaural audio, instead of binaural audio, as a modality to help the localization. In addition, prior works usually localize sound sources in the form of object-level bounding boxes in images or videos and evaluate the localization accuracy by examining the overlap between the ground-truth and predicted bounding boxes. This is too rough since a real sound source is often only a part of an object. In this paper, we propose a deep learning method for pixel-level sound source localization by leveraging both binaural recordings and the corresponding videos. Specifically, we design a novel Binaural Audio-Visual Network (BAVNet), which concurrently extracts and integrates features from binaural recordings and videos. We also propose a point-annotation strategy to construct pixel-level ground truth for network training and performance evaluation. Experimental results on Fair-Play and YT-Music datasets demonstrate the effectiveness of the proposed method and show that binaural audio can greatly improve the performance of localizing the sound sources, especially when the quality of the visual information is limited.

Audio-Visual Tensor Fusion Network for Piano Player Posture Classification

Playing the piano in the correct position is important because the correct position helps to produce good sound and prevents injuries. Many studies have been conducted in the field of piano playing posture recognition that combines various techniques. Most of these techniques are based on analyzing visual information. However, in the piano education field, it is essential to utilize audio information in addition to visual information due to the deep relationship between posture and sound. In this paper, we propose an audio-visual tensor fusion network (simply, AV-TFN) for piano performance posture classification. Unlike existing studies that used only visual information, the proposed method uses audio information to improve the accuracy in classifying the postures of professional and amateur pianists. For this, we first introduce a dataset called C3Pap (Classic piano performance postures of amateur and professionals) that contains actual piano performance videos in diverse environments. Furthermore, we propose a data structure that represents audio-visual information. The proposed data structure represents audio information on the color scale and visual information on the black and white scale for representing relativeness between them. We call this data structure an audio-visual tensor. Finally, we compare the performance of the proposed method with state-of-the-art approaches: VN (Visual Network), AN (Audio Network), AVN (Audio-Visual Network) with concatenation and attention techniques. The experiment results demonstrate that AV-TFN outperforms existing studies and, thus, can be effectively used in the classification of piano playing postures.

Audio-visual networks and the road to 5G

Multiple upcoming 5G use-cases seem to particularly depend on network deployment and role model questions. Service reliability, liability and ultimately adequate licensing of spectrum are likely to be the primary success factors, independent of features to be implemented in 5G new radio. Here, this article discusses standardization enabling different audio-visual production use-cases and flexible models within the 5G system.

Disrupted local neural activity and functional connectivity in subjective tinnitus patients: evidence from resting-state fMRI study.

The study aimed to investigate the abnormal alterations of both the intra-regional brain activity and inter-regional functional connectivity (FC) in patients with subjective tinnitus (ST) using resting-state functional MRI (rs-fMRI) methods. Twenty-five ST patients and 25 normal controls (NCs) were included and underwent resting-state functional magnetic resonance imaging scans. ReHo, fALFF, and seed-based FC were calculated and compared between ST patients and NCs. Meanwhile, correlation analyses were calculated between altered connectivity and clinical data in ST patients. Compared with NCs, ST patients exhibited increased ReHo and fALFF values in the right middle temporal gyrus (MTG), and the ReHo values were also increased in the right cuneus. In contrast, decreased ReHo values in ST patients were observed in the right middle frontal gyrus (MFG) and left cerebellar anterior lobe. Considering these brain areas with altered ReHo and fALFF clusters as seeds, the right MTG (ReHo) exhibited decreased connectivity with the right MFG, lingual gyrus, and left cerebellar posterior lobe, besides, the right cuneus showed decreased connectivity with the right MTG. In ST patients, the decreased FC between the right MTG (ReHo) and the right MFG was also positively correlated with the Tinnitus handicap inventory score (r = 0.675, P = 0.001). The present study revealed that ST patients had altered regional neural activity and inter-regional connectivity in partial auditory and non-auditory brain regions, mainly involving the default mode network and audio-visual network, which could further improve our understanding of the neuroimaging mechanism in ST.

Engaging and educating patients in prostate imaging via social media.

Prostate cancer (PCa) continues to be a major health problem. A key challenge in PCa management is the over-treatment of indolent disease and under-treatment of aggressive disease. Increasingly, this challenge is being addressed by decision making guided by multi-parametric MRI (mpMRI). In numerous studies, mpMRI has accurately detected and staged PCa, and also directed the accurate sampling of suspicious prostate lesions via a targeted approach [1, 2]. These investigations have brought new strategies for PCa management, including image-guided active surveillance and image-guided focal therapy, into clinical use. While PCa patients are exposed to these new clinical paradigms using MRI, they still may have an incomplete understanding, or ongoing questions, regarding the role of imaging in their disease [3]. Although radiologists, who perform and interpret such imaging, are well positioned to address these issues and enhance patients’ understanding of prostate MRI, they do not routinely meet with PCa patients whose imaging exams they interpret. One way for radiologists to reach and educate PCa patients regarding MRI is through radiologist-led direct public education sessions [4]. However, considering the large number of men at risk or already diagnosed with PCa, such in-person group education may be insufficient for reaching the majority of patients. Social media provides an additional valuable source of patient information. Social media alternatives, including microblogging networks (e.g., Twitter ), social networks (e.g., Facebook ), and audiovisual networks (e.g., Youtube ), enable patients to rapidly search and access information regarding their diagnosis and treatment. Social media presents a powerful opportunity to establish connections between patients and their physicians, facilitating direct patient–physician communication and efficient patient education with up-to-date information. Nonetheless, efforts are required to ensure that the information available on social media platforms be correct, objective, and evidence-based. In order to foster the engagement and education of PCa patients regarding the growing role of imaging in their condition, the Society of Abdominal Radiology’s Prostate Cancer Disease Focus Panel has created a Twitter account ‘‘@SAR_ProstateDFP.’’ This social media account will allow radiologists, who are experts in PCa imaging, to share the most up-to-date, objective, evidence-based information regarding imaging with their patients and quickly address patients’ questions and concerns.

Auditory click stimuli evoke event-related potentials in the visual cortex

The objective of this study was to determine whether unimodal auditory stimuli evoke event-related potentials (ERPs) in brain areas normally designated as the visual cortex (VC). The topographical distribution of ERPs evoked by auditory click stimuli was measured from (a) electroencephalographic electrodes on the scalp of six neurologically normal adult human participants and (b) intracranial electrodes implanted on the cortex of one epileptic adult human participant. In all participants, unimodal click stimuli evoked ERPs over both the auditory cortex (AC) and the VC. Relative amplitudes of ERPs at different scalp electrodes did not support the idea that the ERPs over VC were volume-conducted versions of those over AC, and intracranial records confirmed the origin of some click-evoked ERPs in both V1 and other regions of VC. We conclude that unimodal auditory stimuli can evoke ERPs in VC. This finding adds to the earlier evidence for the effect of visual stimuli on AC by providing new evidence for bidirectional functional connectivity in the audio-visual network of the human brain. The implication is that not only do visual stimuli affect hearing; auditory stimuli also affect visual perception.

Providing 3D Video Services: The Challenge From 2D to 3DTV Quality of Experience

Recently, three-dimensional (3D) video has decisively burst onto the entertainment industry scene, and has arrived in households even before the standardization process has been completed. 3D television (3DTV) adoption and deployment can be seen as a major leap in television history, similar to previous transitions from black and white (B&W) to color, from analog to digital television (TV), and from standard definition to high definition. In this paper, we analyze current 3D video technology trends in order to define a taxonomy of the availability and possible introduction of 3D-based services. We also propose an audiovisual network services architecture which provides a smooth transition from two-dimensional (2D) to 3DTV in an Internet Protocol (IP)-based scenario. Based on subjective assessment tests, we also analyze those factors which will influence the quality of experience in those 3D video services, focusing on effects of both coding and transmission errors. In addition, examples of the application of the architecture and results of assessment tests are provided. © 2012 Alcatel-Lucent.

Riflessioni sul rapporto tra struttura e significato nei testi audiovisivi

The term ‘audiovisual text’ defines a structure, which is constituted of image, text and sound; its meaning results from the interaction of the components. This is common to multimedia works, a complex in which the sound movie can be seen a particular case. The article discusses first the specific morphology that derives from the superposition of two temporal flows: image and sound. Second, it asks how the meaning of certain music can be transformed through the combination with visual or textual elements. Third, it considers the audiovisual network in a ‘vertical’ perspective, as a system of semantic transformations through the combination and hierarchy of the medial dimensions.