Audio Analytics Research Articles

“I like the sound of that”: understanding the effectiveness of audio in ads

PurposeSports advertisements such as the Super Bowl showcase products and brands that have invested increasingly large sums financially to gain viewers’ attention. However, how audio features in advertisements impact viewers' behavior remains unexplored.Design/methodology/approachUsing the lens of signaling theory, this research uses advanced data analytics of voice and music audio in Super Bowl advertisements to examine its impacts on viewers.FindingsResults show that advertisement viewers prefer more voiced frames and have a greater liking behavior of voiced frames with a low intensity (less loud) and a higher F1 frequency, which is typically associated with male vocal tracts. A fast music tempo works better for longer advertisements. The model controls for various types of ad appeals. The research underlines the importance of voice and music in signaling successful brand features that are likely to increase the ad-liking behavior of consumers (positive effect).Research limitations/implicationsThe current research implies that brands advertising through sports ads must carefully select voice actors and music in order to provide the most positive signals for a brand to have the most significant effect and, thus, a greater return on the high sums invested in the ads.Originality/valueFirst, this research contributes in terms of a new research process for using audio analytics in advertising. The detailed research process outlined can be used for future research examining audio and music from advertisements. Second, our findings provide additional support to the important role of voice features (e.g. intensity and frequency) as signals in inducing responses from consumers (Biswas et al., 2019; Hagtvedt and Brasel, 2016). Third, the study surfaces a new theoretical association: the effect of tempo in moderating the relationship between duration and propensity to like an ad.

Fake Voice Detection System

Fake Voice Detection System

Audio signal transmission method in network-based audio analytics system

The subject matter of the article is аudio signal transmission method in network-based audio analytics system. The creation of a network-based audio analytics system leads to the emergence of new classes of load sources that transmit packetized sound data. Therefore, without constructing adequate mathematical models, it is impossible to build a well-functioning network-based audio analytics system. A fundamental question in traffic theory is the question of load source models. The development of an method for transmitting audio signals in a network-based audio analytics system becomes necessary. Based on this, the goal of the work is to create methods an method for transmitting audio signals in a network-based audio analytics system to ensure efficiency and accuracy in audio analytics. The following tasks were solved in the article: the formation of a model for the system's load sources, investigation of connection and traffic management, implementation of control and traffic monitoring functions in the network, research of methods to ensure the quality of audio signal transmission and the development of a method of transmitting an audio signal by virtual routes switching. To achieve these goals, the following methods are used: mathematical signal processing, data compression algorithms, optimization of network protocols, and the use of high-speed network connections. The obtained results include modeling of the system's load sources, examination of connection and traffic management, investigation of methods to ensure the quality of audio signal transmission and a method of transmitting an audio signal by virtual routes switching was proposed. In conclusion, the possibilities of using simulation modeling of nodes in the network-based audio analytics system are highly limited. This is explained by the fact that the acceptable level of information loss in data centers is very low. The use of the developed method enables effective control and processing of sound information in real-time. This method can find broad applications in various fields, including security, healthcare, management systems, and other industries where the analysis of audio signals is a crucial element.

Are acoustics enough? Semantic effects on auditory salience in natural scenes

Auditory salience is a fundamental property of a sound that allows it to grab a listener's attention regardless of their attentional state or behavioral goals. While previous research has shed light on acoustic factors influencing auditory salience, the semantic dimensions of this phenomenon have remained relatively unexplored owing both to the complexity of measuring salience in audition as well as limited focus on complex natural scenes. In this study, we examine the relationship between acoustic, contextual, and semantic attributes and their impact on the auditory salience of natural audio scenes using a dichotic listening paradigm. The experiments present acoustic scenes in forward and backward directions; the latter allows to diminish semantic effects, providing a counterpoint to the effects observed in forward scenes. The behavioral data collected from a crowd-sourced platform reveal a striking convergence in temporal salience maps for certain sound events, while marked disparities emerge in others. Our main hypothesis posits that differences in the perceptual salience of events are predominantly driven by semantic and contextual cues, particularly evident in those cases displaying substantial disparities between forward and backward presentations. Conversely, events exhibiting a high degree of alignment can largely be attributed to low-level acoustic attributes. To evaluate this hypothesis, we employ analytical techniques that combine rich low-level mappings from acoustic profiles with high-level embeddings extracted from a deep neural network. This integrated approach captures both acoustic and semantic attributes of acoustic scenes along with their temporal trajectories. The results demonstrate that perceptual salience is a careful interplay between low-level and high-level attributes that shapes which moments stand out in a natural soundscape. Furthermore, our findings underscore the important role of longer-term context as a critical component of auditory salience, enabling us to discern and adapt to temporal regularities within an acoustic scene. The experimental and model-based validation of semantic factors of salience paves the way for a complete understanding of auditory salience. Ultimately, the empirical and computational analyses have implications for developing large-scale models for auditory salience and audio analytics.

Revolutionizing Physics Education: Enhancing High School Students' Understanding of Standing Wave Concepts through Mictester-Based Smartphone Experiments

The study aims to test the effectiveness of using the audio analysis software, Mictester, on smartphones in experiments on standing waves in order to improve students' understanding of concepts in high school. The sample consisted of 19 students of 11th grade at State High School 1 SoE. The instrument used was a test sheet of conceptual understanding of standing wave matter. The results show a significant improvement in students' understanding of concepts after the application of Mictester for audio analysis on Android devices/smartphones. Thus, standing wave experiments that utilize Mictesters software for audio analytics can be an alternative in physics learning, replacing conventional experimental equipment such as oscilloscopes, thereby reducing the cost of experiments.

Smooth Talking and Fast Music: Understanding the Importance of Voice and Music in Travel and Tourism Ads via Acoustic Analytics

Travel and tourism advertising is critical in developing positive associations to attract visitor patronage and build sustainable post-pandemic tourism. An important part of an advertising message is delivered through voice and other aspects of audio, for example, music. However, how audio features impact viewers of travel and tourism advertisements remains unexplored in the research. This study implements advanced audio analytics to test how various features impact upon viewers. The results show that voice quality is important in developing positive affect; advertisement viewers prefer speakers with quieter voices (less mean intensity) that have a higher level of clarity (higher harmonics-to-noise ratio). This can be explained via the heuristic route of the heuristic-systematic model. Music tempo was found to be important in stimulating reactions from advertisements, with faster music being associated with a higher level of positive affect. The paper concludes with practical and theoretical implications, limitations, and suggestions for future research.

OPTIMIZATION OF A BASIC NETWORK IN AUDIO ANALYTICS SYSTEMS

Relevance. The sound is a source of data that provides information necessary for survival and warns of potential dangers. Audio analytics solutions allows detecting and responding in time to illegal actions and violations of the law, which are accompanied by appropriate sounds. Therefore, the problem of reducing delays in the transmission of audio streams in network-based systems of audio analytics becomes relevant. The object of research is the process of audio signals transmitting. The subject of the research is mathematical models of audio and video streams transmission in network systems. The purpose of this paper is to develop an approximate method for quickly solving optimization equations for a network of connecting links and to assess the adequacy of the developed method. Research results. A method for selecting the network structure of the audio analytics system is proposed. The optimization of the network structure of audio analytics system under non-ordinary Poisson load is presented. The optimization of the basic network of audio analytics system was studied according to the cost criterion. Comparisons of the optimization results shows that the given link distribution options are not significantly different from each other and are close in cost.

Fast Detection and Classification of Dangerous Urban Sounds Using Deep Learning

Video analytics is an integral part of surveillance cameras. Compared to video analytics, audio analytics offers several benefits, including less expensive equipment and upkeep expenses. Additionally, the volume of the audio datastream is substantially lower than the video camera datastream, especially concerning real-time operating systems, which makes it less demanding of the data channel’s bandwidth needs. For instance, automatic live video streaming from the site of an explosion and gunshot to the police console using audio analytics technologies would be exceedingly helpful for urban surveillance. Technologies for audio analytics may also be used to analyze video recordings and identify occurrences. This research proposed a deep learning model based on the combination of convolutional neural network (CNN) and recurrent neural network (RNN) known as the CNN-RNN approach. The proposed model focused on automatically identifying pulse sounds that indicate critical situations in audio sources. The algorithm’s accuracy ranged from 95% to 81% when classifying noises from incidents, including gunshots, explosions, shattered glass, sirens, cries, and dog barking. The proposed approach can be applied to provide security for citizens in open and closed locations, like stadiums, underground areas, shopping malls, and other places.

Mathematical model of the passage of audio signals in network-based audio analytics systems

Relevance. Nowadays, more and more audio and video analytics systems work together. But in many cases, it is necessary to quickly transfer the audio stream. Therefore, the task of reducing delays in transmission of audio streams in network-based audio analytics systems becomes relevant. The object of research is the process of audio signal transmission. The subject of research is mathematical models of audio and video streams in network systems. The purpose of this paper is to develop a mathematical model for the passage of audio signals in network-based audio analytics systems. Research results. The statement of the problem of optimizing the network of connecting lines according to the criterion of minimum capital costs is proposed. Optimization equations for a unidirectional three-node basic structure are compiled based on the problem of minimizing capital costs. Optimization equations are obtained for a bidirectional three-node basic structure. Obtained optimization equations are generalized to any structure of a network of connecting lines. The direction of further research is the development of an approximate method for quickly solving optimization equations for a network of connecting lines and assessing the adequacy of the developed model.

Applying the FN-corrector to improve the quality of audio event classification

The paper deals with the problem of acoustic events classification which is actively applied to the problems of a safe city, smart home, IoT devices, and for the detection of industrial accidence. A solution to improve the accuracy of classifiers without changing their structure and collecting additional data is proposed. The main data source for the experiments was the TUT Urban Acoustic Scenes 2018, Development Dataset. The paper presents the way to increase the accuracy of audio event classification by using the FN-corrector. The FN-corrector is a linear two-stage classifier performing the transformation of the feature space into a linearly separable space and the linear separation of one class from another. If a corrector is applied, the responses of the original classifier generate four classes: positive (P), negative (N), false positive (FP), and false negative (FN). As a result, it becomes possible to train two types of correctors: the FP-corrector separating positive and false positive classifier responses, and the FN-corrector separating negative and false negative classifier responses. In the experiments, the VGGish convolutional neural network was used as the initial classifier. The audio signal is converted into a spectrogram and is fed to the input of the neural network which forms the spectrogram feature description and performs a classification. As an example, two ”confused“ classes are selected to demonstrate the increase in classification accuracy. Using the feature description of audio recordings of these classes, an FN-corrector was built, trained and connected to the original classifier. The response from the classifier, as well as the feature description, has been passed to the corrector input. Next, the corrector translated the feature space into a new basis (into a linearly separable space) and classified the classifier answer responding to the question whether the original classifier makes a mistake on such a feature vector or not. If the original classifier made a mistake, then his answer is changed by the corrector to the opposite, otherwise the answer remains the same. The results of the experiments demonstrated a decrease in the level of class confusion and, accordingly, an increase in the accuracy of the original classifier without changing its structure and without collecting an additional data set. The results obtained can be used on IoT devices that have significant limitations on the size of the models used, as well as in solving the problems of domain adaptation which is relevant in audio analytics

ANALYSIS OF THE SOUND EVENT DETECTION METHODS AND SYSTEMS

Detection and recognition of loud sounds and characteristic noises can significantly increase the level of safety and ensure timely response to various emergency situations. Audio event detection is the first step in recognizing audio signals in a continuous audio input stream. This article presents a number of problems that are associated with the development of sound event detection systems, such as the deviation for each environment and each sound category, overlapping audio events, unreliable training data, etc. Both methods for detecting monophonic impulsive audio event and polyphonic sound event detection methods which are used in the state-of-the-art sound event detection systems are presented. Such systems are presented in Detection and Classification of Acoustic Scenes and Events (DCASE) challenges and workshops, which take place every year. Beside a majority of works focusing on the improving overall performance in terms of accuracy many other aspects have also been studied. Several systems presented at DCASE 2021 task 4 were considered, and based on their analysis, there was a conclusion about possible future for sound event detection systems. Also the actual directions in the development of modern audio analytics systems are presented, including the study and use of various architectures of neural networks, the use of several data augmentation techniques, such as universal sound separation, etc.

Autonomous Surveillance of Infants’ Needs Using CNN Model for Audio Cry Classification

Infants portray suggestive unique cries while sick, having belly pain, discomfort, tiredness, attention and desire for a change of diapers among other needs. There exists limited knowledge in accessing the infants’ needs as they only relay information through suggestive cries. Many teenagers tend to give birth at an early age, thereby exposing them to be the key monitors of their own babies. They tend not to have sufficient skills in monitoring the infant’s dire needs, more so during the early stages of infant development. Artificial intelligence has shown promising efficient predictive analytics from supervised, and unsupervised to reinforcement learning models. This study, therefore, seeks to develop an android app that could be used to discriminate the infant audio cries by leveraging the strength of convolution neural networks as a classifier model. Audio analytics from many kinds of literature is an untapped area by researchers as it’s attributed to messy and huge data generation. This study, therefore, strongly leverages convolution neural networks, a deep learning model that is capable of handling more than one-dimensional datasets. To achieve this, the audio data in form of a wave was converted to images through Mel spectrum frequencies which were classified using the computer vision CNN model. The Librosa library was used to convert the audio to Mel spectrum which was then presented as pixels serving as the input for classifying the audio classes such as sick, burping, tired, and hungry. The study goal was to incorporate the model as an android tool that can be utilized at the domestic level and hospital facilities for surveillance of the infant’s health and social needs status all time round.

LADEA: A Software Infrastructure for Audio Delivery and Analytics

The LoRa technology enables long distance links with reduced power consumption at low cost, the main limitation being the low bandwidth that it offers. With LoRa, remote locations, like rural areas, can benefit from connectivity based services that would otherwise be impossible. In this work, we describe a LoRa architecture that can include generic external data sources using an MQTT-based interface. We particularly focus on audio sources aiming to two basic services: a voice messaging system that allows users who cannot read or write to send voice notes, and an audio compression service to extract the main features from the audio input to use it for developing intelligent ML-based audio analytics.

DENet: a deep architecture for audio surveillance applications

In the last years, a big interest of both the scientific community and the market has been devoted to the design of audio surveillance systems, able to analyse the audio stream and to identify events of interest; this is particularly true in security applications, in which the audio analytics can be profitably used as an alternative to video analytics systems, but also combined with them. Within this context, in this paper we propose a novel recurrent convolutional neural network architecture, named DENet; it is based on a new layer that we call denoising-enhancement (DE) layer, which performs denoising and enhancement of the original signal by applying an attention map on the components of the band-filtered signal. Differently from state-of-the-art methodologies, DENet takes as input the lossless raw waveform and is able to automatically learn the evolution of the frequencies-of-interest over time, by combining the proposed layer with a bidirectional gated recurrent unit. Using the feedbacks coming from classifications related to consecutive frames (i.e. that belong to the same event), the proposed method is able to drastically reduce the misclassifications. We carried out experiments on the MIVIA Audio Events and MIVIA Road Events public datasets, confirming the effectiveness of our approach with respect to other state-of-the-art methodologies.

Leveraging deep learning with audio analytics to predict the success of crowdfunding projects

In the social Web era, crowdfunding has become an increasingly important channel for entrepreneurs to raise funds from the crowd for their start-up projects. Previous studies have examined various factors, such as textual information of projects and social capital of investors. However, multimedia information on projects such as audio information was rarely studied for analysing crowdfunding successes. This paper designs a novel audio analytics-based deep learning framework that can extract audio features to predict the fundraising outcomes of these projects. In the proposed framework, we suggest transfer learning to train our models, and multi-task learning to extract the deep features of audios. With the proposed features, our model achieves an 8.28% improvement in F1 and a 7.35% AUC comparing to baselines.

Big Data: Big Data Analysis, Issues and Challenges and Technologies

The data generated at an exponential rate has resulted in Big Data. This data has many characteristics and consists of structured, unstructured, and semi-structured data formats. It contains valuable information for the different types of stakeholders based on their need however it is not possible to meet them with the help of traditional tools and techniques. Here the big data technologies play a crucial role to handle, store, and process this tremendous amount of data in real-time. Big data analytics is used to extract meaningful information or patterns from the voluminous data. It can be further divided into three types i.e. text analytics, audio analytics, video analytics, and social media analytics. Big data analytics if followed by big data analysis process plays a significant role in generating meaningful information from big data. Big data analysis process consists of data acquisition, data storage, data management, data analytics, and finally data visualization. However, it is not simple and brings many challenges that need to be resolved. This paper presents the issues and challenges related to big data, prominent characteristics of big data, big data analytics, big data analysis process, and technologies used for processing the massive data.

Speech-music discrimination using deep visual feature extractors

Speech music discrimination is a traditional task in audio analytics, useful for a wide range of applications, such as automatic speech recognition and radio broadcast monitoring, that focuses on segmenting audio streams and classifying each segment as either speech or music. In this paper we investigate the capabilities of Convolutional Neural Networks (CNNs) with regards to the speech - music discrimination task. Instead of representing the audio content using handcrafted audio features, as traditional methods do, we use deep structures to learn visual feature dependencies as they appear on the spectrogram domain (i.e. train a CNN using audio spectrograms as input images). The main contribution of our work focuses on the potentials of using pre-trained deep architectures along with transfer-learning to train robust audio classifiers for the particular task of speech music discrimination. We highlight the supremacy of the proposed methods, compared both to the typical audio-based and deep-learning methods that adopt handcrafted features, and we evaluate our system in terms of classification success and run-time execution. To our knowledge this is the first work that investigates CNNs for the task of speech music discrimination and the first that exploits transfer learning across very different domains for audio modeling using deep-learning in general. In particular, we fine-tune a deep architecture originally trained for the Imagenet classification task, using a relatively small amount of data (almost 80 min of training audio samples) along with data augmentation. We evaluate our system through extensive experimentation against three different datasets. Firstly we experiment on a real-world dataset of more than 10 h of uninterrupted radio broadcasts and secondly, for comparison purposes, we evaluate our best method on two publicly available datasets that were designed specifically for the task of speech-music discrimination. Our results indicate that CNNs can significantly outperform current state-of-the-art in terms of performance especially when transfer learning is applied, in all three test-datasets. All the discussed methods, along with the whole experimental setup and the respective datasets, are openly provided for reproduction and further experimentation.