Set Of Audio Features Research Articles

Emergency Vehicle Classification Using Combined Temporal and Spectral Audio Features with Machine Learning Algorithms

This study presents a novel approach to emergency vehicle classification that leverages a comprehensive set of informative audio features to distinguish between ambulance sirens, fire truck sirens, and traffic noise. A unique contribution lies in combining time domain features, including root mean square (RMS) and zero-crossing rate, to capture the temporal characteristics, like signal energy changes, with frequency domain features derived from short-time Fourier transform (STFT). These include spectral centroid, spectral bandwidth, and spectral roll-off, providing insights into the sound’s frequency content for differentiating siren patterns from traffic noise. Additionally, Mel-frequency cepstral coefficients (MFCCs) are incorporated to capture the human-like auditory perception of the spectral information. This combination captures both temporal and spectral characteristics of the audio signals, enhancing the model’s ability to discriminate between emergency vehicles and traffic noise compared to using features from a single domain. A significant contribution of this study is the integration of data augmentation techniques that replicate real-world conditions, including the Doppler effect and noise environment considerations. This study further investigates the effectiveness of different machine learning algorithms applied to the extracted features, performing a comparative analysis to determine the most effective classifier for this task. This analysis reveals that the support vector machine (SVM) achieves the highest accuracy of 99.5%, followed by random forest (RF) and k-nearest neighbors (KNNs) at 98.5%, while AdaBoost lags at 96.0% and long short-term memory (LSTM) has an accuracy of 93%. We also demonstrate the effectiveness of a stacked ensemble classifier, and utilizing these base learners achieves an accuracy of 99.5%. Furthermore, this study conducted leave-one-out cross-validation (LOOCV) to validate the results, with SVM and RF achieving accuracies of 98.5%, followed by KNN and AdaBoost, which are 97.0% and 90.5%. These findings indicate the superior performance of advanced ML techniques in emergency vehicle classification.

Fall detection from audios with Audio Transformers

Fall detection for the elderly is a well-researched problem with several proposed solutions, including wearable and non-wearable techniques. While the existing techniques have excellent detection rates, their adoption by the target population is lacking due to the need for wearing devices and user privacy concerns. Our paper provides a novel, non-wearable, non-intrusive, and scalable solution for fall detection, deployed on an autonomous mobile robot equipped with a microphone. The proposed method uses ambient sound input recorded in people’s homes. We specifically target the bathroom environment as it is highly prone to falls and where existing techniques cannot be deployed without jeopardizing user privacy. The present work develops a solution based on a Transformer architecture that takes noisy sound input from bathrooms and classifies it into fall/no-fall class with an accuracy of 0.8673. Further, the proposed approach is extendable to other indoor environments, besides bathrooms and is suitable for deploying in elderly homes, hospitals, and rehabilitation facilities without requiring the user to wear any device or be constantly ”watched” by the sensors. • A non-wearable, non-invasive, and scalable method for detecting fall events amongst the elderly using sound as input is developed in this work. • The paper proposes a new set of audio features, called ‘Diff’ features that show better performance than the traditional features such as log mel spectrograms. • The underlying method is a novel audio classification Transformer model that takes noisy environmental sounds as input and correctly classifies them into fall and no-fall classes with an accuracy of 0.8673.

Singer Identification by Vocal Parts Detection and Singer Classification Using LSTM Neural Networks

Abstract: Identification of singers is considered an important research area in audio signal processing. It has acquired the scientist’s intrigues in two primary branches,1) recognizing vocal parts of polyphonic music, and 2) Classifying Singer. Here, we plan to handle the two issues, simultaneously. Techniques like GMM, SVM, and HMM have previously been utilized in classifying singers. In this work, we proposed a system for singer identification using Deep Learning and Feed Forward Neural Networks, which to the best of our knowledge have not been used for this purpose before. Preprocessing involves examining large sets of audio features to extract the most efficient set for the recognition stage. Our work is divided into several stages. To begin with, the vocal parts of all music files are identified utilizing an LSTM network which can perform well for the time series information, for example, audio signals. Then, at that point, an MLP network is integrated and contrasted with an SVM classifier in order to classify the gender of the singers. At last, one more LSTM network is involved to recognize every singer ID, and contrasted with MLP network in a similar task. In each step, various classifiers are analyzed and the outcomes are looked at, which affirm the effectiveness of our technique contrasted with the best in class.

COVID-19 Diagnosis from Crowdsourced Cough Sound Data

The highly contagious and rapidly mutating COVID-19 virus is affecting individuals worldwide. A rapid and large-scale method for COVID-19 testing is needed to prevent infection. Cough testing using AI has been shown to be potentially valuable. In this paper, we propose a COVID-19 diagnostic method based on an AI cough test. We used only crowdsourced cough sound data to distinguish between the cough sound of COVID-19-positive people and that of healthy people. First, we used the COUGHVID cough database to segment only the cough sound from the original cough data. An effective audio feature set was then extracted from the segmented cough sounds. A deep learning model was trained on the extracted feature set. The COVID-19 diagnostic system constructed using this method had a sensitivity of 93% and a specificity of 94%, and achieved better results than models trained by other existing methods.

Audiovisual speaker indexing for Web-TV automations

The current paper introduces a multimodal framework to provide Web-TV automations for live broadcasting and overall big streaming data management. The term indexing refers to the spatiotemporal localization of speakers participating in a discussion panel. Multiple modalities acting in parallel form the data-driven decision-making pipeline. The automated workflow includes the tasks of active speaker detection and localization, frame selection, and creation of a semantically annotated database. For improved performance and robustness, an information fusion model is proposed, which makes use of different audio and visual modalities. Audio-driven Voice Activity Detection follows the Enhanced Temporal Integration methodology applied on a standard audio feature set. For the localization of the dominant audio source, the argument that maximizes the General Cross-Correlation method is calculated. The visual modalities include face and mouth detection and Visual Voice Activity Detection. A Long Short Term Memory network is trained with mouth image sequences to determine voice activity. The values of the audio and visual Voice Activity Detection modules, as well as the General Cross-Correlation result, are used to train an Adaptive Neuro-Fuzzy model, which is responsible for the final decision. Experimental results prove the superiority of the information fusion approach compared to unimodal audio or visual models.

Can empirical mode decomposition improve heartbeat detection in fetal phonocardiography signals?

Background and objectiveA fetal phonocardiography signal can be hard to interpret and classify due to various sources of additive noise in the womb, spanning from fetal movement to maternal heart sounds. Nevertheless, the non-invasive nature of the method makes it potentially suitable for long-term monitoring of fetal health, especially since it can be implemented on ubiquitous devices such as smartphones. We have employed empirical mode decomposition for the extraction of intrinsic mode functions that would enable the utilization of additional characteristics from the signal. MethodsFetal heart recordings from 7 pregnant women in the 3rd trimester or pregnancy were taken in parallel with a measurement microphone and a portable Doppler device. Signal peaks positions from the Doppler were taken as the locations of S1 heart sounds and subsequently used as classification labels for the microphone signal. After employing a moving window approach for segmentation, more than 7600 observations were stored in the final dataset. The 135 extracted features consisted of typical audio temporal and spectral characteristics, each taken from separate sets of audio signals and intrinsic mode functions. We have used a number of metrics and methods to validate the usability of features, including univariate analysis of feature ranking and importance. Furthermore, we have used machine learning to train a number of classifiers to validate the usability of features based on intrinsic mode functions, taking prediction accuracy as the comparison metric. ResultsFeatures extracted from intrinsic mode functions combined with audio features significantly improve accuracy in comparison to using only audio features. The improvements of detection accuracy obtained with a selected set of combined features spanned from 3.8% to even 10.3% based on the employed classifier. ConclusionsWe have utilized empirical mode decomposition as a method of extracting features relevant for fetal heartbeat classification. The results show consistent improvements in detection accuracy when these characteristics are added to a set of conventional audio features. This implies substantial benefits of applying empirical mode decomposition and lays the groundwork for future research on fetal heartbeat detection.

Clustering analysis of crowd noise from collegiate basketball games

The relationship between crowd noise and crowd behavioral dynamics is a relatively unexplored field of research. Signal processing and machine learning (ML) may be useful in classifying and predicting crowd emotional state. As a precursor to performing ML, it is instructive to identify which crowd acoustic events an unsupervised ML algorithm would classify as unique. An initial set of audio features have been extracted from high-fidelity noise recordings of crowds at Brigham Young University men’s and women’s basketball games. A K-Means clustering analysis was conducted on half-second segments of the recordings using extracted features consisting of numerous statistical and spectral characteristics. For example, a clustering analysis performed on a one-twelfth-octave spectrogram of crowd noise recordings reveals there are approximately six unique events that occur during a game. The clusters for different audio feature sets are compared with human labeling of the different acoustical events. Implications for further ML algorithm development based on various sets of audio features are discussed.

Prediction of three articulatory categories in vocal sound imitations using models for auditory receptive fields.

Vocal sound imitations provide a new challenge for understanding the coupling between articulatory mechanisms and the resulting audio. In this study, the classification of three articulatory categories, phonation, supraglottal myoelastic vibrations, and turbulence, have been modeled from audio recordings. Two data sets were assembled, consisting of different vocal imitations by four professional imitators and four non-professional speakers in two different experiments. The audio data were manually annotated by two experienced phoneticians using a detailed articulatory description scheme. A separate set of audio features was developed specifically for each category using both time-domain and spectral methods. For all time-frequency transformations, and for some secondary processing, the recently developed Auditory Receptive Fields Toolbox was used. Three different machine learning methods were applied for predicting the final articulatory categories. The result with the best generalization was found using an ensemble of multilayer perceptrons. The cross-validated classification accuracy was 96.8% for phonation, 90.8% for supraglottal myoelastic vibrations, and 89.0% for turbulence using all the 84 developed features. A final feature reduction to 22 features yielded similar results.

A Combined Motion-Audio School Bullying Detection Algorithm

School bullying is a common social problem, which affects children both mentally and physically, making the prevention of bullying a timeless topic all over the world. This paper proposes a method for detecting bullying in school based on activity recognition and speech emotion recognition. In this method, motion and voice data are gathered by movement sensors and a microphone, followed by extraction of a set of motion and audio features to distinguish bullying incidents from daily life events. Among extracted motion features are both time-domain and frequency-domain features, while audio features are computed with classical MFCCs. Feature selection is implemented using the wrapper approach. At the next stage, these motion and audio features are merged to form combined feature vectors for classification, and LDA is used for further dimension reduction. A BPNN is trained to recognize bullying activities and distinguish them from normal daily life activities. The authors also propose an action transition detection method to reduce computational complexity for practical use. Thus, the bullying detection algorithm will only run, when an action transition event has been detected. Simulation results show that the combined motion-audio feature vector outperforms separate motion features and acoustic features, achieving an accuracy of 82.4% and a precision of 92.2%. Moreover, with the action transition method, the computation cost can be reduced by half.

Improved Audio Steganalytic Feature and Its Applications in Audio Forensics

Digital multimedia steganalysis has attracted wide attention over the past decade. Currently, there are many algorithms for detecting image steganography. However, little research has been devoted to audio steganalysis. Since the statistical properties of image and audio files are quite different, features that are effective in image steganalysis may not be effective for audio. In this article, we design an improved audio steganalytic feature set derived from both the time and Mel-frequency domains for detecting some typical steganography in the time domain, including LSB matching, Hide4PGP, and Steghide. The experiment results, evaluated on different audio sources, including various music and speech clips of different complexity, have shown that the proposed features significantly outperform the existing ones. Moreover, we use the proposed features to detect and further identify some typical audio operations that would probably be used in audio tampering. The extensive experiment results have shown that the proposed features also outperform the related forensic methods, especially when the length of the audio clip is small, such as audio clips with 800 samples. This is very important in real forensic situations.

Detecting Parkinson's disease from sustained phonation and speech signals.

This study investigates signals from sustained phonation and text-dependent speech modalities for Parkinson’s disease screening. Phonation corresponds to the vowel /a/ voicing task and speech to the pronunciation of a short sentence in Lithuanian language. Signals were recorded through two channels simultaneously, namely, acoustic cardioid (AC) and smart phone (SP) microphones. Additional modalities were obtained by splitting speech recording into voiced and unvoiced parts. Information in each modality is summarized by 18 well-known audio feature sets. Random forest (RF) is used as a machine learning algorithm, both for individual feature sets and for decision-level fusion. Detection performance is measured by the out-of-bag equal error rate (EER) and the cost of log-likelihood-ratio. Essentia audio feature set was the best using the AC speech modality and YAAFE audio feature set was the best using the SP unvoiced modality, achieving EER of 20.30% and 25.57%, respectively. Fusion of all feature sets and modalities resulted in EER of 19.27% for the AC and 23.00% for the SP channel. Non-linear projection of a RF-based proximity matrix into the 2D space enriched medical decision support by visualization.

Modeling Timbre Similarity of Short Music Clips.

There is evidence from a number of recent studies that most listeners are able to extract information related to song identity, emotion, or genre from music excerpts with durations in the range of tenths of seconds. Because of these very short durations, timbre as a multifaceted auditory attribute appears as a plausible candidate for the type of features that listeners make use of when processing short music excerpts. However, the importance of timbre in listening tasks that involve short excerpts has not yet been demonstrated empirically. Hence, the goal of this study was to develop a method that allows to explore to what degree similarity judgments of short music clips can be modeled with low-level acoustic features related to timbre. We utilized the similarity data from two large samples of participants: Sample I was obtained via an online survey, used 16 clips of 400 ms length, and contained responses of 137,339 participants. Sample II was collected in a lab environment, used 16 clips of 800 ms length, and contained responses from 648 participants. Our model used two sets of audio features which included commonly used timbre descriptors and the well-known Mel-frequency cepstral coefficients as well as their temporal derivates. In order to predict pairwise similarities, the resulting distances between clips in terms of their audio features were used as predictor variables with partial least-squares regression. We found that a sparse selection of three to seven features from both descriptor sets—mainly encoding the coarse shape of the spectrum as well as spectrotemporal variability—best predicted similarities across the two sets of sounds. Notably, the inclusion of non-acoustic predictors of musical genre and record release date allowed much better generalization performance and explained up to 50% of shared variance (R2) between observations and model predictions. Overall, the results of this study empirically demonstrate that both acoustic features related to timbre as well as higher level categorical features such as musical genre play a major role in the perception of short music clips.

Efficient audio-driven multimedia indexing through similarity-based speech / music discrimination

In this paper, an audio-driven algorithm for the detection of speech and music events in multimedia content is introduced. The proposed approach is based on the hypothesis that short-time frame-level discrimination performance can be enhanced by identifying transition points between longer, semantically homogeneous segments of audio. In this context, a two-step segmentation approach is employed in order to initially identify transition points between the homogeneous regions and subsequently classify the derived segments using a supervised binary classifier. The transition point detection mechanism is based on the analysis and composition of multiple self-similarity matrices, generated using different audio feature sets. The implemented technique aims at discriminating events focusing on transition point detection with high temporal resolution, a target that is also reflected in the adopted assessment methodology. Thereafter, multimedia indexing can be efficiently deployed (for both audio and video sequences), incorporating the processes of high resolution temporal segmentation and semantic annotation extraction. The system is evaluated against three publicly available datasets and experimental results are presented in comparison with existing implementations. The proposed algorithm is provided as an open source software package in order to support reproducible research and encourage collaboration in the field.

TRADITIONAL MALAYSIAN MUSICAL GENRES CLASSIFICATION BASED ON THE ANALYSIS OF BEAT FEATURE IN AUDIO

Interest on automated genre classification systems is growing following the increase in the number of musical digital data collections. Many of these systems have been researched and developed to classify Western musical genres such as pop, rock or classical. However, adapting these systems for the classification of Traditional Malay Musical (TMM) genres which includes Gamelan, Inang and Zapin, is difficult due to the differences in musical structures and modes. This study investigates the effects of various factors and audio feature set combinations towards the classification of TMM genres. Results from experiments conducted in several phases show that factors such as dataset size, track length and location¸ together with various combinations of audio feature sets comprising Short Time Fourier Transform (STFT), Mel-Frequency Cepstral Coefficients (MFCCs) and Beat Features affect classification. Based on parameters optimized for TMM genres, classification performances were evaluated against three groups of human subjects: experts, trained and untrained. Performances of both machine and human were shown to be comparable.

Quantitative Study of Music Listening Behavior in a Smartphone Context

Context-based services have attracted increasing attention because of the prevalence of sensor-rich mobile devices such as smartphones. The idea is to recommend information that a user would be interested in according to the user’s surrounding context. Although remarkable progress has been made to contextualize music playback, relatively little research has been made using a large collection of real-life listening records collected in situ . In light of this fact, we present in this article a quantitative study of the personal, situational, and musical factors of musical preference in a smartphone context, using a new dataset comprising the listening records and self-report context annotation of 48 participants collected over 3wk via an Android app. Although the number of participants is limited and the population is biased towards students, the dataset is unique in that it is collected in a daily context, with sensor data and music listening profiles recorded at the same time. We investigate 3 core research questions evaluating the strength of a rich set of low-level and high-level audio features for music usage auto-tagging (i.e., music preference in different user activities), the strength of time-domain and frequency-domain sensor features for user activity classification, and how user factors such as personality traits are correlated with the predictability of music usage and user activity, using a closed set of 8 activity classes. We provide an in-depth discussion of the main findings of this study and their implications for the development of context-based music services for smartphones.

LIRIS-ACCEDE: A Video Database for Affective Content Analysis

Research in affective computing requires ground truth data for training and benchmarking computational models for machine-based emotion understanding. In this paper, we propose a large video database, namely LIRIS-ACCEDE, for affective content analysis and related applications, including video indexing, summarization or browsing. In contrast to existing datasets with very few video resources and limited accessibility due to copyright constraints, LIRIS-ACCEDE consists of 9,800 good quality video excerpts with a large content diversity. All excerpts are shared under creative commons licenses and can thus be freely distributed without copyright issues. Affective annotations were achieved using crowdsourcing through a pair-wise video comparison protocol, thereby ensuring that annotations are fully consistent, as testified by a high inter-annotator agreement, despite the large diversity of raters' cultural backgrounds. In addition, to enable fair comparison and landmark progresses of future affective computational models, we further provide four experimental protocols and a baseline for prediction of emotions using a large set of both visual and audio features. The dataset (the video clips, annotations, features and protocols) is publicly available at: http://liris-accede.ec-lyon.fr/.

Probabilistic Detection Methods for Acoustic Surveillance Using Audio Histograms

Acoustic surveillance is gaining importance given the pervasive nature of multimedia sensors being deployed in all environments. In this paper, novel probabilistic detection methods using audio histograms are proposed for acoustic event detection in a multimedia surveillance environment. The proposed detection methods use audio histograms to classify events in a well-defined acoustic space. The proposed methods belong to the category of novelty detection methods, since audio data corresponding to the event is not used in the training process. These methods hence alleviate the problem of collecting large amount of audio data for training statistical models. These methods are also computationally efficient since a conventional audio feature set like the Mel frequency cepstral coefficients in tandem with audio histograms are used to perform acoustic event detection. Experiments on acoustic event detection are conducted on the SUSAS database available from Linguistic data consortium. The performance is measured in terms of false detection rate and true detection rate. Receiver operating characteristics curves are obtained for the proposed probabilistic detection methods to evaluate their performance. The proposed probabilistic detection methods perform significantly better than the acoustic event detection methods available in literature. A cell phone-based alert system for an assisted living environment is also discussed as a future scope of the proposed method. The performance evaluation is presented as number of successful cell phone transactions. The results are motivating enough for the system to be used in practice.

Drumkit simulator from everyday desktop objects

In this paper, an augmented reality application for drumkit simulation is presented. The system is capable of classifying any percussive sounds produced by the user from an everyday desktop environment, e.g. clapping, snapping, stroking different objects with a pencil, etc., recognizing up to six different classes of drum hits. These different types of user-generated sounds will subsequently be associated to predefined drumkit sounds, resulting in a natural and intuitive audio interface for drummers and percussionists, which only requires a computer with a built-in microphone. A set of audio features and classification techniques are evaluated for the implementation of the aforementioned system.

Combining Language Modeling and LSA on Greek Song “Words” for Mood Classification

The present work presents a novel approach to song mood classification. Two language models, one absolute and one relative, are experimented with. Two distinct audio feature sets are compared against each other, and the significance of the inclusion of text stylistic features is established. Furthermore, Latent Semantic Analysis is innovatively combined with language modeling, and depicts the discriminative power of the latter. Finally, song “words” are defined in a broader sense that includes lyrics words as well as audio words, and LSA is applied to this augmented vocabulary with highly promising results. The methodology is applied to Greek songs, that are classified into one of four valence and into one of four arousal categories.

On Computer-Assisted Orchestration

In 2003, I presented to Ircam a proposal for a long-term research project on the subject of computer-assisted orchestration. The results of this research project lead to the prototype softwares, ‘Orchidee’ and now, ‘Ato-ms’, both used by numerous composers. With these systems, composers can specify a target sound and replicate it with a given, predetermined orchestra. The target sound, defined as a set of audio and symbolic features, can be constructed either by analysing a pre-recorded sound, or through a compositional process. The orchestration procedure uses large pre-analysed instrumental sound databases to offer composers a set of sound combinations. This procedure relies on a set of features that describe different aspects of the sound. Almost 10 years after the start of this project, it is time to look back at what was accomplished from the musical stand point, and to open some new perspectives on the subject, like the introduction of target descriptors for orchestral qualities, and orchestral layers.