Audio Feature Extraction Research Articles

Research on sound classification based on SVM

Sound is a ubiquitous natural phenomenon that contains a wealth of information that constantly enhances our understanding of the objective world. With the continuous development of computer network technology and communication technology, audio information has become a very important part. Audio is a non-semantic symbolic representation and an unstructured binary stream. Because the audio itself lacks the description of content semantics and structured organization, it brings great difficulty to the audio classification work. The research of digital audio classification will become more and more important with the increasing number of digital audio resources in the network. Digital audio classification technology is the key technology to solve this problem. It is the key to solve the problem of audio structure and extract audio structured information and content semantics. It is a research hot spot in the field of audio analysis. It has important application value in many fields, such as audio retrieval, video summary and auxiliary video analysis. This paper studies the structure of audio, the analysis and extraction of audio features, the digital audio classifier based on support vector machines (SVM) and the audio segmentation technology based on BCI. SVM is an important achievement of machine learning research in recent years. As a new machine learning method, SVM can solve practical problems such as small sample, nonlinearity and high dimension, so it has become a new research hot spot after the study of neural network. Experiments show that the SVM-based audio classification algorithm has good classification effect, and the smoothed audio segmentation results are more accurate. With the further development of the research, the research results will be well applied in practice.

Predicting emotion in music through audio pattern analysis

Music plays a big part in the development of humanity and has inspired so many people to do great things. Which is why a study was conducted focusing on music and its relation to human emotions. It tackles on how an emotional response is generated from an audio pattern through the use of a model constructed by the researchers. The work done is aimed to generate ratings of the emotion conveyed in a song represented as valence and arousal. Audio feature extraction and Artificial Neural Networks played big parts in the development of the research in creating the regression model and predicting the human emotion being emphasized in an audio pattern. The findings of the research will be used to give meaning to audio patterns and how their structure and composition leads to different emotional states of man.

Automatic music genre recognition for in-car infotainment

Abstract Automatic music genre recognition is fundamental tool for music retrieval, recommendation and personalisation in smart infotainment systems and music streaming services. Such systems may be helpful especially for in-car audio, because driver’s interaction with such infotainment systems could become a major subject of his/her distraction. There are two important tasks to be considered for better genre classification, which present classifier and audio features extraction. In the proposed system, timbral textural and pitch content features were used for genre classification. Timbral texture includes the Mel-Frequency Cepstral Coefficients (MFCC) along with other spectral characteristics. For the pitch content the features extracted from Chroma are selected. The aim of this work is to explore possibilities of music genres classification from audio signal and to create a system for automatic recognition of music genres in the MATLAB programming environment. A functional system for recognition of music genres was developed on the GTZAN data set with ten different musical genres such as rock, pop, classical etc. We examined several classification methods including GMM, SVM, and k-NN. The experimental results show that by using both types of features, the classification accuracy of 69.7% is achieved for the k-NN classifier.

Media Orchestration Between Streams and Devices via New MPEG Timed Metadata

The proliferation of new capabilities in affordable smart devices capable of capturing, processing, and rendering audiovisual media content triggers a need for coordination and orchestration between these devices and their capabilities and of the content flowing from and to such devices. The upcoming Moving Picture Experts Group (MPEG) Media Orchestration standard (MORE, ISO/IEC 23001-13) enables the temporal and spatial orchestration of multiple media and metadata streams. The temporal orchestration is about time synchronization of media and sensor captures, processing, and renderings, for which the MORE standard uses and extends a Digital Video Broadcasting standard. The spatial orchestration is about the alignment of (global) position, altitude, and orientation for which the MORE standard provides dedicated timed metadata. Other types of orchestration involve timed metadata for the region of interest, perceptual quality of media, audio-feature extraction, and media timeline correlation. This paper presents the status of the MORE standard as well as the associated technical and experimental support materials. We also link MORE to the recently initiated MPEG immersive project. © 2018 Society of Motion Picture and Television Engineers, Inc.

Nonnegative OPLS for Supervised Design of Filter Banks: Application to Image and Audio Feature Extraction

Audio or visual data analysis tasks usually have to deal with high-dimensional and nonnegative signals. However, most data analysis methods suffer from overfitting and numerical problems when data have more than a few dimensions needing a dimensionality reduction preprocessing. Moreover, interpretability about how and why filters work for audio or visual applications is a desired property, especially when energy or spectral signals are involved. In these cases, due to the nature of these signals, the nonnegativity of the filter weights is a desired property to better understand its working. Because of these two necessities, we propose different methods to reduce the dimensionality of data while the nonnegativity and interpretability of the solution are assured. In particular, we propose a generalized methodology to design filter banks in a supervised way for applications dealing with nonnegative data, and we explore different ways of solving the proposed objective function consisting of a nonnegative version of the orthonormalized partial least-squares method. We analyze the discriminative power of the features obtained with the proposed methods for two different and widely studied applications: texture and music genre classification. Furthermore, we compare the filter banks achieved by our methods with other state-of-the-art methods specifically designed for feature extraction.

Multimodal deep representation learning for video classification

Real-world applications usually encounter data with various modalities, each containing valuable information. To enhance these applications, it is essential to effectively analyze all information extracted from different data modalities, while most existing learning models ignore some data types and only focus on a single modality. This paper presents a new multimodal deep learning framework for event detection from videos by leveraging recent advances in deep neural networks. First, several deep learning models are utilized to extract useful information from multiple modalities. Among these are pre-trained Convolutional Neural Networks (CNNs) for visual and audio feature extraction and a word embedding model for textual analysis. Then, a novel fusion technique is proposed that integrates different data representations in two levels, namely frame-level and video-level. Different from the existing multimodal learning algorithms, the proposed framework can reason about a missing data type using other available data modalities. The proposed framework is applied to a new video dataset containing natural disaster classes. The experimental results illustrate the effectiveness of the proposed framework compared to some single modal deep learning models as well as conventional fusion techniques. Specifically, the final accuracy is improved more than 16% and 7% compared to the best results from single modality and fusion models, respectively.

AUDIO-VISUAL SPEECH-PROCESSING SYSTEM FOR POLISH APPLICABLE TO HUMAN-COMPUTER INTERACTION

This paper describes audio-visual speech recognition system for Polish language and a set of performance tests under various acoustic conditions. We first present the overall structure of AVASR systems with three main areas: audio features extraction, visual features extraction and subsequently, audiovisual speech integration. We present MFCC features for audio stream with standard HMM modeling technique, then we describe appearance and shape based visual features. Subsequently we present two feature integration techniques, feature concatenation and model fusion. We also discuss the results of a set of experiments conducted to select best system setup for Polish, under noisy audio conditions. Experiments are simulating human-computer interaction in computer control case with voice commands in difficult audio environments. With Active Appearance Model (AAM) and multistream Hidden Markov Model (HMM) we can improve system accuracy by reducing Word Error Rate for more than 30%, comparing to audio-only speech recognition, when Signal-to-Noise Ratio goes down to 0dB.

A New E-Health Tool for Early Identification of Voice and Neurological Pathologies by Speech Processing

The objective of this study is to develop a non-invasive method of early identification and classification of voice pathologies and neurological diseases by speech processing. We will present a new automatic medical diagnosis tool which can assist specialists in their medical diagnosis. The developed strategy is based on speech acquisition of the patient followed by audio features extraction, training and recognition by using the HTK toolkit. The computed parameters are compared to standard values from a codebook database. The experiments and tests are conducted by using the MEEI pathological database of KEY Pentax. The obtained results give good discrimination with a mean pathology recognition ratio about 95%. Finally, this E-Health application is helpful for the prevention of specific diseases and improving the quality of patient care as well as reducing the costs of healthcare.

Transforming the Commonplace through Machine Perception: Light Field Synthesis and Audio Feature Extraction in the Rover Project

Rover is a mechatronic imaging device inserted into quotidian space, transforming the sights and sounds of the everyday through its peculiar modes of machine perception. Using computational light field photography and machine listening, it creates a kind of cinema following the logic of dreams: suspended but mobile, familiar yet infinitely variable in detail. Rover draws on diverse traditions of robotic exploration, landscape and still-life depiction, and audio field recording to create a hybrid form between photography and cinema. This paper describes the mechatronic, machine perception, and audio-visual synthesis techniques developed for the piece.

Audio Data Security and Feature Extraction over Cloud

Audio Data Security and Feature Extraction over Cloud

Using multi-stream hierarchical deep neural network to extract deep audio feature for acoustic event detection

Extraction of effective audio features from acoustic events definitely influences the performance of Acoustic Event Detection (AED) system, especially in adverse audio situations. In this study, we propose a framework for extracting Deep Audio Feature (DAF) using multi-stream hierarchical Deep Neural Network (DNN). The DAF outputted from the proposed framework fuses the potential complementary information of multiple input feature streams and thus could be more discriminative than those input features for AED. We take two input feature streams and the hierarchical DNNs with two stages as an example for showing the extraction of DAF. The effectiveness of different audio features for AED is evaluated on two audio corpora, i.e. BBC (British Broadcasting Corporation) audio dataset and TV audio dataset with different signal-to-noise ratios. Experimental results show that DAF outperforms other features for AED under several experimental conditions.

Cough detection by ensembling multiple frequency subband features

Abstract Cough is a common symptom in respiratory diseases. Objectively evaluating the quantity and intensity of cough by pattern recognition technologies can provide valuable clinical information for cough diagnosis and monitoring. Cough detection is the basis of cough diagnosis and analysis. It aims at detecting cough events and their exact boundaries from an audio stream. From signal characteristics, it is found that energy distribution scatters in the cough spectrum, which is obviously different from speech signals. However, almost all feature extraction methods for cough detection in previous works are derived from the speech recognition domain. In this article, subband features are obtained by using gammatone filterbank and an audio feature extraction method. Support Vector Machine (SVM), K-Nearest Neighbors (KNN) and Random Forest (RF) are trained with the corresponding subband features and ensemble method combines the outputs to make the final decision. Experiments are conducted on both synthetic data and real data. The real data is collected from 18 patients with respiratory diseases in clinical environments and annotated by human experts. Experiment results demonstrate that ensembling multiple frequency subbands helps to impove performance in cough detection. Compared with other methods, our method can improve the accuracy by 3.2%.

Modified DCTNet for audio signals classification

In this paper, we investigate DCTNet for audio signal classification. Its output feature is related to Cohen's class of time-frequency distributions. We introduce the use of adaptive DCTNet (A-DCTNet) for audio signals feature extraction. The A-DCTNet applies the idea of constant-Q transform, with its center frequencies of filterbanks geometrically spaced. The A-DCTNet is adaptive to different acoustic scales, and it can better capture low frequency acoustic information that is sensitive to human audio perception than features such as Mel-frequency spectral coefficients (MFSC). We use features extracted by the A-DCTNet as input for classifiers. Experimental results show that the A-DCTNet and Recurrent Neural Networks (RNN) achieve state-of-the-art performance in bird song classification rate, and improve artist identification accuracy in music data. They demonstrate A-DCTNet's applicability to signal processing problems.

A Review of Physical and Perceptual Feature Extraction Techniques for Speech, Music and Environmental Sounds

Endowing machines with sensing capabilities similar to those of humans is a prevalent quest in engineering and computer science. In the pursuit of making computers sense their surroundings, a huge effort has been conducted to allow machines and computers to acquire, process, analyze and understand their environment in a human-like way. Focusing on the sense of hearing, the ability of computers to sense their acoustic environment as humans do goes by the name of machine hearing. To achieve this ambitious aim, the representation of the audio signal is of paramount importance. In this paper, we present an up-to-date review of the most relevant audio feature extraction techniques developed to analyze the most usual audio signals: speech, music and environmental sounds. Besides revisiting classic approaches for completeness, we include the latest advances in the field based on new domains of analysis together with novel bio-inspired proposals. These approaches are described following a taxonomy that organizes them according to their physical or perceptual basis, being subsequently divided depending on the domain of computation (time, frequency, wavelet, image-based, cepstral, or other domains). The description of the approaches is accompanied with recent examples of their application to machine hearing related problems.

AUDIO DESCRIPTIVE ANALYSIS OF SINGER AND MUSICAL INSTRUMENT IDENTIFICATION IN NORTH INDIAN CLASSICAL MUSIC

Music information retrieval (MIR) has reached to a reasonably stable state after advancement in the Low Level audio Descriptors (LLDs) and feature extraction techniques. The analysis of sound has now become simple by the continuous efforts and research of MIR community in the field of signal processing from last two decades. In north Indian classical music, a singer is accompanied by some instruments such as harmonium, violin or flute. These instruments are tuned in the same musical scale (pitch range) in which the singer is signing. Separate researches have been made in recent past to identify a musical instrument and a singer. In this paper, we have analyzed the low level audio descriptors, for singing voice and musical instrument sound together, that appears to human ear as similar with respect to ‘timbre’, to see if we could treat them same and use identification/ classification routines to classify them into their classes. We have used Hybrid Selection algorithm from wrapper technique(the one that uses classifier also in feature selection process) to identify and extract the features and K-Means and K nearest neighbor classifiers to classify and cross verify the accuracy of classification. The accuracy of classification achieved was 91.1% which clearly proves that musical instruments and singing voice that sounds similar in timbral aspect can be grouped together and classification is possible with mixed database of instruments and singing voices.

Bird acoustic activity detection based on morphological filtering of the spectrogram

Audio event recognition methods based on the Hidden Markov Model / Gaussian Mixture Model (HMM/GMM) often depend on a large number of mixture components or multi-stage models that require significant computational and memory resources during their operation. A widely used approach for coping with complexity is employing an acoustic activity detector, which selects for further processing only those portions of the audio that are considered promising. As a result, the audio feature extraction and the subsequent pattern recognition stages will process only a subset of the original audio stream, helping to reduce the misclassification rates while lowering the computational demands. In the present work we propose a method for bird acoustic activity detection, based on morphological filtering of the spectrogram seen as an image. The practical significance of the proposed method is validated on the automated acoustic recognition of Southern Lapwing Vanellus chilensis, a common Neotropical bird species. Compared with other methods of acoustic activity detection it demonstrates advantageous performance.

A Study on English Audio Segmentation Methods Based on Threshold Value and Energy Sequence

As a new method of learning, the mobile platform is playing an increasingly important role in English learning. In order to fit in with the construction requirements of the English mobile learning platform, it is obliged to carry out methods of positive studies on English audio segmentation. In this study, the author makes a brief analysis on English audio segmentation based on threshold value and energy sequence, provides the basic process of English audio segmentation, and carries out a discussion on key technical points of the scheme, which including windowing process, audio feature extraction and dual threshold. In the end, the author makes a research on the operation of English audio segmentation, aiming at drawing more attentions to this study.

Time–frequency audio feature extraction based on tensor representation of sparse coding

A time–frequency audio feature extraction scheme is proposed, in which features are decomposed from a frequency–time–scale tensor. The tensor, derived from a weight vector and a Gabor dictionary in sparse coding, represents the frequency, time centre and scale of transient time–frequency components with different dimensions. The distinguishing Gabor atoms are represented by individual tensor elements, and their associated coding weights are represented by tensor element values. The experimental results of sound effects classification showed performance improvement against that of sparse coding features.

Automated Cough Assessment on a Mobile Platform.

The development of an Automated System for Asthma Monitoring (ADAM) is described. This consists of a consumer electronics mobile platform running a custom application. The application acquires an audio signal from an external user-worn microphone connected to the device analog-to-digital converter (microphone input). This signal is processed to determine the presence or absence of cough sounds. Symptom tallies and raw audio waveforms are recorded and made easily accessible for later review by a healthcare provider. The symptom detection algorithm is based upon standard speech recognition and machine learning paradigms and consists of an audio feature extraction step followed by a Hidden Markov Model based Viterbi decoder that has been trained on a large database of audio examples from a variety of subjects. Multiple Hidden Markov Model topologies and orders are studied. Performance of the recognizer is presented in terms of the sensitivity and the rate of false alarm as determined in a cross-validation test.

Wearable Audio Monitoring: Content-Based Processing Methodology and Implementation

Developing audio processing tools for extracting social-audio features are just as important as conscious content for determining human behavior. Psychologists speculate these features may have evolved as a way to establish hierarchy and group cohesion because they function as a subconscious discussion about relationships, resources, risks, and rewards. In this paper, we present the design, implementation, and deployment of a wearable computing platform capable of automatically extracting and analyzing social-audio signals. Unlike conventional research that concentrates on data which have been recorded under constrained conditions, our data were recorded in completely natural and unpredictable situations. In particular, we benchmarked a set of integrated algorithms (sound speech detection and classification, sound level meter calculation, voice and nonvoice segmentation, speaker segmentation, and prediction) to obtain speech and environmental sound social-audio signals using an in-house built wearable device. In addition, we derive a novel method that incorporates the recently published audio feature extraction technique based on power normalized cepstral coefficient and gap statistics for speaker segmentation and prediction. The performance of the proposed integrated platform is robust to natural and unpredictable situations. Experiments show that the method has successfully segmented natural speech with 89.6% accuracy.