Audio Signal Analysis Research Articles

Analysis of audio signal using various transforms for enhanced audio processing

Audio Signals are the portrayal of sounds. It changes with respect to frequencies rather than time, and it shows more information in the frequency domain. So it is much appropriate to evaluate in the frequency domain rather than the time domain. By using different transforms like DFT, DST, DCT, MDCT, Integer MDCT, the time domain audio signal can be converted into a frequency domain signal. The signal is reconstructed to analyze the features like mean square error, Signal to noise ratio, Peak signal to noise ratio between the original and reconstructed signal. Other features like energy, entropy, zero crossing rates (ZCR) were also considered for the evaluation. In this paper, different audio file formats were taken for interpretation. It includes wave file, mp3 file, m4a file, aac file, where wave file is in uncompressed format and mp3, m4a, aac are in compressed format. These compressed files come under lossy compression. The above-mentioned features are used for applications like music information retrieval (MIR). MIR includes onset detection, pitch detection and to measure the noise and loudness of the music.

Lightweight Deep Learning Model for Assessment of Substitution Voicing and Speech after Laryngeal Carcinoma Surgery.

Simple SummaryA total laryngectomy involves the full and permanent separation of the upper and lower airways, resulting in the loss of voice and inability to interact vocally. To identify, extract, and evaluate replacement voicing following laryngeal oncosurgery, we propose employing convolutional neural networks for categorization of speech representations (spectrograms). With an overall accuracy of 89.47 percent, our technique has the greatest true-positive rate of any of the tested state-of-the-art methodologies.Laryngeal carcinoma is the most common malignant tumor of the upper respiratory tract. Total laryngectomy provides complete and permanent detachment of the upper and lower airways that causes the loss of voice, leading to a patient’s inability to verbally communicate in the postoperative period. This paper aims to exploit modern areas of deep learning research to objectively classify, extract and measure the substitution voicing after laryngeal oncosurgery from the audio signal. We propose using well-known convolutional neural networks (CNNs) applied for image classification for the analysis of voice audio signal. Our approach takes an input of Mel-frequency spectrogram (MFCC) as an input of deep neural network architecture. A database of digital speech recordings of 367 male subjects (279 normal speech samples and 88 pathological speech samples) was used. Our approach has shown the best true-positive rate of any of the compared state-of-the-art approaches, achieving an overall accuracy of 89.47%.

Music Feature Extraction Method Based on Internet of Things Technology and Its Application.

Due to the influence of factors such as strong music specialization, complex music theory knowledge, and various variations, it is difficult to identify music features. We have developed a music characteristic identification system using the Internet-based method. The physical sensing layer of our designed system deploys audio sensors on various coordinates to capture the raw audio signal and performs audio signal processing and analysis using the TMS320VC5402 digital signal processor; the Internet transport layer places audio sensors at various locations to capture the raw audio signal. The TMS320VC5402 digital signal processor is used for audio signal diagnosis and treatment. The network transport layer transmits the finished audio signal to the data base of song signal in the application layer of the system; the song characteristic analysis block in the application layer adopts dynamics. The music characteristic analysis block in the applied layer adopts dynamic time warping algorithm to acquire the maximal resemblance between the test template and the reference template to achieve music signal characteristic identification and identify music tunes and music modes based on the identification results. The application layer music feature analysis block adopts dynamic time regularization algorithm and mel-frequency cepstrum coefficient to achieve music signal feature recognition and identify music tunes and music patterns based on the recognition results. We have verified through experiments, and the results show that the system operates consistently, can obtain high-quality music samples, and can extract good music characteristics.

Aircraft Engine Audio Signal Analysis in Assisting Maintenance Inspections

As the core component of modern commercial aircraft, turbofan engines have long been the center of focus in aircraft maintenance. Being subject to high temperatures and immense pressures causes problems for the engine components, such as the fan blades, as they are frequently burdened with the potential of overhaul and malfunction. Over many years, the industry has seen various methods of engine inspection and maintenance, ranging from manual inspection to computing large quantities of pre-existing data. Within, audio signal analysis has stood out as a productive, non-invasive method, with many alternate studies analyzing sound signals from components such as the combustion chamber. However, many of these methods, despite demonstrating good accuracy, are incredibly complex and require sophisticated apparatus. Therefore, this study begins by investigating the sound generation process of turbofan engines, especially how the features and form of the fan blade characterize its audio signals. This investigation proposes a solution that utilizes a fast Multi-Class Support Vector Machine (SVM) algorithm based on fan-blade-related audio signals from a perspective similar to the classification of music and images through supervised machine learning. Experimental results show that this fast Multi-Class SVM is more effective than traditional machine learning methods in its accuracy, F1-score, and other indicators.

A Deep Convolutional Neural Network Based Approach for Effective Neonatal Cry Classification

Background: The cry is the universal language for babies to communicate with others. Infant cry classification is a kind of speech recognition problem that should be treated wisely. In the last few years, it has been gaining its momentum which will be very helpful for the caretaker. Objective: This study aims to develop infant cry classification system predictive model by converting the audio signals into spectrogram image then implementing deep convolutional neural network. It performs end to end learning process and thereby reducing the complexity involved in audio signal analysis and improves the performance using optimization technique. Method: A time frequency-based analysis called Short Time Fourier Transform (STFT) is applied to generate the spectrogram. 256 DFT (Discrete Fourier Transform) points are considered to compute the Fourier transform. A Deep convolutional neural network called AlexNet with few enhancements is done in this work to classify the recorded infant cry. To improve the effectiveness of the above mentioned neural network, Stochastic Gradient Descent with Momentum (SGDM) is used to train the algorithm. Results: A deep neural network-based infant cry classification system achieves a maximum accuracy of 95% in the classification of sleepy cries. The result shows that convolutional neural network with SGDM optimization acquires higher prediction accuracy. Conclusion: Since this proposed work is compared with convolutional neural network with SGD and Naïve Bayes and based on the result, it is implied the convolutional neural network with SGDM performs better than the other techniques.

Segmentation of Multitalker Mixtures Based on Local Feature Contrasts and Auditory Glimpses

The separation of unknown sources from an acoustic scene remains one of the main challenges in the analysis of audio signals. In this paper we address the problem of source separation in multitalker scenarios, attempting a speaker-independent segmentation of the scene, based on the analysis of relative changes in the grouping features, followed by an ideal binary mask (IBM) estimation. In the presented algorithm, periodicity, power, and spatial features are combined in a general framework to estimate the on- and offsets of acoustic sources. A deep neural network (DNN) is then applied to convert relative feature changes into spectro-temporal contrast maps which are used to extract auditory glimpses, i.e. spectro-temporal segments which are dominated by the same source. By exploiting relative feature changes instead of absolute source properties, the presented glimpse formation is well applicable to unseen acoustic conditions or speakers. Using retrospective labeling, the formed glimpses are then assigned to fixed azimuth locations, providing the final IBM estimate. A systematic evaluation with up to five competing speakers and interfering diffuse noise shows that the segmentation procedure generalizes well to unseen speakers as well as to untrained noise conditions and source numbers. A comparison between the final IBM estimates and the output of a baseline system, which performs the source segregation based on spatial cues only, shows that the contrast based segmentation utilizing multiple features and spectro-temporal context can improve the quality of the estimated IBMs, in particular in conditions with multiple interfering talkers.

Fundamental Frequencies Contour Extraction Based on the Extended Harmonic-Percussive Source Separation

Introduction. Amidst the multiplicity of audio signal processing tasks connected with the necessity of source separation, the problem of obtaining of its most prominent components occupies special place. One of the approaches, dedicated to solve such task, is based on melody extraction from musical audio recording. If generalized, such technique can be applied to a wider class of audio signals for extraction from them of the so-called fundamental frequencies contour. For its realization, an attempt was made to combine the method of median filtering with the salience estimation method, for their application at various stages of the analysis of the input audio signal. A combination of methods is used to obtain the f_0-representation of the melody, based on the processing of the filtered values obtained at the first step. The purpose of the article is to obtain the trajectory of f_0-values of the input audio signal and filtering of the corresponding to this trajectory harmonics. Results. The proposed method is effective for use in audio signal processing systems for fundamental frequencies contour of the most prominent tonal components extraction and its further reuse. Conclusions. Realized spectrum decomposition technique, based on the tendency of the time-frequency distribution of its constituent sounds, allows to effectively extract melodic contours from non-melodic contours. Nevertheless, there is a necessity for further research regarding the distribution model of harmonic and percussive characteristics relative to each other. Such a model should be extended with heuristics for more accurate filtering of stable in time tonal components of complex audio signals.

Deep Learning-Based Speech Emotion Recognition Using Multi-Level Fusion of Concurrent Features

The detection and classification of emotional states in speech involves the analysis of audio signals and text transcriptions. There are complex relationships between the extracted features at different time intervals which ought to be analyzed to infer the emotions in speech. These relationships can be represented as spatial, temporal and semantic tendency features. In addition to emotional features that exist in each modality, the text modality consists of semantic and grammatical tendencies in the uttered sentences. Spatial and temporal features have been extracted sequentially in deep learning-based models using convolutional neural networks (CNN) followed by recurrent neural networks (RNN) which may not only be weak at the detection of the separate spatial-temporal feature representations but also the semantic tendencies in speech. In this paper, we propose a deep learning-based model named concurrent spatial-temporal and grammatical (CoSTGA) model that concurrently learns spatial, temporal and semantic representations in the local feature learning block (LFLB) which are fused as a latent vector to form an input to the global feature learning block (GFLB). We also investigate the performance of multi-level feature fusion compared to single-level fusion using the multi-level transformer encoder model (MLTED) that we also propose in this paper. The proposed CoSTGA model uses multi-level fusion first at the LFLB level where similar features (spatial or temporal) are separately extracted from a modality and secondly at the GFLB level where the spatial-temporal features are fused with the semantic tendency features. The proposed CoSTGA model uses a combination of dilated causal convolutions (DCC), bidirectional long short-term memory (BiLSTM), transformer encoders (TE), multi-head and self-attention mechanisms. Acoustic and lexical features were extracted from the interactive emotional dyadic motion capture (IEMOCAP) dataset. The proposed model achieves 75.50% and 75.82% of weighted and unweighted accuracy, 75.32% and 75.57% of recall and F1 score respectively. These results imply that concurrently learned spatial-temporal features with semantic tendencies learned in a multi-level approach improve the model’s effectiveness and robustness.

STUDY OF FUNDAMENTAL MACROSCOPIC 
 CHARACTERISTICS OF TRAFFIC FLOWS 
 BY WAVELET ANALYSIS OF AUDIO SIGNALS

The paper proposes an approach to the study of the traffic flow characteristics by wavelet analysis of audio signals, which makes it possible to build fundamental macroscopic models of traffic flows with the possibility of dividing by types of vehicles.

FEATURES OF ANALYSIS OF MULTICHANNEL AUDIO SIGNALSFEATURES OF ANALYSIS OF MULTICHANNEL AUDIO SIGNALS

The rapid growth of audio content has led to the need to use tools for analysis and quality control of audio signals using software and hardware and modules. The fastest-growing industry is software and programming languages.The Python programming language today has the most operational and visual capabilities for working with sound. When developing programs for computational signal analysis, it provides the optimal balance of high and low-level programming functions. Compared to Matlab or other similar solutions, Python is free and allows you to create standalone applications without the need for large, permanently installed files and a virtual environment.

Biomedical Diagnosis of Infant Cry Signal Based on Analysis of Cepstrum by Deep Feedforward Artificial Neural Networks

The automatic analysis and detection of audio signals is an important field of research with promising applications in various biomedical engineering problems such as speech, heart murmur, and lung sound analysis and classification. In this regard, automatic classification of infant vocalizations is becoming an appealing research area for medical diagnosis in clinical milieu. Indeed, the analysis and classification of infant cry records is a conventional non-inva-sive technique to distinguish between healthy and unhealthy infants.

Soft computing based audio signal analysis for accident prediction

PurposeRoad accidents, an inadvertent mishap can be detected automatically and alerts sent instantly with the collaboration of image processing techniques and on-road video surveillance systems. However, to rely exclusively on visual information especially under adverse conditions like night times, dark areas and unfavourable weather conditions such as snowfall, rain, and fog which result in faint visibility lead to incertitude. The main goal of the proposed work is certainty of accident occurrence.Design/methodology/approachThe authors of this work propose a method for detecting road accidents by analyzing audio signals to identify hazardous situations such as tire skidding and car crashes. The motive of this project is to build a simple and complete audio event detection system using signal feature extraction methods to improve its detection accuracy. The experimental analysis is carried out on a publicly available real time data-set consisting of audio samples like car crashes and tire skidding. The Temporal features of the recorded audio signal like Energy Volume Zero Crossing Rate 28ZCR2529 and the Spectral features like Spectral Centroid Spectral Spread Spectral Roll of factor Spectral Flux the Psychoacoustic features Energy Sub Bands ratio and Gammatonegram are computed. The extracted features are pre-processed and trained and tested using Support Vector Machine (SVM) and K-nearest neighborhood (KNN) classification algorithms for exact prediction of the accident occurrence for various SNR ranges. The combination of Gammatonegram with Temporal and Spectral features of the validates to be superior compared to the existing detection techniques.FindingsTemporal, Spectral, Psychoacoustic features, gammetonegram of the recorded audio signal are extracted. A High level vector is generated based on centroid and the extracted features are classified with the help of machine learning algorithms like SVM, KNN and DT. The audio samples collected have varied SNR ranges and the accuracy of the classification algorithms is thoroughly tested.Practical implicationsDenoising of the audio samples for perfect feature extraction was a tedious chore.Originality/valueThe existing literature cites extraction of Temporal and Spectral features and then the application of classification algorithms. For perfect classification, the authors have chosen to construct a high level vector from all the four extracted Temporal, Spectral, Psycho acoustic and Gammetonegram features. The classification algorithms are employed on samples collected at varied SNR ranges.

Android Collusion Detection by means of Audio Signal Analysis with Machine Learning techniques

Smartphones, tablets and other mobile devices have become objects that we can no longer do without, as a matter of fact for us they are like an extension of our body and many people are addicted to them; this behavior is a consequence of the use we make of it, since these devices allow us to manage sensitive data (i.e., financial ones) and access information of different types (i.e., photos, messages or health data). For this reason it is essential to detect the harmful behaviors present within our smartphones, taking into account the weaknesses of the current anti-malware mechanisms. In this article we propose an approach capable of discriminating trusted applications from those that instead have malicious behavior, since they are involved in a colluding attack. We resort to the processing of the audio signal extracted from the conversion of an application into an audio file. The processing allows to generate a vector of characteristics to be analyzed with different classifiers. The experimental analysis is performed on a set of Android applications consisting of 359 trusted and (colluding) untrusted applications, showing the effectiveness of our method in detecting colluding applications.

Infant Crying Classification by Using Genetic Algorithm and Artificial Neural Network

Cry as the only way of communication of babies with the surrounding environment can be happened for many reasons such as diseases, suffocation, hunger, cold and heat feeling, pain and etc. So, the analysis and detection of its source are very important for parents and health care providers. So the present study designed with the aim to test the performance of neural networks in the identification of the source of babies crying. The present study combines the genetic algorithm and artificial neural network with (Linear Predictive Coding) LPC and MFCC (Mel-Frequency Cepstral Coefficients) to classify the babies crying. The results of this study indicate the superiority of the proposed method compared to the other previous methods. This method could achieve the highest accuracy in the classification of newborns crying among the previous studies. Developing methods for classification audio signal analysis are promising and can be effectively applied in different areas such as babies crying.

Comparing traffic intensity estimates employing passive acoustic Radar and microwave Doppler Radar sensor

The purpose of our applied research project is to develop an autonomous road sign with built-in radar devices of our design. In this paper, we show that it is possible to calibrate the acoustic vector sensor in such a way that it can be used to measure traffic volume and to count the vehicles involved in the traffic through the analysis of the noise emitted by them. Signals obtained from a Doppler radar are used as a reference source. Although the acoustical vector sensor (AVS), being the embodiment of acoustic radar, has a lower accuracy than Doppler radar in vehicle counting and it is not able to measure the vehicle speed with the same precision, it has some advantages over the Doppler sensor. Namely, it does not emit any signals, it is not susceptible to electromagnetic interferences and it allows for further analysis of audio signals, such as assessment of the road surface state (e.g., wet/dry). The acoustic radar we developed is a new proposition of the acoustic method for road traffic monitoring. In addition, our research allowed a comparison of the efficiency of both methods, i.e., microwave and acoustic ones. [Project No. POIR.04.01.04-0089/16 entitled: “INZNAK – Intelligent road signs with V2X interface for adaptive traffic controlling” is financed by the Polish National Centre for Research and Development from under the EU Operational Programme Innovative Economy.]

Анализ спектра звукового сигнала

Анализ спектра звукового сигнала

Popular Music as Entertainment Communication: How Perceived Semantic Expression Explains Liking of Previously Unknown Music

Our contribution addresses popular music as essential part of media entertainment offerings. Prior works explained liking for specific music titles in ‘push scenarios’ (radio programs, music recommendation, curated playlists) by either drawing on personal genre preferences, or on findings about ‘cognitive side effects’ leading to a preference drift towards familiar and society-wide popular tracks. However, both approaches do not satisfactorily explain why previously unknown music is liked. To address this, we hypothesise that unknown music is liked the more it is perceived as emotionally and semantically expressive, a notion based on concepts from media entertainment research and popular music studies. By a secondary analysis of existing data from an EU-funded R&D project, we demonstrate that this approach is more successful in predicting 10000 listeners’ liking ratings regarding 549 tracks from different genres than all hitherto theories combined. We further show that major expression dimensions are perceived relatively homogeneous across different sociodemographic groups and countries. Finally, we exhibit that music is such a stable, non-verbal sign-carrier that a machine learning model drawing on automatic audio signal analysis is successfully able to predict significant proportions of variance in musical meaning decoding.

OSAS assessment with entropy analysis of high resolution snoring audio signals

Snoring is one the earliest symptoms of OSAS and is considered a coarse indicator of muscular tone deficiency that may compromise the regular breathing cycle. The present work intends to systematize the snore audio analysis in a cross-sectional study, with convenience sampling, of 67 individuals that undertook multi-parametric PSG analysis, during the diagnostics and OSAS severity classification process. A complete recording audio session was performed for each of the subjects while undergoing a PSG at the clinical facilities. Audio records were offline processed, in order to synchronize with the PSG data, and to determine the individual events (snores) features such as timing and Shannon entropy. This latter is taken as a stochastic measurement of complexity of the snore events and cross correlated with the clinical 5-class classification (Control, Snore, Mild, Moderate, or Severe) performed by the clinical team. For each patient, the 75–25 percentile difference, for the set of entropy values, has been calculated and the determined values were clustered according to the patients’ medical class. The statistical distribution of each class returns parameters evolving with OSAS severity in a strictly monotonic behaviour. Those parameters are p50c − p25c, and p50c and p25c after the introduction of a weighting factor. These are, therefore, the best features to be used on an event driven analysis of time-series snores that aims at class discrimination.

A computational model for predicting perceived musical expression in branding scenarios

We describe the development of a computational model predicting listener-perceived expressions of music in branding contexts. Representative ground truth from multi-national online listening experiments was combined with machine learning of music branding expert knowledge, and audio signal analysis toolbox outputs. A mixture of random forest and traditional regression models is able to predict average ratings of perceived brand image on four dimensions. Resulting cross-validated prediction accuracy (R²) was Arousal: 61%, Valence: 44%, Authenticity: 55%, and Timeliness: 74%. Audio descriptors for rhythm, instrumentation, and musical style contributed most. Adaptive sub-models for different marketing target groups further increase prediction accuracy.

Alohomora: Motion-Based Hotword Detection in Head-Mounted Displays

With the development of multimedia and computer graphics technologies, virtual reality (VR) is attracting more and more attention from both the academic communities and industrial companies. A head-mounted display (HMD) is the core equipment of VR. It envelops the entire sight of the wearer and reacts to some specific actions, mainly the head movement. Different from common video watching or game playing, VR poses the strict requirement of immersion so interaction methods need to be carefully designed. The hotword-based interaction as a typical hands-free method is very suitable for VR scenarios. However, the traditional hotword detection methods use a microphone to permit audio signal analysis. They not only incur significant recording overheads but are also susceptible to the surrounding noises. Instead of using the audio signals, we propose a motion-based hotword detection method called Alohomora. A multivariate time series (MTS) classification is formulated for processing the sensor data from multiple dimensions and types of motion sensors. We use a word extraction method for extracting and selecting patterns from MTS of motion data. Then, a classification model is trained using those discriminative patterns and finally the hotword can be detected in time. Alohomora is purely based on the motion sensors in HMDs without using any extra components such as microphone. As head tracking is always necessary in VR applications themselves, the overhead of Alohomora is nearly negligible. Finally, through extensive experiments, the final detection accuracy of Alohomora can exceed 90%.