Spectrogram Analysis Research Articles

Vibration spectrogram analysis for bearing fault diagnosis based on grad-cam for feature selection and statistical approach

Vibration spectrogram analysis for bearing fault diagnosis based on grad-cam for feature selection and statistical approach

Unsteadiness in vacuum ejector and their sources

This study investigates unsteadiness in the secondary chambers of a vacuum ejector during its transient startup phase. Experiments reveal unsteadiness at various frequencies across different nozzle pressure ratios. These frequencies are categorized as symmetric (in-phase) or asymmetric (counter-phase) through cross-spectral phase analysis. Dynamic mode decomposition, using time-resolved schlieren images, elucidates the corresponding flow structures and acoustic fields. The mode shapes display alternating density gradients along both the longitudinal and lateral axes, which explain the oscillatory behavior of the jet. These oscillations arise from instabilities in the jet's shear layer and acoustic disturbances due to the jet being confined within a duct. Using empirical relations, spectrogram analysis, time-resolved schlieren imaging, and classical wave theory, the sources of unsteadiness in the secondary chamber are identified as jet screech, its harmonics, and modes due to duct acoustics (symmetric and asymmetric). Additionally, the screech feedback loop in the vacuum ejector is found to be distinct from that in open jets, and the effect of screech on duct-mode acoustics is also observed.

Identifying chaotic dynamics in noisy time series through multimodal deep neural networks

Abstract Chaos detection is the problem of identifying whether a series of measurements is being sampled from an underlying set of chaotic dynamics. The unavoidable presence of measurement noise significantly affects the performance of chaos detectors, as discerning chaotic dynamics from stochastic signals becomes more challenging. This paper presents a computationally efficient multi-modal deep neural network tailored for chaos detection by combining information coming from the analysis of time series, recurrence plots and spectrograms. The proposed approach is the first one suitable for multi-class classification of chaotic systems while being robust with respect to measurement noise, and is validated on a dataset of 15 different chaotic and non-chaotic dynamics subject to white, pink or brown coloured noise.

2D Spectrogram analysis using vision transformer to detect mispronounced Arabic utterances for children

Pronunciation feedback is essential for teaching languages to children, urging the need to create computer-assisted pronunciation training (CAPT) systems to automate this process. Most of the current CAPT systems for Arabic Language examined a wide age range, including children, with a dominant adults’ portion. However, these systems did not consider many challenges facing children’s voice, in terms of the acoustic difference between children and adults. Moreover, due to the lack of publicly Arabic datasets, Arabic-related systems were evaluated on datasets consisting of the sound of Arabic letters, with minor efforts to contain words or sentences. In this paper, we propose the Arabic Utterance Mispronunciation Detector for Children (AUMD-Child) system that detects the mispronunciation of children for Arabic words at real-time. The proposed system is based on the fusion of Vision Transformer (ViT) and transfer learning-based model (AlexNet). A novel public dataset is constructed, in which 16 words have been collected from children aged between 7 and 12 years old to evaluate our proposed system. The experimental results, compared to the state-of-the-art models in literature, showed that the accuracy of our proposed AUMD-Child system exceeded the handcrafted features-based models by an average of 7 %, the CNN features-based models using AlexNet and SVM by 5 %, transfer learning-based model using AlexNet by 4 %, and transformers-based model using ViT by 2 %, achieving an average accuracy of 91.81 % for detecting the mispronunciation patterns in Arabic words but in an average processing time of 33 ms, exceeding the other state-of-the-art models with 13 %, 0.03 %, 0.06 %, and 23 % respectively.

Effect of Aging on Long-Term Functional Outcomes After Open Partial Laryngectomy.

Open partial horizontal laryngectomies (OPHLs) represent a comparable alternative to total laryngectomy and nonsurgical protocols in selected cases. While short-term functional outcomes of OPHLs have been widely investigated, few have focused on the effect of aging on residual laryngeal structures. Retrospective cohort study. Tertiary care academic center. Patients who underwent OPHLs after at least 1 year follow-up and optimal functional rehabilitation were included in the study. Swallowing function was assessed according to PAS (Penetration aspiration scale) and Pooling scores. Spectrogram analysis of voice was conducted according to Yanagihara classification and acoustic parameters were also recorded. Subjective questionnaire data about phonation and swallowing were also recorded. Data obtained were compared among patients according to age at time of surgery, evaluation and duration of follow-up. Ninety-seven patients were enrolled with a mean age at surgery and evaluation of 63 and 70 years old, respectively. Median follow-up length was 5 years. OPHL type II was mostly performed. No significant correlation was observed between most of the analyzed variables and patient's age at the time of surgery and at the time of evaluation. Some acoustic parameters were negatively correlated with follow-up length, while Jitter, NHR (Noise-Harmonic Ratio), and Global grade and Roughness were significantly higher in patients >65 years old. Patients who complete rehabilitation reach equally good results as their younger peers with stability over time. Finally, the effects of aging on residual larynx are of minor entity compared to the nonoperated patients. Level IV-retrospective cohort study.

Spectrogram Analysis for Lovebird Voice Characterization Based on Age

Lovebirds are known to be energetic and active birds. The chirps produced by these birds are also known to be very melodious. This study aims to analyze the sound characterization of lovebirds of various age ranges such as the amplitude and frequency produced. The method used in this research is a qualitative analysis non-experiment. This research uses spectrogram analysis in the process of processing existing data. The results obtained in this study are the frequency values at the age of 1 month, 2 months, 4 months, and 7 months in a row, namely 3257.81 Hz, 3416.01 Hz, 3533.20 Hz, and 3937.50 Hz. In addition, the amplitude values obtained at the age of 1 month, 2 months, 4 months, and 7 months were 2.96 Pct FS, 2.99 Pct FS, 3.29 Pct FS, and 8.45 Pct FS, respectively. This shows that the increase in frequency and amplitude values is proportional to the increase in age. Then it is also obtained that the frequency value produced in the age range of one to seven months is in the range of 3000 Hz - 4000 Hz.

가상악기를 활용한 거문고와 첼로의 음색비교 연구

This study aimed to compare and analyze the timbre of the geomungo and cello, focusing on harmonic analysis to examine the tonal characteristics of both instruments. Using pizzicato, a common playing technique for both instruments, A2, A3, and A4 notes were recorded, and the sound sources were generated using virtual instruments. By comparing the timbre of the two instruments through spectrum, spectrogram, envelope, and sound pressure analysis methods, it was found that the geomungo showed a stronger influence of harmonics than the fundamental frequency, characterized by a very fast attack and rapid release, which are key factors in forming the geomungo’s clear and bright tone. In the case of the cello, the fundamental frequency had a stronger influence than the harmonics, and it was found that the presence of subharmonics and non-harmonic partials, along with a fast attack and slow release, contribute to its deep and rich tone. This study provides an understanding of the tonal characteristics of the geomungo and cello, and it is hoped that this will contribute to the harmonious fusion of traditional Korean and Western music, as well as foster new musical diversity and creativity.

Detection and Classification of Obstructive Sleep Apnea Using Audio Spectrogram Analysis

Sleep disorders are steadily increasing in the population and can significantly affect daily life. Low-cost and noninvasive systems that can assist the diagnostic process will become increasingly widespread in the coming years. This work aims to investigate and compare the performance of machine learning-based classifiers for the identification of obstructive sleep apnea–hypopnea (OSAH) events, including apnea/non-apnea status classification, apnea–hypopnea index (AHI) prediction, and AHI severity classification. The dataset considered contains recordings from 192 patients. It is derived from a recently released dataset which contains, amongst others, audio signals recorded with an ambient microphone placed ∼1 m above the studied subjects and apnea/hypopnea accurate events annotations performed by specialized medical doctors. We employ mel spectrogram images extracted from the environmental audio signals as input of a machine-learning-based classifier for apnea/hypopnea events classification. The proposed approach involves a stacked model which utilizes a combination of a pretrained VGG-like audio classification (VGGish) network and a bidirectional long short-term memory (bi-LSTM) network. Performance analysis was conducted using a 5-fold cross-validation approach, leaving out patients used for training and validation of the models in the testing step. Comparative evaluations with recently presented methods from the literature demonstrate the advantages of the proposed approach. The proposed architecture can be considered a useful tool for supporting OSAHS diagnoses by means of low-cost devices such as smartphones.

Anterior thalamic nucleus local field potentials during focal temporal lobe epileptic seizures.

To analyze the local field potentials (LFPs) in patients with focal drug-resistant epilepsy (DRE) from the anterior nucleus of the thalamus (ANT) during inter-ictal state and seizure state. ANT stereotactic EEG (SEEG) recordings were studied in four patients with focal temporal lobe epilepsy. SEEG data was classified as inter-ictal and ictal state and sub-categorized into electrographic (ESz), focal aware seizure (FAS), focal with impaired awareness (FIA), or focal to bilateral tonic-clonic seizure (FBTC). LFP was analyzed at 4 Hz, 8 Hz, 16 Hz, 32 Hz, high gamma (100 Hz), and ripples (200 Hz) using spectrogram analysis and a statistical comparison of normalized power spectral density (PSD) averaged during seizures versus pre-ictal baseline segments. The LFP recordings were analyzed for 162 seizures (127 ESz, 23 FAS, 6 FIA, and 6 FBTC). Based on time-frequency data (spectrogram), a broad band of activity, occurring between 2 and 6 Hz and centered at 4 Hz, and thin-band activity occurring specifically at 8 Hz on the frequency spectrogram were observed during the inter-ictal state. Statistically significant changes in LFP-PSD were seen for FAS, FIA, and FBTC. We observed a significant gain in LFP at the lower frequency band during FAS at 4 Hz, FIA, and FBTC at 4, 8, and 16 Hz while also observing increases at higher frequencies during FBTC at 100 and 200 Hz and a decrease during FAS seizures at 32 Hz. In contrast, no significant change in LFP power was seen for electrographic seizures. Our observations from a limited dataset indicate that all clinical seizure types, but not electrographic seizures, caused a change in ANT-LFP based on the magnitude of the associated power spectral density (PSD). Future work will be needed to validate the use of ANT-LFP at these frequencies as accurate measurements of seizure occurrence and severity. This work represents a first step toward understanding ANT thalamic LFP patterns during focal seizures and developing adaptive DBS strategies.

Vision Transformer Based Classification of Neurological Disorders from Human Speech

In this study, we introduce a transformative approach to achieve high-accuracy classification of distinct health categories, including Parkinson's disease, Multiple Sclerosis (MS), healthy individuals, and other categories, utilizing a transformer-based neural network. The cornerstone of this approach lies in the innovative conversion of human speech into spectrograms, which are subsequently transformed into visual images. This transformation process enables our network to capture intricate vocal patterns and subtle nuances that are indicative of various health conditions. The experimental validation of our approach underscores its remarkable performance, achieving exceptional accuracy in differentiating Parkinson's disease, MS, healthy subjects, and other categories. This breakthrough opens doors to potential clinical applications, offering an innovative, non-invasive diagnostic tool that rests on the fusion of spectrogram analysis and transformer-based models.

Enhancing Yarn Quality Wavelength Spectrogram Analysis: A Semi-Supervised Anomaly Detection Approach with Convolutional Autoencoder

Abnormal detection plays a pivotal role in the routine maintenance of industrial equipment. Malfunctions or breakdowns in the drafting components of spinning equipment can lead to yarn defects, thereby compromising the overall quality of the production line. Fault diagnosis of spinning equipment entails the examination of component defects through Wavelet Spectrogram Analysis (WSA). Conventional detection techniques heavily rely on manual experience and lack generality. To address this limitation, this current study leverages machine learning technology to formulate a semi-supervised anomaly detection approach employing a convolutional autoencoder. This method trains deep neural networks with normal data and employs the reconstruction mode of a convolutional autoencoder in conjunction with Kernel Density Estimation (KDE) to determine the optimal threshold for anomaly detection. This facilitates the differentiation between normal and abnormal operational modes without the necessity for extensive labeled fault data. Experimental results from two sets of industrial data validate the robustness of the proposed methodology. In comparison to conventional Autoencoder and prevalent machine learning techniques, the proposed approach demonstrates superior performance across evaluation metrics such as Accuracy, Recall, Area Under the Curve (AUC), and F1-score, thereby affirming the feasibility of the suggested model.

Liszt’s Étude S.136 no.1: audio data analysis of two different piano recordings

In this paper, we review the main signal processing tools of Music Information Retrieval (MIR) from audio data, and we apply them to two recordings (by Leslie Howard and Thomas Rajna) of Franz Liszt’s Étude S.136 no.1, with the aim of uncovering the macro-formal structure and comparing the interpretative styles of the two performers. In particular, after a thorough spectrogram analysis, we perform a segmentation based on the degree of novelty, in the sense of spectral dissimilarity, calculated frame-by-frame via the cosine distance. We then compare the metrical, temporal and timbrical features of the two executions by MIR tools. Via this method, we are able to identify in a data-driven way the different moments of the piece according to their melodic and harmonic content, and to find out that Rajna’s execution is faster and less various, in terms of intensity and timbre, than Howard’s one. This enquiry represents a case study able to show the potentialities of MIR from audio data in supporting traditional music score analyses and in providing objective information for statistically founded musical execution analyses.

Distinguishing Hawai‘i Creole neva and néva: prosodic evidence from podcast interviews

Abstract This paper investigates prosodic and contextual differentiation strategies between preverbal past tense negator neva and adverb never in Hawai‘i Creole. It aims to demonstrate differing syntactic restrictions and advocates for treating these words as two distinct morphemes. The analyses are based on phonological data gathered from interviews uploaded onto YouTube by Hawaiiverse, a Local podcast. As demonstrated through spectrogram analyses, HC néva (=ɴᴇᴠᴇʀ) is marked by a stressed accent on its first syllable and prominence on word-initial /n/, whereas neva (=ᴅɪᴅɴ’ᴛ) is marked by a lack of these features under typical circumstances. This suggests that Hawai‘i Creole morphophonology depends more on stress-timed features than previously researched (cf. syllable-timed features). Neva-néva ambiguity may arise when stress does not clearly indicate which word is being used, and when context cannot be relied upon to distinguish meaning. By exploring these intricacies, this investigation offers insight into how future researchers may approach analysing other English-lexified creoles (and varieties of English) which also use never as a preverbal past tense negator.

Predicting COVID-19 Cough Sounds Using Spectrogram Analysis Across Multiple Classes

The COVID-19 pandemic has underscored the need for effective diagnostic tools. One promising avenue involves analyzing cough sounds to glean insights into respiratory health. This study presents a new method for predicting COVID-19 cough sounds using spectrogram analysis across various classes. We leverage advanced deep learning models such as DenseNet121, VGG16, ResNet50, and Inception Net, alongside our novel CNN architecture, to extract pertinent features from cough sound spectrograms. We use a diverse dataset encompassing cough sounds from COVID-19 positive and negative cases, as well as other respiratory conditions, for model training and assessment. Our results demonstrate the effectiveness of our approach in accurately categorizing COVID-19 cough sounds, outperforming existing models. This methodology shows promise as a non-invasive, scalable, and economical tool for early COVID-19 detection and monitoring, aiding public health efforts during the pandemic.

Multichannel Hierarchical Analysis of Time-Resolved Hyperspectral Data for Advanced Colorimetric E-Nose.

The colorimetric sensor-based electronic nose has been demonstrated to discriminate specific gaseous molecules for various applications, including health or environmental monitoring. However, conventional colorimetric sensor systems rely on RGB sensors, which cannot capture the complete spectral response of the system. This limitation can degrade the performance of machine learning analysis, leading to inaccurate identification of chemicals with similar functional groups. Here, we propose a novel time-resolved hyperspectral (TRH) data set from colorimetric array sensors consisting of 1D spatial, 1D spectral, and 1D temporal axes, which enables hierarchical analysis of multichannel 2D spectrograms via a convolution neural network (CNN). We assessed the outstanding classification performance of the TRH data set compared to an RGB data set by conducting a relative humidity (RH) concentration classification. The time-dependent spectral response of the colorimetric sensor was measured and trained as a CNN model using TRH and RGB sensor systems at different RH levels. While the TRH model shows a high classification accuracy of 97.5% for the RH concentration, the RGB model yields 72.5% under identical conditions. Furthermore, we demonstrated the detection of various functional volatile gases with the TRH system by using experimental and simulation approaches. The results reveal distinct spectral features from the TRH system, corresponding to changes in the concentration of each substance.

Hear You Say You: An Efficient Framework for Marine Mammal Sounds’ Classification

Marine mammals and their ecosystem face significant threats from, for example, military active sonar and marine transportation. To mitigate this harm, early detection and classification of marine mammals are essential. While recent efforts have utilized spectrogram analysis and machine learning techniques, there remain challenges in their efficiency. Therefore, we propose a novel knowledge distillation framework, named XCFSMN, for this problem. We construct a teacher model that fuses the features extracted from an X-vector extractor, a DenseNet and Cross-Covariance attended compact Feed-Forward Sequential Memory Network (cFSMN). The teacher model transfers knowledge to a simpler cFSMN model through a temperature-cooling strategy for efficient learning. Compared to multiple convolutional neural network backbones and transformers, the proposed framework achieves state-of-the-art efficiency and performance. The improved model size is approximately 20 times smaller and the inference time can be 10 times shorter without affecting the model’s accuracy.

Implementing machine learning techniques for continuous emotion prediction from uniformly segmented voice recordings.

Emotional recognition from audio recordings is a rapidly advancing field, with significant implications for artificial intelligence and human-computer interaction. This study introduces a novel method for detecting emotions from short, 1.5 s audio samples, aiming to improve accuracy and efficiency in emotion recognition technologies. We utilized 1,510 unique audio samples from two databases in German and English to train our models. We extracted various features for emotion prediction, employing Deep Neural Networks (DNN) for general feature analysis, Convolutional Neural Networks (CNN) for spectrogram analysis, and a hybrid model combining both approaches (C-DNN). The study addressed challenges associated with dataset heterogeneity, language differences, and the complexities of audio sample trimming. Our models demonstrated accuracy significantly surpassing random guessing, aligning closely with human evaluative benchmarks. This indicates the effectiveness of our approach in recognizing emotional states from brief audio clips. Despite the challenges of integrating diverse datasets and managing short audio samples, our findings suggest considerable potential for this methodology in real-time emotion detection from continuous speech. This could contribute to improving the emotional intelligence of AI and its applications in various areas.

CNN-based noise reduction for multi-channel speech enhancement system with discrete wavelet transform (DWT) preprocessing.

Speech enhancement algorithms are applied in multiple levels of enhancement to improve the quality of speech signals under noisy environments known as multi-channel speech enhancement (MCSE) systems. Numerous existing algorithms are used to filter noise in speech enhancement systems, which are typically employed as a pre-processor to reduce noise and improve speech quality. They may, however, be limited in performing well under low signal-to-noise ratio (SNR) situations. The speech devices are exposed to all kinds of environmental noises which may go up to a high-level frequency of noises. The objective of this research is to conduct a noise reduction experiment for a multi-channel speech enhancement (MCSE) system in stationary and non-stationary environmental noisy situations with varying speech signal SNR levels. The experiments examined the performance of the existing and the proposed MCSE systems for environmental noises in filtering low to high SNRs environmental noises (-10 dB to 20 dB). The experiments were conducted using the AURORA and LibriSpeech datasets, which consist of different types of environmental noises. The existing MCSE (BAV-MCSE) makes use of beamforming, adaptive noise reduction and voice activity detection algorithms (BAV) to filter the noises from speech signals. The proposed MCSE (DWT-CNN-MCSE) system was developed based on discrete wavelet transform (DWT) preprocessing and convolution neural network (CNN) for denoising the input noisy speech signals to improve the performance accuracy. The performance of the existing BAV-MCSE and the proposed DWT-CNN-MCSE were measured using spectrogram analysis and word recognition rate (WRR). It was identified that the existing BAV-MCSE reported the highest WRR at 93.77% for a high SNR (at 20 dB) and 5.64% on average for a low SNR (at -10 dB) for different noises. The proposed DWT-CNN-MCSE system has proven to perform well at a low SNR with WRR of 70.55% and the highest improvement (64.91% WRR) at -10 dB SNR.

CARgram: CNN-based accident recognition from road sounds through intensity-projected spectrogram analysis

Road surveillance systems play an important role in traffic monitoring and detecting hazardous events. In recent years, several artificial intelligence-based approaches have been proposed for this purpose, typically based on the analysis of the acquired video streams. However, occlusions, poor lighting conditions, and heterogeneity of the events may often reduce their effectiveness and reliability. To overcome the limitations mentioned, scientific and industrial research has therefore focused on integrating such solutions with audio recognition methods. By automatically identifying anomalous traffic sounds, e.g., car crashes and skids, they help reduce false positives and missed alarms. Following this trend, in this work, we propose an innovative pipeline for the analysis of intensity-projected audio spectrograms from streams of traffic sounds, which exploits both (i) a visual approach based on a custom, special-purpose Convolutional Neural Network for the identification of anomalous events on the sound signal; and, (ii) a novel multi-representational encoding of the input, which proved to significantly improve the recognition accuracy of the neural models. The validation results of the proposed pipeline on the public MIVIA dataset, with a 0.96% of false positive rate, showed to be the best performance against the state-of-the-art competitors. Notably, following such findings, a prototype implementation has been deployed on a real-world video surveillance infrastructure.

Singing in a Second Language: Pronunciation Errors Made by Sri Lankan Decedent Students' when Singing Sinhala Songs Under the Influence of the Australian English Language

When a speaker/vocal performer sings in a second language, the pronunciation can be labelled as a different or fake accent compared to the native speakers. On a theoretical level, it is very difficult for learners of a second language (L2) to get rid of the phonetic effects of (L1) and sometimes impossible, unless the language learner can process both languages from birth. This research aims to identify the pronunciation errors made by Sri Lankan decedent students' according to the consonant sounds of the Inattentional Phonetic Alphabet when they sing Sinhala songs under the influence of the Australian English Language. This research specifically questions what consonant substitutes are employed by the aforementioned speakers for the irreplaceable consonant identity of Sinhala sound production. In this research, Sinhala and English phonemes were compared based on the data from Cox & Palethrope (2007), Rajapakshe (1994 & 1997) and previous research of this researcher (Penahetipola, 2011). The research fundamentally employs a comparison of the phonetic inventory of both languages based on the IPA charts and data from spectrogram analysis of speech sounds made by 45 Australian-born children of Sinhala migrants who are 10-12 and study Sinhala as a second language. Prenasalised stops and nasals have been identified as the most noticeable substitutes. Also, retroflex, palatal, and bilabial are the concerned sound production. The finding is very useful in developing language-speaking programs for speakers and vocal performers who practice Sinhala as their second language.