Characteristics Of Audio Signals Research Articles

Automated Determination of Music Genre and Linguistic Categorization

The main objective of the project is to build a system that allows users to predict genre as well as language out of some music . The Automated determination of musical genre and linguistic categorization is a machine learning based prediction model which is used to predict the genres and languages of an audio . Musical genres are categorical labels created by humans to characterize piece of music. A musical genre is characterized by common characteristics typically related to the instrumentation, rhythmic structure and harmonic content of the music. The performance and relative importance of the proposed features are investigated by training the statistical pattern recognition classifiers using real world audio collections and using machine learning algorithm.In this project, we built systems called music genre and language classification and the effects of different feature extraction methods and their various combinations were also investigated. This task is of great interest in music information retrieval and recommendation systems. In recent years, several machine learning algorithms have been developed to tackle this task, using features such spectral, temporal, and tonal characteristics of audio signals. Music genre prediction and language classification is a task that involves automatically predicting a piece of music into a specific genre category as well as language based on its audio features. Keywords-machine-learning,music,music genre and language classification

HBNET: A blended ensemble model for the detection of cardiovascular anomalies using phonocardiogram.

Cardiac diseases are highly detrimental illnesses, responsible for approximately 32% of global mortality [1]. Early diagnosis and prompt treatment can reduce deaths caused by cardiac diseases. In paediatric patients, it is challenging for paediatricians to identify functional murmurs and pathological murmurs from heart sounds. The study intends to develop a novel blended ensemble model using hybrid deep learning models and softmax regression to classify adult, and paediatric heart sounds into five distinct classes, distinguishing itself as a groundbreaking work in this domain. Furthermore, the research aims to create a comprehensive 5-class paediatric phonocardiogram (PCG) dataset. The dataset includes two critical pathological classes, namely atrial septal defects and ventricular septal defects, along with functional murmurs, pathological and normal heart sounds. The work proposes a blended ensemble model (HbNet-Heartbeat Network) comprising two hybrid models, CNN-BiLSTM and CNN-LSTM, as base models and Softmax regression as meta-learner. HbNet leverages the strengths of base models and improves the overall PCG classification accuracy. Mel Frequency Cepstral Coefficients (MFCC) capture the crucial audio signal characteristics relevant to the classification. The amalgamation of these two deep learning structures enhances the precision and reliability of PCG classification, leading to improved diagnostic results. The HbNet model exhibited excellent results with an average accuracy of 99.72% and sensitivity of 99.3% on an adult dataset, surpassing all the existing state-of-the-art works. The researchers have validated the reliability of the HbNet model by testing it on a real-time paediatric dataset. The paediatric model's accuracy is 86.5%. HbNet detected functional murmur with 100% precision. The results indicate that the HbNet model exhibits a high level of efficacy in the early detection of cardiac disorders. Results also imply that HbNet has the potential to serve as a valuable tool for the development of decision-support systems that aid medical practitioners in confirming their diagnoses. This method makes it easier for medical professionals to diagnose and initiate prompt treatment while performing preliminary auscultation and reduces unnecessary echocardiograms.

Lightweight network based features fusion for steel rolling ambient sound classification

With the intelligent development of industrial production, sound monitoring technology has been widely used to monitor the operation status of mechanical facilities, and this progress has gradually become a research hotspot in the steel manufacturing industry. However, the complex composition of sound sources and the high computational requirements of most models limit their applicability to industrial scenarios. This work presents a targeted approach for remote monitoring of rolling sound. They are using visual features of audio signals to design classification models on the self-harvested dataset for more efficient adaptation to complex production sites. An in-depth analysis of the actual sound reveals that it is characterized by high similarity, complexity, and partial synchronization. To optimize interclass and intraclass data engineering few-sample, multi-feature fusion, and data augmentation methods were combined to characterize the details fully. In addition, by leveraging the deep small convolutions of VGGNet and the randomness of stochastic pooling, local features are effectively extracted. Finally, global average pooling with final softmax layer stacking is used to classify the rolling signals to reduce the number of parameters, avoid overfitting, and perform a global analysis of the features. Experimental results on the rolling sound dataset show that the method proposed achieves an accuracy of 91.26%, with 92.66%, and 95.88% on the ESC10 and MIMII datasets, respectively. These results confirm that the method can be widely applied to sound classification for multi-category rolling processes, showing good performance and scalability.

Music Tone Synthesis based Anti-Interference Dynamic Integral Neural Network optimized with Artificial Hummingbird Optimization Algorithm

Music Tone Synthesis is an applied science or method that implies to identify and study of specific orchestral tone currently part of a music. It is especially effective in the area of oral music training system classes, somewhere be able to support in the practice and growth of musicians. Music Tone Synthesis provides singer and writer to produce large number of noises as well as imitate several tools and impact it cannot be flexible otherwise realistic for create over standard classical instrument. In this manuscript, Music Tone Synthesis based Anti-Interference Dynamic Integral Neural Network enhanced with artificial hummingbird Optimization algorithm (MTS-AIDINN-AHOA) is proposed. The input data are obtained from the audio signal. Then the data are pre-processing using Stein Particle Filtering (SPF) to remove the noise. The pre-processed data is given into the Two-sided Offset Quaternion Linear Canonical transform (TSOQLCT) for extracting the musical features such as melody, harmony, tempo, and dynamics. After this the extracted feature is provided to the Anti-Interference Dynamic Integral Neural Network (AIDINN) is used for the music tone synthesis and it is classified as pitch, chronaxie, volume, tone color. In general, the Anti-Interference Dynamic Integral Neural Network (AIDINN) does no express adapting optimization strategies to determine ideal parameters to assure precise prediction. Thus, it is proposed to utilize the Artificial Hummingbird Optimization Algorithm enhancement AIDINN for Music Tone Synthesis. The proposed MTS-AIDINN-AHOA method is implemented on MATLAB. Then performance of proposed technique is evaluated to other existing techniques. The proposed technique attains 26.36%, 20.69% and 35.29% higher accuracy, 19.23%, 23.56%, and 33.96% higher precision, 26.28%, 31.26%, and 19.66%higher recall, 28.96%, 33.21% and 23.89%higher specificity comparing with the existing methods such as a research on Musical Tone Recognition Method Based on Improved RNN for Vocal Music Teaching Network Courses (MTS-RNN), Music Timbre Extracted from Audio Signal Features (MTS-BPNN)and Feature Extraction and Categorization of Music Content Based on Deep Learning(MTS-SMNN) respectively.

Short Utterance Speaker Recognition Based on Speech High Frequency Information Compensation and Dynamic Feature Enhancement Methods

This work aims to further compensate for the weaknesses of feature sparsity and insufficient discriminative acoustic features in existing short-duration speaker recognition. To address this issue, we propose the Bark-scaled Gauss and the linear filter bank superposition cepstral coefficients (BGLCC), and the multidimensional central difference (MDCD) acoustic feature extracted method. The Bark-scaled Gauss filter bank focuses on low-frequency information, while linear filtering is uniformly distributed, therefore, the filter superposition can obtain more discriminative and richer acoustic features of short-duration audio signals. In addition, the multi-dimensional central difference method captures better dynamics features of speakers for improving the performance of short utterance speaker verification. Extensive experiments are conducted on short-duration text-independent speaker verification datasets generated from the VoxCeleb, SITW, and NIST SRE corpora, respectively, which contain speech samples of diverse lengths, and different scenarios. The results demonstrate that the proposed method outperforms the existing acoustic feature extraction approach by at least 10% in the test set. The ablation experiments further illustrate that our proposed approaches can achieve substantial improvement over prior methods.

Sounding Video Generator: A Unified Framework for Text-Guided Sounding Video Generation

As a combination of visual and audio signals, video is inherently multi-modal. However, existing video generation methods are primarily intended for the synthesis of visual frames, whereas audio signals in realistic videos are disregarded. In this work, we concentrate on a rarely investigated problem of text-guided sounding video generation and propose the Sounding Video Generator (SVG), a unified framework for generating realistic videos along with audio signals. Specifically, we present the SVG-VQGAN to transform visual frames and audio mel-spectrograms into discrete tokens. SVG-VQGAN applies a novel hybrid contrastive learning method to model inter-modal and intra-modal consistency and improve the quantized representations. A cross-modal attention module is employed to extract associated features of visual frames and audio signals for contrastive learning. Then, a Transformer-based decoder is used to model associations between texts, visual frames, and audio signals at token level for auto-regressive sounding video generation. AudioSet-Cap, a human annotated text-video-audio paired dataset, is produced for training SVG. Experimental results demonstrate the superiority of our method when compared with existing text-to-video generation methods as well as audio generation methods on Kinetics and VAS datasets.

Construction of a digital library of vocal music resources based on second-order data decomposition algorithm

Abstract In this paper, in order to solve the modal aliasing problem of the EMD decomposition algorithm and EEMD decomposition algorithm and optimize the vocal audio data decomposition on the basis of the adaptive VMD algorithm, the EEMD-VMD second-order data decomposition algorithm is proposed to decompose the digitized time series of vocal music resources, and to update the construction of the digital library of vocal music resources. Combined with the decomposition steps of the second-order data decomposition algorithm in the vocal resources digitized library, the overall framework of the vocal resources digitized library is designed and constructed with the main line of vocal resource types and the secondary line of content categories. Design the experimental environment, preprocess the vocal audio signal features, explore the modal number K value size of the EEMD-VMD second-order data decomposition algorithm, analyze the vocal audio denoising performance of the algorithm, and compare the correlation coefficients of the IMF components of the decomposition algorithms with the original audio signals. Explore the second-order data decomposition algorithm’s emotional classification of vocal resources and the performance of vocal signal enhancement, add noise sources, and use subjective and objective evaluation methods to assess the quality of vocal PESQ and STOI after decomposition processing. The EEMD-VMD second-order data decomposition algorithm performs the most prominently in street noise environments. The enhancement results in the case of signal-to-noise ratios of -5, -10, and -15 are 3.59, respectively, 4.36, 4.29, and the mean value of the enhancement performance reaches 4.08. The second-order data decomposition algorithm’s precise processing of vocal resources provides better quality digital resources for the construction of a digital library of vocal resources.

Innovative Teaching of College Vocal Music Courses Aided by Information Integration Technology

Abstract The article adopts the correlation function method to extract the fundamental frequency features of vocal speech signals and processes this information through the cepstrum method. Then, the audio signals are corrected by linear smoothing to match the features of the students’ audio signals with the features of the musical score to determine the accuracy of the vocal music. The study also includes quantization, frame-splitting, and windowing preprocessing of the audio signals, and constructing an unwritten audio signal feature set based on correlation feature selection. The article also proposes an innovative metaphorical method for improving the teaching mode of vocal music courses in colleges and universities. It analyzes the auxiliary role of information technology in teaching vocal music courses by combining simulation experiments with the effects of practical application. The study results show that the students’ vocal singing fundamental frequency range is close to the standard, roughly 123 to 200, with a maximum error value of 24. With the assistance of information fusion technology, the average score increment of the students’ vocal singing reaches 6.088 points, proving this method’s effectiveness in improving the quality of verbal music teaching.

A Nature Inspired Optimization Algorithm for Parkinson's Disease Classification Through Speech Analysis

This research takes a unique approach to improve the accuracy of Parkinson's disease prediction through speech signal analysis by turning to the natural world for inspiration. This research proposes an optimization algorithm modelled on the hunting strategies of eagles. This approach begins with the extraction of Mel-frequency cepstral coefficients (MFCCs) from the audio signals. Subsequently, the proposed algorithm, inspired by the keen eyesight and hunting strategy of eagles, is employed to select the most relevant MFCC features. This optimization ensures a reduced dimensionality while maintaining the integrity and relevance of the feature set. Eagle's highly effective three-pronged approach to hunting - consisting of soaring (exploring), scanning (assessing), and swooping (taking action) - is the starting point for this method. To put this into practice, the proposed algorithm treats each potential combination of audio signal features as a 'solution', or 'prey'. The usefulness or 'fitness' of each solution is evaluated according to how accurately it can predict Parkinson's disease using a Support Vector Machine (SVM) classifier on a validation dataset. At the start, our algorithm generates an array of solutions. It then takes an 'eagle's eye view', assessing and refining these solutions in successive iterations. This mimics the eagle's hunt, where it identifies its target from a distance, before swooping in to capture it. By testing the proposed eagle-inspired algorithm on the Parkinson's Disease Voice Initiative (PDVI) dataset, it was found that it offers better accuracy in differentiating between those with Parkinson's and those without, compared to traditional methods. Specifically, using the proposed algorithm's selected features, the SVM model achieved an accuracy of 93.2% compared to an 88.5% accuracy when using all MFCC features. Furthermore, the computation time was reduced by 15% due to the dimensionality reduction introduced by the proposed algorithm. In conclusion, this study opens up a promising new direction in the early detection and treatment of Parkinson's disease, demonstrating the value of looking to nature for solving complex problems.

A novel study for depression detecting using audio signals based on graph neural network

Depression is a prevalent mental health disorder. The absence of specific biomarkers makes clinical diagnosis highly subjective. This makes it difficult to make a definitive diagnosis for the patient. Recently, deep learning methods have shown promise for depression detection. However, current methods tend to focus solely on the connections within or between audio signals, leading to limitations in the model’s ability to recognize depression-related cues in audio signals and affecting its classification performance. To address these limitations, we propose a graph neural network approach for depression recognition that incorporates potential connections within and between audio signals. Specifically, we first use a gated recurrent unit (GRU) to extract time-series information between frame-level features of audio signals. We then construct two graph neural network modules sequentially to explore the potential connections within and between audio signals. The first graph network module constructs a graph using the frame-level features of each audio sample as nodes. The output is obtained as a graph-embedded feature vector representation after the graph convolution layers. Subsequently, the output graph embedding feature vector representation of the first graph network model is used as the nodes of the graph to construct the second graph network. The internal relationship between audio signals is encoded by the property of node neighborhood information propagation. In addition, we use a pre-trained emotion recognition network to extract emotional features that are highly correlated with depression. By further strengthening the connection weights among nodes in the second graph network through a self-attention mechanism, relevant cues are provided for the model to complete depression detection from audio signals. We conducted extensive experiments on three depression datasets, including DAIC-WOZ, MODMA, and D-Vlog. The proposed model achieves better results on several performance evaluation metrics such as accuracy, F1-score, precision, and recall compared to all the compared algorithms, validating its effectiveness.

An Investigation of ECAPA-TDNN Audio Type Recognition Method Based on Mel Acoustic Spectrograms

Audio signals play a crucial role in our perception of our surroundings. People rely on sound to assess motion, distance, direction, and environmental conditions, aiding in danger avoidance and decision making. However, in real-world environments, during the acquisition and transmission of audio signals, we often encounter various types of noises that interfere with the intended signals. As a result, the essential features of audio signals become significantly obscured. Under the interference of strong noise, identifying noise segments or sound segments, and distinguishing audio types becomes pivotal for detecting specific events and sound patterns or isolating abnormal sounds. This study analyzes the characteristics of Mel’s acoustic spectrogram, explores the application of the deep learning ECAPA-TDNN method for audio type recognition, and substantiates its effectiveness through experiments. Ultimately, the experimental results demonstrate that the deep learning ECAPA-TDNN method for audio type recognition, utilizing Mel’s acoustic spectrogram as features, achieves a notably high recognition accuracy.

Retracted: Music Timbre Extracted from Audio Signal Features

Retracted: Music Timbre Extracted from Audio Signal Features

Ideal ratio mask estimation based on cochleagram for audio-visual monaural speech enhancement

In this paper, the estimation of the ideal ratio mask (IRM) has been carried out based on speech cochleagram and visual cues using Audio-Visual Multichannel Convolutional Neural Network (AVMCNN) to enhance the speech signal. Recently several researchers have shown that speech enhancement using visual data as an additional input along with the audio data is more effective in minimizing the acoustic noise present in the speech signal. This work proposes a novel CNN-based audio-visual IRM estimation model. In the proposed audio-visual IRM estimation model, the dynamics of both audio and visual signal features are extracted using multichannel CNN and contextually combined for speech enhancement. The enhanced speech obtained using the proposed model is evaluated based on speech quality and intelligibility. The evaluation results signify that the proposed audio-visual mask estimation model shows improved performance over the audio-only, visual-only, and existing audio-visual mask estimation models. In turn, the proposed AVMCNN model proves its effectiveness in combining the dynamics of the audio features with the visual speech features for speech enhancement.

Emotion Classification Algorithm for Audiovisual Scenes Based on Low-Frequency Signals

Since informatization and digitization came into life, audio signal emotion classification has been widely studied and discussed as a hot issue in many application fields. With the continuous development of artificial intelligence, in addition to speech and music audio signal emotion classification technology, which is widely used in production life, its application is also becoming more and more abundant. Current research on audiovisual scene emotion classification mainly focuses on the frame-by-frame processing of video images to achieve the discrimination of emotion classification. However, those methods have the problems of algorithms with high complexity and high computing cost, making it difficult to meet the engineering needs of real-time online automatic classification. Therefore, this paper proposes an automatic algorithm for the detection of effective movie shock scenes that can be used for engineering applications by exploring the law of low-frequency sound effects on the perception of known emotions, based on a database of movie emotion scene clips in 5.1 sound format, extracting audio signal feature parameters and performing dichotomous classification of shock and other types of emotions. As LFS can enhance a sense of shock, a monaural algorithm for detecting emotional scenes with impact using a subwoofer (SW) is proposed, which trained a classification model using SW monaural features and achieved a maximum accuracy of 87% on the test set using a convolutional neural network (CNN) model. To expand the application scope of the above algorithm, a monaural algorithm for detecting emotional scenes with impact based on low-pass filtering (with a cutoff frequency of 120 Hz) is proposed, which achieved a maximum accuracy of 91.5% on the test set using a CNN model.

Fault diagnosis of plunger pump based on audio signal combined with meta-transfer learning

To solve the problems of insufficient samples and weak fault features of audio signals in the fault diagnosis of plunger pump, this paper proposes a fault diagnosis method of plunger pump based on audio signal combined with meta-transfer learning (MTL-PAFD). The method takes the audio signals of the plunger pump as samples, which are acquired by a single sensor. Through the Gammatone filter bank processing, the representation ability of the audio signal under strong noise interference is effectively improved. Then combined with meta-transfer learning, the few-shot fault diagnosis of plunger pump is realized. In addition, according to the actual needs of fault diagnosis of plunger pump, the test method of meta-transfer learning in fault diagnosis application is improved, which can process unknown fault classes adaptively. Experimental results show that MTL-PAFD has a fault diagnosis accuracy of 91.41% for seen classes. After fast adaptive learning, it can achieve an accuracy of 89.64% when identifying unseen fault classes.

Knowledge-Based Features for Speech Analysis and Classification: Pronunciation Diagnoses

Accurate pronunciation of speech sounds is essential in communication. As children learn their native language, they refine the movements necessary for intelligible speech. While there is variability in the order of acquisition of speech sounds, there are some sounds that are more complex and are later developing. The rhotic /r/ is a later-developing sound in English, and some children require intervention to achieve accurate production. Additionally, individuals learning English as a second language may have difficulty learning accurate /r/ production, especially if their native language does not have an /r/, or the /r/ they produce is at a different place of articulation. The goal of this research is to provide a novel approach on how a knowledge-based intelligence program can provide immediate feedback on the accuracy of productions. In the proposed approach, the audio signals will first be detected, after which features of audio signals will be extracted, and finally, knowledge-based intelligent classification will be performed. Based on the obtained knowledge and application scenarios, novel features are proposed and used to classify various speaker scenarios.

Voice activity detection in the presence of transient based on graph

Voice activity detection remains a significant challenge in the presence of transients since transients are more dominant than speech, though it has achieved satisfactory performance in quasi-stationary noisy environments. This paper studies the differences between speech and transients in nonlinear dynamic characteristics and proposes a new method for accurately detecting speech and transients. Limited by algorithm complexity, previous research has proposed few detectors to model speech and transients based on contextual information and thus failing to detect transient frames accurately. To address this challenge, our study proposes to map features of audio signals to a time series complex network, a kind of graph data, analyzed by the Laplacian and adjacency matrix of graphs, then classified by the support vector machine (SVM) classifier. The proposed algorithm can analyze a more extended speech period, allowing the full utilization of contextual information of preceding and following frames. The experimental results show that the performance of this method has obvious superiority over other existing algorithms.

Source Acquisition Device Identification from Recorded Audio Based on Spatiotemporal Representation Learning with Multi-Attention Mechanisms

Source acquisition device identification from recorded audio aims to identify the source recording device by analyzing the intrinsic characteristics of audio, which is a challenging problem in audio forensics. In this paper, we propose a spatiotemporal representation learning framework with multi-attention mechanisms to tackle this problem. In the deep feature extraction stage of recording devices, a two-branch network based on residual dense temporal convolution networks (RD-TCNs) and convolutional neural networks (CNNs) is constructed. The spatial probability distribution features of audio signals are employed as inputs to the branch of the CNN for spatial representation learning, and the temporal spectral features of audio signals are fed into the branch of the RD-TCN network for temporal representation learning. This achieves simultaneous learning of long-term and short-term features to obtain an accurate representation of device-related information. In the spatiotemporal feature fusion stage, three attention mechanisms-temporal, spatial, and branch attention mechanisms-are designed to capture spatiotemporal weights and achieve effective deep feature fusion. The proposed framework achieves state-of-the-art performance on the benchmark CCNU_Mobile dataset, reaching an accuracy of 97.6% for the identification of 45 recording devices, with a significant reduction in training time compared to other models.

A Lightweight Architecture for Query-by-Example Keyword Spotting on Low-Power IoT Devices

Keyword spotting (KWS) is a task to recognize a keyword or a particular command in a continuous audio stream, which can be effectively applied to a voice trigger system that automatically monitors and processes speech signals. This paper focuses on the problem of user-defined keyword spotting in low-resource settings. A lightweight neural network architecture is developed for tackling the keyword detection task using query-by-example (QbyE) techniques. The architecture uses a convolutional recurrent neural network (CRNN) to extract the frame-level features of input audio signals. A customized model compression method is proposed to compress the network, making it suitable for low power settings. In the keyword enrollment, all enrolled keyword examples are merged to generate a single keyword template, which is responsible for detecting a target keyword in keyword search. To improve the efficiency of keyword searching, a segmental local normalized DTW algorithm is introduced. Experiments on the real-world collected datasets show that our approach consistently outperforms the state-of-the-art methods, and the proposed system can run on an ARM Cortex-A7 processor and achieve real-time keyword detection.

DeFT-AN: Dense Frequency-Time Attentive Network for Multichannel Speech Enhancement

In this study, we propose a dense frequency-time attentive network (DeFT-AN) for multichannel speech enhancement. DeFT-AN is a mask estimation network that predicts a complex spectral masking pattern for suppressing the noise and reverberation embedded in the short-time Fourier transform (STFT) of an input signal. The proposed mask estimation network incorporates three different types of blocks for aggregating information in the spatial, spectral, and temporal dimensions. It utilizes a spectral transformer with a modified feed-forward network and a temporal conformer with sequential dilated convolutions. The use of dense blocks and transformers dedicated to the three different characteristics of audio signals enables more comprehensive enhancement in noisy and reverberant environments. The remarkable performance of DeFT-AN over state-of-the-art multichannel models is demonstrated based on two popular noisy and reverberant datasets in terms of various metrics for speech quality and intelligibility.