Harmonic-Percussive Source Separation Research Articles

Spectral Features Analysis for Print Quality Prediction in Additive Manufacturing: An Acoustics-Based Approach.

Quality prediction in additive manufacturing (AM) processes is crucial, particularly in high-risk manufacturing sectors like aerospace, biomedicals, and automotive. Acoustic sensors have emerged as valuable tools for detecting variations in print patterns by analyzing signatures and extracting distinctive features. This study focuses on the collection, preprocessing, and analysis of acoustic data streams from a Fused Deposition Modeling (FDM) 3D-printed sample cube (10 mm × 10 mm × 5 mm). Time and frequency-domain features were extracted at 10-s intervals at varying layer thicknesses. The audio samples were preprocessed using the Harmonic-Percussive Source Separation (HPSS) method, and the analysis of time and frequency features was performed using the Librosa module. Feature importance analysis was conducted, and machine learning (ML) prediction was implemented using eight different classifier algorithms (K-Nearest Neighbors (KNN), Support Vector Machine (SVM), Gaussian Naive Bayes (GNB), Decision Trees (DT), Logistic Regression (LR), Random Forest (RF), Extreme Gradient Boosting (XGB), and Light Gradient Boosting Machine (LightGBM)) for the classification of print quality based on the labeled datasets. Three-dimensional-printed samples with varying layer thicknesses, representing two print quality levels, were used to generate audio samples. The extracted spectral features from these audio samples served as input variables for the supervised ML algorithms to predict print quality. The investigation revealed that the mean of the spectral flatness, spectral centroid, power spectral density, and RMS energy were the most critical acoustic features. Prediction metrics, including accuracy scores, F-1 scores, recall, precision, and ROC/AUC, were utilized to evaluate the models. The extreme gradient boosting algorithm stood out as the top model, attaining a prediction accuracy of 91.3%, precision of 88.8%, recall of 92.9%, F-1 score of 90.8%, and AUC of 96.3%. This research lays the foundation for acoustic based quality prediction and control of 3D printed parts using Fused Deposition Modeling and can be extended to other additive manufacturing techniques.

Environmental Sound Classification Framework Based on L-mHP Features and SE-ResNet50 Network Model

Environmental sound classification (ESC) tasks are attracting more and more attention. Due to the complexity of the scene and personnel mobility, there are some difficulties in understanding and generating environmental sound models for ESC tasks. To address these key issues, this paper proposes an audio classification framework based on L-mHP features and the SE-ResNet50 model and improves a dual-channel data enhancement scheme based on a symmetric structure for model training. Firstly, this paper proposes the L-mHP feature to characterize environmental sound. The L-mHP feature is a three-channel feature consisting of a Log-Mel spectrogram, a harmonic spectrogram, and a percussive spectrogram. The harmonic spectrogram and percussive spectrogram can be obtained by harmonic percussive source separation (HPSS) of a Log-Mel spectrogram. Then, an improved audio classification model SE-ResNet50 is proposed based on the ResNet-50 model. In this paper, a dual-channel data enhancement scheme based on a symmetric structure is promoted, which not only makes the audio variants more diversified, but also makes the model focus on learning the time–frequency mode in the acoustic features during the training process, so as to improve the generalization performance of the model. Finally, the audio classification experiment of the framework is carried out on public datasets. An experimental accuracy of 94.92%, 99.67%, and 90.75% was obtained on ESC-50, ESC-10 and UrbanSound8K datasest, respectively. In order to simulate the classification performance of the framework in the actual environment, the framework was also evaluated on a self-made sound dataset with different signal-to-noise ratios. The experimental results show that the proposed audio classification framework has good robustness and feasibility.

Automatic wheeze segmentation using harmonic-percussive source separation and empirical mode decomposition.

Wheezes are adventitious respiratory sounds commonly present in patients with respiratory conditions. The presence of wheezes and their time location are relevant for clinical reasons, such as understanding the degree of bronchial obstruction. Conventional auscultation is usually employed to analyze wheezes, but remote monitoring has become a pressing need during recent years. Automatic respiratory sound analysis is required to reliably perform remote auscultation. In this work we propose a method for wheeze segmentation. Our method starts by decomposing a given audio excerpt into intrinsic mode frequencies using empirical mode decomposition. Then, we apply harmonic-percussive source separation to the resulting audio tracks and get harmonic-enhanced spectrograms, which are processed to obtain harmonic masks. Subsequently, a series of empirically derived rules are applied to find wheeze candidates. Finally, the candidates stemming from the different audio tracks are merged and median filtered. In the evaluation stage, we compare our method to three baselines on the ICBHI 2017 Respiratory Sound Database, a challenging dataset containing various noise sources and background sounds. Using the full dataset, our method outperforms the baselines, achieving an F1 of 41.9%. Our method's performance is also better than the baselines across several stratified results focusing on five variables: recording equipment, age, sex, body-mass index, and diagnosis. We conclude that, contrary to what has been reported in the literature, wheeze segmentation has not been solved for real life scenario applications. Adaptation of existing systems to demographic characteristics might be a promising step in the direction of algorithm personalization, which would make automatic wheeze segmentation methods clinically viable.

Fundamental Frequencies Contour Extraction Based on the Extended Harmonic-Percussive Source Separation

Introduction. Amidst the multiplicity of audio signal processing tasks connected with the necessity of source separation, the problem of obtaining of its most prominent components occupies special place. One of the approaches, dedicated to solve such task, is based on melody extraction from musical audio recording. If generalized, such technique can be applied to a wider class of audio signals for extraction from them of the so-called fundamental frequencies contour. For its realization, an attempt was made to combine the method of median filtering with the salience estimation method, for their application at various stages of the analysis of the input audio signal. A combination of methods is used to obtain the f_0-representation of the melody, based on the processing of the filtered values obtained at the first step. The purpose of the article is to obtain the trajectory of f_0-values of the input audio signal and filtering of the corresponding to this trajectory harmonics. Results. The proposed method is effective for use in audio signal processing systems for fundamental frequencies contour of the most prominent tonal components extraction and its further reuse. Conclusions. Realized spectrum decomposition technique, based on the tendency of the time-frequency distribution of its constituent sounds, allows to effectively extract melodic contours from non-melodic contours. Nevertheless, there is a necessity for further research regarding the distribution model of harmonic and percussive characteristics relative to each other. Such a model should be extended with heuristics for more accurate filtering of stable in time tonal components of complex audio signals.

In-situ acoustic monitoring of direct energy deposition process with deep learning-assisted signal denoising

In-situ monitoring is crucial for detecting process anomalies and ensuring part quality in additive manufacturing. Acoustic-based monitoring techniques offer extra benefits such as adjustable sensor setup and lower hardware costs. In the direct energy deposition (DED) process, acoustic signals generated by laser-material interactions carry information about underlying complex physical mechanisms such as melting, solidification, crack propagation, and pore formation. This paper presents a novel acoustic-based in-situ monitoring method for the DED process. The raw acoustic signal is made up of laser-material interaction sound as well as noise from machine movement, inert gas flow, and powder flow. A deep learning model is developed to build an end-to-end signal denoising framework to minimize environmental noise and extract the laser-material interaction sound. Audio equalization, bandpass filtering, and Harmonic-Percussive Source Separation algorithm are used to produce a cleaned laser-material interaction sound as the model's ground truth target. Acoustic data is collected from experiments using different DED machines, materials, and varied process parameters to train the deep learning model. The proposed deep learning-assisted signal denoising strategy lays the groundwork for acoustic-based in-situ defect detection of the DED process.

CNN-Based Acoustic Scene Classification System

Acoustic scene classification (ASC) categorizes an audio file based on the environment in which it has been recorded. This has long been studied in the detection and classification of acoustic scenes and events (DCASE). This presents the solution to Task 1 of the DCASE 2020 challenge submitted by the Chung-Ang University team. Task 1 addressed two challenges that ASC faces in real-world applications. One is that the audio recorded using different recording devices should be classified in general, and the other is that the model used should have low-complexity. We proposed two models to overcome the aforementioned problems. First, a more general classification model was proposed by combining the harmonic-percussive source separation (HPSS) and deltas-deltadeltas features with four different models. Second, using the same feature, depthwise separable convolution was applied to the Convolutional layer to develop a low-complexity model. Moreover, using gradient-weight class activation mapping (Grad-CAM), we investigated what part of the feature our model sees and identifies. Our proposed system ranked 9th and 7th in the competition for these two subtasks, respectively.

Adversarial Unsupervised Domain Adaptation for Harmonic-Percussive Source Separation

This letter addresses the problem of domain adaptation for the task of music source separation. Using datasets from two different domains, we compare the performance of a deep learning-based harmonic-percussive source separation model under different training scenarios, including supervised joint training using data from both domains and pre-training in one domain with fine-tuning in another. We propose an adversarial unsupervised domain adaptation approach suitable for the case where no labelled data (ground-truth source signals) from a target domain is available. By leveraging unlabelled data (only mixtures) from this domain, experiments show that our framework can improve separation performance on the new domain without losing any considerable performance on the original domain. The letter also introduces the Tap & Fiddle dataset, a dataset containing recordings of Scandinavian fiddle tunes along with isolated tracks for “foot-tapping” and “violin”.

Development of an Acoustic System for UAV Detection.

The purpose of this paper is to investigate the possibility of developing and using an intelligent, flexible, and reliable acoustic system, designed to discover, locate, and transmit the position of unmanned aerial vehicles (UAVs). Such an application is very useful for monitoring sensitive areas and land territories subject to privacy. The software functional components of the proposed detection and location algorithm were developed employing acoustic signal analysis and concurrent neural networks (CoNNs). An analysis of the detection and tracking performance for remotely piloted aircraft systems (RPASs), measured with a dedicated spiral microphone array with MEMS microphones, was also performed. The detection and tracking algorithms were implemented based on spectrograms decomposition and adaptive filters. In this research, spectrograms with Cohen class decomposition, log-Mel spectrograms, harmonic-percussive source separation and raw audio waveforms of the audio sample, collected from the spiral microphone array—as an input to the Concurrent Neural Networks were used, in order to determine and classify the number of detected drones in the perimeter of interest.

Impulsive spike enhancement on gamelan audio using harmonic percussive separation

Impulsive spikes often occur in audio recording of gamelan where most existing methods reduce it. This research offers new method to enhance audio impulsive spike in gamelan music that is able to reduce, eliminate and even strengthen spikes. The process separates audio components into harmonics and percussive components. Percussion component is set to rise or lowered, and the results of the process combined with harmonic components again. This study proposes a new method that allows reducing, eliminating and even amplifying the spike. From the similarity test using the Cosine Distance method, it is seen that spike enhancement through Harmonic Percussive Source Separation (HPSS) has an average Cosine Distance value of 0.0004 or similar to its original, while Mean Square Error (MSE) has an average value of 0.0004 that is very small in average error and also very similar. From the Perceptual Evaluation of Audio Quality (PEAQ) testing with Harmonic Percussive Source Separation (HPSS), it has a better quality with an average Objective Difference Grade (ODG) of -0.24 or Imperceptible.

Towards Music Structural Segmentation across Genres

This article faces the problem of how different audio features and segmentation methods work with different music genres. A new annotated corpus of Chinese traditional Jingju music is presented. We incorporate this dataset with two existing music datasets from the literature in an integrated retrieval system to evaluate existing features, structural hypotheses, and segmentation algorithms outside a Western bias. A harmonic-percussive source separation technique is introduced to the feature extraction process and brings significant improvement to the segmentation. Results show that different features capture the structural patterns of different music genres in different ways. Novelty- or homogeneity-based segmentation algorithms and timbre features can surpass the investigated alternatives for the structure analysis of Jingju due to their lack of harmonic repetition patterns. Findings indicate that the design of audio features and segmentation algorithms as well as the consideration of contextual information related to the music corpora should be accounted dependently in an effective segmentation system.