Spectrogram Data Research Articles

High performance method for COPD features extraction using complex network.

Objectives. The paper proposes a novel methodology for the classification of Chronic Obstructive Pulmonary Disease (COPD) utilizing respiratory sound attributes.Methods. The approach involves segmenting respiratory sounds into individual breaths and conducting extensive studies on this dataset. Spectral Transforms, various Wavelet Transforms are applied to capture distinct signal features. Complex Network is also employed to extract characteristic elements, generating novel representations of spectrogram data based on graph factors, including entropy, density, and position. The normalized and enriched data is then used to develop COPD classifiers using six machine learning algorithms, fine-tuning with appropriate training details and hyperparameter tuning.Results. Our results demonstrate robust performance, with ROC curves consistently exhibiting an Area Under the Curve (AUC) > 96% across different time-frequency transformations. Notably, the Random Forest algorithm achieves an AUC of 99.67%, outperforming other algorithms. Moreover, the Wavelet Daubechies 2 (Db2) consistently approaches 98% accuracy, particularly noteworthy in conjunction with the Naive Bayes algorithm.Conclusion. This study diagnosis patients through spectrogram images extracted from lung sounds. The application of Inverse Transforms, Complex Network, and Optimized Classification Algorithms yielded results beyond expectations. This methodology provides a promising approach for accurate COPD diagnosis, leveraging Machine Learning techniques applied to respiratory sound analysis.

FPGA‐Based Implementation of Real‐Time Cardiologist‐Level Arrhythmia Detection and Classification in Electrocardiograms Using Novel Deep Learning

ABSTRACTCardiac arrhythmia refers to irregular heartbeats caused by anomalies in electrical transmission in the heart muscle, and it is an important threat to cardiovascular health. Conventional monitoring and diagnosis still depend on the laborious visual examination of electrocardiogram (ECG) devices, even though ECG signals are dynamic and complex. This paper discusses the need for an automated system to assist clinicians in efficiently recognizing arrhythmias. The existing machine‐learning (ML) algorithms have extensive training cycles and require manual feature selection; to eliminate this, we present a novel deep learning (DL) architecture. Our research introduces a novel approach to ECG classification by combining the vision transformer (ViT) and the capsule network (CapsNet) into a hybrid model named ViT‐Cap. We conduct necessary preprocessing operations, including noise removal and signal‐to‐image conversion using short‐time Fourier transform (SIFT) and continuous wavelet transform (CWT) algorithms, on both normal and abnormal ECG data obtained from the MIT‐BIH database. The proposed model intelligently focuses on crucial features by leveraging global and local attention to explore spectrogram and scalogram image data. Initially, the model divides the images into smaller patches and linearly embeds each patch. Features are then extracted using a transformer encoder, followed by classification using the capsule module with feature vectors from the ViT module. Comparisons with existing conventional models show that our proposed model outperforms the original ViT and CapsNet in terms of classification accuracy for both binary and multi‐class ECG classification. The experimental findings demonstrate an accuracy of 99% on both scalogram and spectrogram images. Comparative analysis with state‐of‐the‐art methodologies confirms the superiority of our framework. Additionally, we configure a field‐programmable gate array (FPGA) to implement the proposed model for real‐time arrhythmia classification, aiming to enhance user‐friendliness and speed. Despite numerous suggestions for high‐performance FPGA accelerators in the literature, our FPGA‐based accelerator utilizes optimization of loop parallelization, FP data, and multiply accumulation (MAC) unit. Our accelerator architecture achieves a 57% reduction in processing time and utilizes fewer resources compared to a floating‐point (FlP) design.

Enhanced multi-ethnic speech recognition using pitch shifting generative adversarial networks

<p><span lang="EN-US">Research in the field of speech recognition is a challenging research area. Various approaches have been applied to build robust models. A problem faced in speech recognition research is overfitting, especially if there is insufficient data to train the model. A large enough amount of data can train the model well, resulting in high accuracy. Data augmentation is an approach often used to increase the quantity of dataset. This research uses a data augmentation approach, namely pitch shifting, to increase the quantity of speech dataset, which is then processed into spectrogram data and then classified using a generative adversarial network (GAN). Using the pitch shifting-generative adversarial network (PS-GAN) model, this research produces high accuracy performance in multi-ethnic speech recognition, namely 98.43%, better than several similar studies.</span></p>

Revolutionizing health monitoring: Integrating transformer models with multi-head attention for precise human activity recognition using wearable devices.

A daily activity routine is vital for overall health and well-being, supporting physical and mental fitness. Consistent physical activity is linked to a multitude of benefits for the body, mind, and emotions, playing a key role in raising a healthy lifestyle. The use of wearable devices has become essential in the realm of health and fitness, facilitating the monitoring of daily activities. While convolutional neural networks (CNN) have proven effective, challenges remain in quickly adapting to a variety of activities. This study aimed to develop a model for precise recognition of human activities to revolutionize health monitoring by integrating transformer models with multi-head attention for precise human activity recognition using wearable devices. The Human Activity Recognition (HAR) algorithm uses deep learning to classify human activities using spectrogram data. It uses a pretrained convolution neural network (CNN) with a MobileNetV2 model to extract features, a dense residual transformer network (DRTN), and a multi-head multi-level attention architecture (MH-MLA) to capture time-related patterns. The model then blends information from both layers through an adaptive attention mechanism and uses a SoftMax function to provide classification probabilities for various human activities. The integrated approach, combining pretrained CNN with transformer models to create a thorough and effective system for recognizing human activities from spectrogram data, outperformed these methods in various datasets - HARTH, KU-HAR, and HuGaDB produced accuracies of 92.81%, 97.98%, and 95.32%, respectively. This suggests that the integration of diverse methodologies yields good results in capturing nuanced human activities across different activities. The comparison analysis showed that the integrated system consistently performs better for dynamic human activity recognition datasets. In conclusion, maintaining a routine of daily activities is crucial for overall health and well-being. Regular physical activity contributes substantially to a healthy lifestyle, benefiting both the body and the mind. The integration of wearable devices has simplified the monitoring of daily routines. This research introduces an innovative approach to human activity recognition, combining the CNN model with a dense residual transformer network (DRTN) with multi-head multi-level attention (MH-MLA) within the transformer architecture to enhance its capability.

Machine learning classification of permeable conducting spheres in air and seawater using electromagnetic pulses

Abstract This paper presents machine learning classification on simulated data of permeable conducting spheres in air and seawater irradiated by low frequency electromagnetic pulses. Classification accuracy greater than 90% was achieved. The simulated data were generated using an analytical model of a magnetic dipole in air and seawater placed 1.5–3.5 m above the center of the sphere in 50 cm increments. The spheres had radii of 40 cm and 50 cm and were of permeable materials, such as steel, and non-permeable materials, such as aluminum. A series RL circuit was analytically modeled as the transmitter coil, and an RLC circuit as the receiver coil. Additive white Gaussian noise was added to the simulated data to test the robustness of the machine learning algorithms to noise. Multiple machine learning algorithms were used for classification including a perceptron and multiclass logistic regression, which are linear models, and a neural network, 1D convolutional neural network (CNN), and 2D CNN, which are nonlinear models. Feature maps are plotted for the CNNs and provide explainability of the salient parts of the time signature and spectrogram data used for classification. The pulses investigated, which expand the literature, include a two-sided decaying exponential, Heaviside step-off, triangular, Gaussian, rectangular, modulated Gaussian, raised cosine, and rectangular down-chirp. Propagation effects, including dispersion and frequency dependent attenuation, are encapsulated by the analytical model, which was verified using finite element modeling. The results in this paper show that machine learning methods are a viable alternative to inversion of electromagnetic induction (EMI) data for metallic sphere classification, with the advantage of real-time classification without the use of a physics-based model. The nonlinear machine learning algorithms used in this work were able to accurately classify metallic spheres in seawater even with significant pulse distortion caused by dispersion and frequency dependent attenuation. This paper presents the first effort towards the use of machine learning to classify metallic objects in seawater based on EMI sensing.

Detection of moisture of flowing grain with a novel deep learning structure using 2D spectrogram data

In order to preserve the stored grain for a long time without deterioration, the moisture content must be accurately known. In this study, the moisture content of flowing grain was determined using radar spectrogram data and CNN structure. A free-space measurement technique-based experimental environment was established for the purpose of collecting the necessary signals to measure the moisture content. The measurement plant is composed of two horn antennas, a vector network analyzer (VNA), and a flowing grain mechanism. In the experimental environment, the S11 and S21 parameter values are recorded from the VNA. The short-time Fourier transforms (STFT) of the recorded signals were then taken, and 2D spectrogram images were created. The dataset, comprising 22,780 images, was split into 80 % for training and 20 % for testing; then, they were subjected to regression analysis using CNNs. First, the mean absolute error (MAE) values of the 27 pre-trained CNN architectures in a single epoch are calculated. Subsequently, the architecture with the best four regression results is automatically selected. The training and testing steps are performed independently for each selected architecture, and the results are recorded. The MAE, root mean square error (RMSE), mean square error (MSE) and mean absolute percentage error (MAPE) metrics are employed to assess the efficacy of the CNN architectures. Among these, ResNet50 is the architecture that yields the most favorable results. Subsequently, a subsequent architecture with fewer parameters and a more expeditious processing time is proposed. The novel deep-learning CNN architecture demonstrated superior performance compared to the pre-trained architectures. The results are as follows: MAE = 0.0411, MSE = 0.0149, RMSE = 0.122, and MAPE = 0.0397. When comparing the time spent on training and testing, the least time-consuming architecture required approximately 72 min, whereas this study was completed in approximately 325 s. The pronounced disparity is readily apparent. The results demonstrate that when the CNN is appropriately modeled and trained, the combination of CNN and appropriate signal processing can effectively determine the moisture content of grains.

Improved mini-batch multiple augmentation for low-resource spoken word recognition

Data augmentation techniques have been useful in dealing with limited data for machine learning tasks. Recently, spectrogram data augmentation techniques have been investigated for voice conversion and sound classification tasks and have produced better results. However, applying multiple data augmentation techniques within a mini-batch has been observed to lead to performance degradation. While applying multiple augmentation methods sequentially has shown performance gains in image data, transferring this approach to spectrogram data leads to loss of acoustic information. Hence, an alternative approach is needed to effectively utilize multiple augmentation methods in the speech domain. This study addressed these challenges in low-resource settings for spoken word recognition within the mini-batch. First, we investigated the effect of data augmentation techniques. Second, we investigated the effect of multiple data augmentation techniques. Finally, we proposed a new approach that uses an alternate mechanism to utilize multiple spectrogram augmentation techniques more effectively. The results of our experiment show that the proposed approach (new pattern) outperforms the sequential approach (traditional pattern) significantly at different scales of datasets, including low-resource settings. In addition, the proposed approach achieves approximately 2x actual speedup over the sequential approach. A combination of frequency-warping and time length control augmentation methods was found to be stable and robust in performance across all datasets evaluated.

Results of the 2023 ISBI challenge to reduce GABA-edited MRS acquisition time.

Use a conference challenge format to compare machine learning-based gamma-aminobutyric acid (GABA)-edited magnetic resonance spectroscopy (MRS) reconstruction models using one-quarter of the transients typically acquired during a complete scan. There were three tracks: Track 1: simulated data, Track 2: identical acquisition parameters with in vivo data, and Track 3: different acquisition parameters with in vivo data. The mean squared error, signal-to-noise ratio, linewidth, and a proposed shape score metric were used to quantify model performance. Challenge organizers provided open access to a baseline model, simulated noise-free data, guides for adding synthetic noise, and in vivo data. Three submissions were compared. A covariance matrix convolutional neural network model was most successful for Track 1. A vision transformer model operating on a spectrogram data representation was most successful for Tracks 2 and 3. Deep learning (DL) reconstructions with 80 transients achieved equivalent or better SNR, linewidth and fit error compared to conventional 320 transient reconstructions. However, some DL models optimized linewidth and SNR without actually improving overall spectral quality, indicating a need for more robust metrics. DL-based reconstruction pipelines have the promise to reduce the number of transients required for GABA-edited MRS.

Machine learning algorithms for detection of visuomotor neural control differences in individuals with PASC and ME.

The COVID-19 pandemic has affected millions worldwide, giving rise to long-term symptoms known as post-acute sequelae of SARS-CoV-2 (PASC) infection, colloquially referred to as long COVID. With an increasing number of people experiencing these symptoms, early intervention is crucial. In this study, we introduce a novel method to detect the likelihood of PASC or Myalgic Encephalomyelitis (ME) using a wearable four-channel headband that collects Electroencephalogram (EEG) data. The raw EEG signals are processed using Continuous Wavelet Transform (CWT) to form a spectrogram-like matrix, which serves as input for various machine learning and deep learning models. We employ models such as CONVLSTM (Convolutional Long Short-Term Memory), CNN-LSTM, and Bi-LSTM (Bidirectional Long short-term memory). Additionally, we test the dataset on traditional machine learning models for comparative analysis. Our results show that the best-performing model, CNN-LSTM, achieved an accuracy of 83%. In addition to the original spectrogram data, we generated synthetic spectrograms using Wasserstein Generative Adversarial Networks (WGANs) to augment our dataset. These synthetic spectrograms contributed to the training phase, addressing challenges such as limited data volume and patient privacy. Impressively, the model trained on synthetic data achieved an average accuracy of 93%, significantly outperforming the original model. These results demonstrate the feasibility and effectiveness of our proposed method in detecting the effects of PASC and ME, paving the way for early identification and management of the condition. The proposed approach holds significant potential for various practical applications, particularly in the clinical domain. It can be utilized for evaluating the current condition of individuals with PASC or ME, and monitoring the recovery process of those with PASC, or the efficacy of any interventions in the PASC and ME populations. By implementing this technique, healthcare professionals can facilitate more effective management of chronic PASC or ME effects, ensuring timely intervention and improving the quality of life for those experiencing these conditions.

A novel vehicle collision detection system: Integrating audio-visual fusion for enhanced performance

In vehicular accidents, the swift and accurate identification of car crashes is paramount, as it serves as the linchpin for prompt responses from emergency services and search-and-rescue operations. This study introduces an innovative multimodal car crash detection system that capitalizes on audio-visual data sourced from dashboard cameras, thus significantly enhancing the precision of automobile collision detection. In contrast to car crash detection systems relying on single-modal inputs, this research employs advanced multimodal methodologies with a neural network architecture that integrates Convolutional Neural Networks (CNN) and Recurrent Neural Networks (RNN) for car crash detection. This system concurrently leverages audio spectrogram and audio feature data, video RGB, video optical flow, and video differential information, enabling collision recognition through a neural network system, resulting in markedly improved collision detection accuracy. A comprehensive comparative analysis is conducted, benchmarking our system against the most cutting-edge existing car crash detection systems. The system employs a unique fusion approach, allowing it to apply its inference to audio-visual information of varying durations compared to the training set. The empirical findings unequivocally affirm our proposed solution’s substantial enhancement in car crash detection.

Utilizing Convolutional Autoencoder for Anomaly Detections in LIGO Spectrogram Data

Massive amounts of data generated from the continuous running of LIGO gravitational-wave detectors comes with a need to search for signals within the data. Gravitational-wave data captured by the detector consist of astronomical events or glitches that last seconds. We present an unsupervised learning using convolutional autoencoder trained on the no-glitch Gravity Spy dataset to do anomaly search on spectrogram data. Reconstruction error is used as the basis and multiple windows are used to improve the model. Results on test data show that the model is capable of detecting signals with significant anomalies such as the Chirp or Koi Fish glitch. Meanwhile, detecting subtle anomalies such as the 1400 Hz Ripples is difficult because its reconstruction error is near the range of noise signals. Validating the result on confirmed gravitational-wave signals shows that the model is capable of gravitational-wave detection.

Feature detection in guided wave ultrasound measurements using simulated spectrograms and generative machine learning

Guided wave ultrasound field measurements capture reflections from aluminothermic welds in the rail track at unknown distances from a transducer. The measured guided wave signals are complex and challenging to interpret due to multiple dispersive modes propagating in the rail, changing environmental conditions, and noise. Data-driven machine learning techniques have been applied to complex signal-processing problems and have shown significant potential in learning and resolving complex problems. This study aims to understand the implications of using various simulated data sets for application within an appropriate learning framework to capture the underlying features of the field measurements and maximise the performance of these techniques. The use of simulated spectrograms, including variations of signal attributes (attenuation, positions of welds, noise, and mode reflection coefficients) generally observed in experimental measurements, allows the reflections of individual modes to be isolated or combined for training. This allows us to present the training data in three distinct forms. The first dataset has the highest mode reflection information density per sample and consists of simulated spectrogram data with multiple reflections of modes from multiple welds, like experimentally obtained spectrograms. The second training dataset consists of spectrograms with multiple mode reflections; however, only for a single weld reflection per spectrogram. The third training set contains a reflection of a single mode from a single weld in each spectrogram. The data-driven models applied are principal component autoregression and variational auto-encoders. The reconstruction error and latent space interpretability were considered as metrics for the algorithms’ ability to learn using test sets, i.e., unseen data. The results show that datasets with sufficient feature variation and higher mode reflection information density better construct the test set of simulated and experimental spectrograms. However, training using the third dataset shows more interpretable latent variables for an artificial growing defect attached to the rail. Furthermore, data-driven machine learning methods trained using simulated spectrogram data are useful for reconstructing and learning features from experimental measurements, provided that the training data have representative mode feature variation and noise.

A deep neural network based reverse radio spectrogram search algorithm

Abstract Modern radio astronomy instruments generate vast amounts of data, and the increasingly challenging radio frequency interference (RFI) environment necessitates ever-more sophisticated RFI rejection algorithms. The ‘needle in a haystack’ nature of searches for transients and technosignatures requires us to develop methods that can determine whether a signal of interest has unique properties, or is a part of some larger set of pernicious RFI. In the past, this vetting has required onerous manual inspection of very large numbers of signals. In this paper, we present a fast and modular deep learning algorithm to search for lookalike signals of interest in radio spectrogram data. First, we trained a β-variational autoencoder on signals returned by an energy detection algorithm. We then adapted a positional embedding layer from classical transformer architecture to a embed additional metadata, which we demonstrate using a frequency-based embedding. Next we used the encoder component of the β-variational autoencoder to extract features from small (∼715 Hz, with a resolution of 2.79 Hz per frequency bin) windows in the radio spectrogram. We used our algorithm to conduct a search for a given query (encoded signal of interest) on a set of signals (encoded features of searched items) to produce the top candidates with similar features. We successfully demonstrate that the algorithm retrieves signals with similar appearance, given only the original radio spectrogram data. This algorithm can be used to improve the efficiency of vetting signals of interest in technosignature searches, but could also be applied to a wider variety of searches for ‘lookalike’ signals in large astronomical data sets.

Back-Guard: Wireless Backscattering Based User Sensing With Parallel Attention Model

With the rapid advance of wireless sensing techniques, it becomes possible to provide a fine-grained user activity tracking service at home and office. Such a technique is of broad applications in various domains such as personal activity diary, elderly care, and customized services. For example, several radio frequency (RF) based sensing systems were recently proposed for human activity recognition. However, most of them focused on specific scenarios and suffered from interference caused by other users and wireless devices. In this work, we propose Back-Guard, a backscattering-based sensing system that achieves accurate and non-intrusive user activity recognition and further user identification/authentication. Back-Guard carefully examines the backscatter spectrogram data and extracts high-level features from both spatial and temporal domains. Leveraging the parallel attention based deep learning model, our system can discriminate different motions and users accurately and robustly in various situations. We implemented a prototype system and collected data from 25 users for more than 2 months. Extensive experiments demonstrate that Back-Guard achieves 93.4 <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\%$</tex-math></inline-formula> activity recognition accuracy and 91.5 <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\%$</tex-math></inline-formula> user identification accuracy, respectively. In particular, Back-Guard can also tackle multiple user scenarios, which has little accuracy reduction when the users are separated, e.g., by around 2 meters.

Analyzing drilling noise in rotational atherectomy: Improving safety and effectiveness through visualization and anomaly detection using autoencoder-A preclinical study.

As the population of aging societies continues to grow, the prevalence of complex coronary artery diseases, including calcification, is expected to increase. Rotational atherectomy (RA) is an essential technique for treating calcified lesions. This study aimed to assess the usefulness of the drilling noise produced during rotablation as a parameter for evaluating the safety and effectiveness of the procedure. A human body model mimicking calcified stenotic coronary lesions was constructed using plastic resin, and burrs of sizes 1.25 and 1.5 mm were utilized. To identify the noise source during rotablation, we activated the ROTAPRO™ rotablator at a rotational speed of 180,000 rpm, recording the noise near the burr (inside the mock model) and advancer (outside). In addition to regular operation, we simulated two major complications: burr entrapment and guidewire transection. The drilling noise recorded in Waveform Audio File Format files was converted into spectrograms for analysis and an autoencoder analyzed the image data for anomalies. The drilling noise from both inside and outside the mock model was predominantly within the 3000 Hz frequency domain. During standard operation, intermittent noise within this range was observed. However, during simulated complications, there were noticeable changes: a drop to 2000 Hz during burr entrapment and a distinct squealing noise during guidewire transection. The autoencoder effectively reduced the spectrogram data into a two-dimensional representation suitable for anomaly detection in potential clinical applications. By analyzing drilling noise, the evaluation of procedural safety and efficacy during RA can be enhanced.

Label-Free identification of fentanyl analogues using surface-enhanced Raman scattering and machine learning algorithm

The identification and analysis of drug abuse is crucial for controlling and combating this global public health problem. In this regard, surface-enhanced Raman spectroscopy (SERS) is a highly sensitive detection technique that is fast, non-destructive, and widely used in various fields. In this study, Ag@Au nanowires (Ag@Au NWs) were synthesized using the seed growth method. The SERS performance of Ag@Au NWs was evaluated using R6G as a probe molecule, and the minimum detection concentration of R6G was found to be 10−9 M. Similarly, the minimum detection concentration of fentanyl was observed to be 10−7 M. Furthermore, the study also involved performing routine Raman detection on 10 fentanyl analogs that were structurally similar. Additionally, density functional theory (DFT) calculations were carried out to distinguish between these substances. The Raman spectra calculated by DFT provided vibrational frequencies and normal mode assignments for each species. By comparing normal Raman spectra, SERS and DFT calculated spectrogram data, we assigned characteristic peaks of 10 substances. Finally, principal component analysis was used to reduce the dimension of the data, and support vector machine were further used to classification of SERS spectral data of the above 10 substances, with an accuracy of 96.4%.

Measuring the properties of f−mode oscillations of a protoneutron star by third-generation gravitational-wave detectors

Core-collapse supernovae are among the astrophysical sources of gravitational waves that could be detected by third-generation gravitational-wave detectors. Here, we analyze the gravitational-wave strain signals from two- and three-dimensional simulations of core-collapse supernovae generated using the code fornax. A subset of the two-dimensional simulations has nonzero core rotation at the core bounce. A dominant source of time changing quadrupole moment is the $l=2$ fundamental mode ($f\text{\ensuremath{-}}\mathrm{mode}$) oscillation of the protoneutron star. From the time-frequency spectrogram of the gravitational-wave strain we see that, starting $\ensuremath{\sim}400\text{ }\text{ }\mathrm{ms}$ after the core bounce, most of the power lies within a narrow track that represents the frequency evolution of the $f\text{\ensuremath{-}}\mathrm{mode}$ oscillations. The $f\text{\ensuremath{-}}\mathrm{mode}$ frequencies obtained from linear perturbation analysis of the angle-averaged profile of the protoneutron star corroborate what we observe in the spectrograms of the gravitational-wave signal. We explore the measurability of the $f\text{\ensuremath{-}}\mathrm{mode}$ frequency evolution of a protoneutron star for a supernova signal observed in the third-generation gravitational-wave detectors. Measurement of the frequency evolution can reveal information about the masses, radii, and densities of the protoneutron stars. We find that if the third-generation detectors observe a supernova within 10 kpc, then we can measure these frequencies to within 5 Hz rms error. We can also measure the energy emitted in the fundamental $f\text{\ensuremath{-}}\mathrm{mode}$ using the spectrogram data of the strain signal. We find that the energy in the $f\text{\ensuremath{-}}\mathrm{mode}$ can be measured to within 20% error for signals observed by Cosmic Explorer using simulations with successful explosion, assuming source distances within 10 kpc.

Advanced thermal fluid leakage detection system with machine learning algorithm for pipe-in-pipe structure

Pipe-in-pipe (PIP) system is essential for high thermal and high pressure fluid transportation. However, in the existing PIP systems, fluid leakage between inner and outer pipe has been difficult to discover or detect, which has worked as bottle neck to utilize PIP system in high risk industries as nuclear reactor, chemical plant or oil drilling systems. Here, we propose a noble PIP leakage detection system utilizing distributed temperature sensing (DTS) with Machine Learning (ML). With the Fourier transformed spectrogram data from DTS, the ML assisted system was able to detect 0.2∼7 ml/min liquid leakage between inner and outer pipe with the accuracy of 91.67% with a single embedded optical fiber. Under varying operating temperature, the system successfully distinguished leakage and non-leakage states using the optimized convolutional neural network. Our developed PIP leakage detection system can be deployed in safety-critical industrial systems for autonomous leakage detection.

차량 내부 소음 예측을 위한 멀티 모달 자기 지도 학습 네트워크

Predicting in-vehicle noise levels is an important issue in automobile industry. In most previous studies, various supervised learning methods that use both the input and output(labeled) variables are used to predict automobile noise levels. However, collecting labeled data for in-vehicle noise prediction is time consuming and expensive. In this study, we propose a multimodal self-supervised learning framework that can predict in-vehicle noise levels with only a small amount of labeled data, so that resources required to collect labeled data can be saved. In our framework, both original acceleration signals and spectrograms converted from the original data are used as the input to predict in-vehicle noise levels. In the first stage, we pretrain the features of the input data based on the relationship between the signal and spectrogram data using only unlabeled data, which is much easier to collect than labeled data. In the second stage, we use a small amount of data to construct the in-vehicle noise prediction model with the pretrained feature extractor. The effectiveness and applicability of the proposed framework are demonstrated using the actual acceleration signal data collected from various locations of electric power steering vehicle noise levels. The proposed framework outperforms the existing supervised learning method especially when a few labeled data are available.

Modelling of Arabic Plosive and Fricative Acoustic Characteristics Articulated by Malay Native Speakers

This study aims to examine the pharyngealized and non-pharyngealized Arabic sounds produced by Malay speakers based on acoustic phonetic approach and to investigate its relation with the second language learning model proposed by Flege (1995). The plosive and fricative associated with the pharyngealized and non-pharyngealized of Arabic sounds is one of the main problems among non-Arabic native speakers in learning Arabic language. Recent studies emphasize the influence of the first language / mother tongue language as the cause of the failure to master the Arabic language as the second language. Hence the frameworks of the similar, different and new sound hypotheses proposed by Flege (1995) were used to get the real picture of the Arabic language speech phenomenon in the second language condition. PRAAT software was used to obtain speech data in the spectrogram and to undergo spectrograph analysis. Subsequently, the findings were analyzed using SPSS to highlight the overall results of the study more thoroughly. VOT (Voice Onset Time) acoustic parameters for the plosive sound and frication of the frication sounds were used during the spectrograph analysis. A total of 2880 spectrogram data were obtained from the subjects: 24 undergraduates from the Bachelor of Islamic Studies with Honors (Arabic Studies and Islamic Civilization) UKM. The results of the production experiment show that Malay plosive sound has negative VOT for voiced stop and short positive VOT for voiceless stop, while Arabic plosive sounds has a model pattern voicing lead versus long lag for voiced and voiceless stops respectively. The results show that Arabic voiceless stop / ت / is aspirated and found to has a longer VOT than Malay voiceless stop /t/. For Arabic pharyngealized sounds, the results show higher values of F1 than non-pharyngealized sounds. In addition, Arabic pharyngealized sounds are found to have shorter VOT than non-pharyngealized Arabic sounds. The findings showed that there were cases where subjects managed to replicate the L2 sound to similar sound of L1 and there were cases of L2 sounds that are foreign to the sound system of L1. With that in mind, it can be emphasized that some studies have supported Flege’s theory that the similar sound between L1 and L2 are not necessarily easy to pronounce, while there are L2 phonemes that are absent in L1, but they are easily learnt by the L2 speaker.