Frequency Masking Research Articles

Echo-Influenced Source Localization Algorithm Using Frequency Masking and Phase Compensation

Echo-Influenced Source Localization Algorithm Using Frequency Masking and Phase Compensation

Experimental verification of the variability of amplitude modulation (AM) from wind turbines in relation to its assessment for annoyance

The analysis of the influence of amplitude modulation (AM) on the annoyance of noise generated by wind turbines (WT) is still the subject of numerous studies. However, there is still a lack of widely accepted quantitative methods that would determine the degree of influence of modulation phenomena on noise perception. The study analyzed the variability of the depth of AM over time within different frequency-ranges. The research was based on long-term measurements at selected measurement locations. Long-term measurement ensured consideration of diverse propagation conditions. The collected data were segmented into time intervals, which included basic AM parameters - depth, modulation frequency, and frequency band in which the AM was observed, as well as wind speed and direction. Statistical analysis of the gathered dataset was used to assess the regularity of distinguished AM parameters in correlation with WT noise energy values and recorded non-acoustic parameters. A significant variability of AM depth was observed depending on the frequency band, masking level, and position relative to the wind direction. By adopting a criterion for assessing the annoyance of WT noise (according to AMWG) with a threshold depth of AM not less than 2 dB, it was noticed in over 30% of the analyzed intervals.

Cascaded Cross-modal Alignment for Visible-Infrared Person Re-Identification

Visible-Infrared Person Re-Identification faces significant challenges due to cross-modal and intra-modal variations. Although existing methods explore semantic alignment from various angles, severe distribution shifts in heterogeneous data limit the effectiveness of single-level alignment approaches. To address this issue, we propose a Cascaded Cross-modal Alignment (CCA) framework that gradually eliminates distribution discrepancies and aligns semantic features from three complementary perspectives in a cascaded manner. First, at the input-level, we propose a Channel-Spatial Recombination (CSR) strategy that strategically reorganizes and preserves crucial details from channel and spatial dimensions to diminish visual discrepancies between modalities, thereby narrowing the modality gap in input images. Second, at the frequency-level, we introduce a Low Frequency Masking (LFM) module to emphasize global details that CSR might overlook by randomly masking low-frequency information, thus driving comprehensive alignment of identity semantics. Third, at the part-level, we design a Prototype-based Semantic Refinement (PSR) module to refine fine-grained features and mitigate the impact of irrelevant areas in LFM. It accurately aligns body parts and enhances semantic consistency guided by global discriminative clues from LFM and flipped views with pose variations. Comprehensive experimental results on the SYSU-MM01 and RegDB datasets demonstrate the superiority of our proposed CCA.

Adaptive Filtering for Multi-Track Audio Based on Time–Frequency Masking Detection

There is a growing need to facilitate the production of recorded music as independent musicians are now key in preserving the broader cultural roles of music. A critical component of the production of music is multitrack mixing, a time-consuming task aimed at, among other things, reducing spectral masking and enhancing clarity. Traditionally, this is achieved by skilled mixing engineers relying on their judgment. In this work, we present an adaptive filtering method based on a novel masking detection scheme capable of identifying masking contributions, including temporal interchangeability between the masker and maskee. This information is then systematically used to design and apply filters. We implement our methods on multitrack music to improve the quality of the raw mix.

Effects of ipsilateral, contralateral, and bilateral noise precursors on psychoacoustical tuning curves in humans

Cochlear tuning and hence auditory frequency selectivity are thought to change in noisy environments by activation of the medial olivocochlear reflex (MOCR). In humans, auditory frequency selectivity is often assessed using psychoacoustical tuning curves (PTCs), a plot of the level required for pure-tone maskers to just mask a fixed-level pure-tone probe as a function of masker frequency. Sometimes, however, the stimuli used to measure a PTC are long enough that they can activate the MOCR by themselves and thus affect the PTC. Here, PTCs for probe frequencies of 500 Hz and 4 kHz were measured in forward masking using short maskers (30 ms) and probes (10 ms) to minimize the activation of the MOCR by the maskers or the probes. PTCs were also measured in the presence of long (300 ms) ipsilateral, contralateral, and bilateral broadband noise precursors to investigate the effect of the ipsilateral, contralateral, and bilateral MOCR on PTC tuning. Four listeners with normal hearing participated in the experiments. At 500 Hz, ipsilateral and bilateral precursors sharpened the PTCs by decreasing the thresholds for maskers with frequencies at or near the probe frequency with minimal effects on thresholds for maskers remote in frequency from the probe. At 4 kHz, by contrast, ipsilateral and bilateral precursors barely affected thresholds for maskers near the probe frequency but broadened PTCs by reducing thresholds for maskers far from the probe. Contralateral precursors barely affected PTCs. An existing computational model was used to interpret the results. The model suggested that despite the apparent differences, the pattern of results is consistent with the ipsilateral and bilateral MOCR inhibiting the cochlear gain similarly at the two probe frequencies and more strongly than the contralateral MOCR.

A novel bearing health sensing technique using smartphone

Bearing health sensing has drawn increasing attention as bearing plays a vulnerable yet essential role in modern industry. Nevertheless, conventional health sensing technologies involve dedicated sensor installation and high costs, which may not meet the monitoring and sensing demands. Given this, we present a health sensing technique to capture the fault information of bearing by leveraging an off-the-shelf smartphone. Specifically, we first exploit only a readily available smartphone to capture the audio signal of bearing. Given the captured audio signal, an adaptive time–frequency masking is proposed to separate the noise, and the fault impulse is then sharpened by the singular value decomposition (SVD). The extensive experimental results illustrate that our technique is able to enhance the bearing health sensing capability of smartphones under various real-world scenarios. Even in noisy mining field conditions, the framework still achieves a notable performance improvement rate of 169 %.

Boosting Robust Multi-Focus Image Fusion With Frequency Mask and Hyperdimensional Computing

Boosting Robust Multi-Focus Image Fusion With Frequency Mask and Hyperdimensional Computing

MSLTE: multiple self-supervised learning tasks for enhancing EEG emotion recognition

Objective. The instability of the EEG acquisition devices may lead to information loss in the channels or frequency bands of the collected EEG. This phenomenon may be ignored in available models, which leads to the overfitting and low generalization of the model. Approach. Multiple self-supervised learning tasks are introduced in the proposed model to enhance the generalization of EEG emotion recognition and reduce the overfitting problem to some extent. Firstly, channel masking and frequency masking are introduced to simulate the information loss in certain channels and frequency bands resulting from the instability of EEG, and two self-supervised learning-based feature reconstruction tasks combining masked graph autoencoders (GAE) are constructed to enhance the generalization of the shared encoder. Secondly, to take full advantage of the complementary information contained in these two self-supervised learning tasks to ensure the reliability of feature reconstruction, a weight sharing (WS) mechanism is introduced between the two graph decoders. Thirdly, an adaptive weight multi-task loss (AWML) strategy based on homoscedastic uncertainty is adopted to combine the supervised learning loss and the two self-supervised learning losses to enhance the performance further. Main results. Experimental results on SEED, SEED-V, and DEAP datasets demonstrate that: (i) Generally, the proposed model achieves higher averaged emotion classification accuracy than various baselines included in both subject-dependent and subject-independent scenarios. (ii) Each key module contributes to the performance enhancement of the proposed model. (iii) It achieves higher training efficiency, and significantly lower model size and computational complexity than the state-of-the-art (SOTA) multi-task-based model. (iv) The performances of the proposed model are less influenced by the key parameters. Significance. The introduction of the self-supervised learning task helps to enhance the generalization of the EEG emotion recognition model and eliminate overfitting to some extent, which can be modified to be applied in other EEG-based classification tasks.

Measuring and modeling Gaussian noise disruption across frequency, level, and hearing status

Recent work suggested that older listeners with minimal hearing loss exhibit more forward masking for a highly fluctuating Gaussian noise (GN) masker relative to a low-fluctuation noise (LFN) masker. This threshold difference (GN–LFN) is referred to as GN disruption and was previously attributed to time-varying cochlear gain resulting from feedback control by the medial olivocochlear (MOC) system. However, thus far, masker conditions have been limited to 80 dB SPL at 4 kHz. The current study aims to challenge the hypothesis that the physiological basis of GN disruption is the MOC system using behavioral measures and computational models across multiple target and masker frequencies (0.5, 1, 2, and 4 kHz), masker levels (80-, 65-, and 50-dB SPL), masker-signal delays (25- and 75-ms), participant age ranges (18–30 and 60–75), and hearing statuses (normal hearing, sensorineural hearing loss). Additionally, speech recognition using IEEE sentences will be measured with GN and LFN maskers. Although testing will not be completed until 2025, preliminary data from 20 younger participants with normal hearing suggest that GN disruption is greater for higher than lower masker and probe frequencies, that GN disruption is similar across masker levels, and that refinements to the computational model are needed to more accurately predict GN disruption.

The effects of preceding sound on psychoacoustic tuning curves measured in simultaneous and forward masking

The medial olivocochlear reflex (MOCR) decreases the gain of the cochlear active process in response to sound. We have used psychoacoustic techniques to show behavioral effects of gain reduction, which could be consistent with the MOCR. We have used paradigms understood to measure frequency selectivity and the input/output function at the level of the cochlea using stimuli (masker and signal) that should be too short to evoke the MOCR. A precursor sound is then presented before these stimuli to evoke the MOCR. Our most recent studies have used forward masking, to avoid the complicating effects of suppression. The current study was designed to examine the effects of suppression, by comparing the effects of a precursor on frequency selectivity measured using forward masking and simultaneous masking. Psychoacoustic tuning curves were measured using simultaneous and forward masking, with and without a precursor. This allowed us to measure the change in tuning with and without the presence of suppression. Broadband precursors and tonal precursors at masker frequencies were measured because previous studies have shown broadening with broadband precursors and sharpening when tonal maskers are the precursors. Results will be discussed in the context of current understanding of the MOCR.

Semi-supervised Learning for Automatic Modulation Recognition Using Haar Time–Frequency Mask and Positional–Spatial Attention

Automatic modulation recognition plays an important role in many military and civilian applications, including cognitive radio, spectrum sensing, signal surveillance, and interference identification. Due to the powerful ability of deep learning to extract hidden features and perform classification, it can extract highly separative features from massive signal samples. Considering the condition of limited training samples, we propose a semi-supervised learning framework based on Haar time–frequency (HTF) mask data augmentation and the positional–spatial attention (PSA) mechanism. Specifically, the HTF mask is designed to increase data diversity, and the PSA is designed to address the limited receptive field of the convolutional layer and enhance the feature extraction capability of the constructed network. Extensive experimental results obtained on the public RML2016.10a dataset show that the proposed semi-supervised framework utilizes 1% of the given labeled data and reaches a recognition accuracy of 92.09% under 6 dB signals.

Normalized formula for removing damping effect in recovering bridge mode shapes using a moving and a stationary vehicle

Recovering bridge mode shapes using the response of a moving test vehicle has become a technique of interest. However, the mode shapes recovered tend to be distorted by bridge damping. To remove the distortion effect, a normalized formula that utilizes the responses of a moving and a stationary vehicle is devised for recovering bridge mode shapes with no prior knowledge of bridge damping ratios. Specifically, the moving test vehicle is used to recover the global modal response of the bridge throughout its span length at different instants as the vehicle moves over, and the stationary vehicle to generate a reference response at a fixed location of the bridge for removing the damping effect. In this paper, closed-form solutions are derived for the dynamic responses of the bridge and two test vehicles. The contact response is used in place of the vehicle response to alleviate the vehicle frequency's masking effect on bridge frequencies. The formula for recovering the modal mode shape is derived by normalizing the instantaneous amplitude of the Hilbert transform (HT) for the bridge from the moving test vehicle with respect to that by the stationary vehicle. The following conclusions are drawn: (1) the damping distortion effect on bridge mode shapes can be removed by the normalized formula; (2) the normalized formula is robust with regard to bridge damping, stationary vehicle location, and vehicle speed for beam-type bridges; (3) the mode shapes recovered by the present method are acceptable, even in the presence of pavement roughness, when aided by an accompanying truck.

Non-intrusive Speech Intelligibility Prediction Method for Reverberant Speech Using Neural Network-Based Frequency Segmentation and Masking Front-end

In conventional non-intrusive speech intelligibility estimation, reverberation is extracted from the time-frequency representation of the input by explicit filter bank processing or spectral masking. However, these filter banks and masking processes are not always optimal. We replaced these processes with convolutional neural networks using rectangular kernels restricted to the frequency direction and masking such as a self-attention mechanism. We believe that this will enable feature extraction that is optimal for intelligibility estimation and will enable its estimation with high accuracy that generalizes well to input under various conditions. We further applied this front-end CNN to a previously proposed prediction model using speech enhancement. As a result, the estimation accuracy was improved compared to conventional front-ends using fixed filter banks, and this prediction showed a correlation coefficient with the subjective evaluation of 0.84 compared to 0.80 with the fixed filter bank.

Improved STFT analysis using time-frequency masking for chatter detection in the milling process

In the field of machining, the purposeful increase in cutting depth and rotation speed can potentially cause chatter vibration. Chatter lowers surface quality, tool and spindle life, and produces excessive noise, which limits the productivity of the machining process. In this paper, an improved Short-Time Fourier Transform (STFT) involving time–frequency masking and adaptive thresholding is proposed. Initially, hammering tests were undertaken to ascertain the inherent natural frequency of the cutting system. Subsequently, milling tests were conducted across varied conditions utilizing an accelerometer sensor and a microphone (capturing sound signals) to acquire the requisite experimental data. A 3-level Variational Mode Decomposition (VMD) was used for pre-processing due to its inherentcapabilities for noise reduction. The chatter threshold is established by averaging the total intrinsic mode function (IMF) energy levels that have been selected. To demonstrate the efficacy of the proposed method, root mean square (RMS) and skewness were calculated as chatter indicators. The improved STFT time–frequency representation (TFR) was compared to the conventional TFR. Notably, both indicators demonstrate effectiveness and chatter sensitivity, while the improved STFT offers greater time–frequency resolution than its conventional counterpart and can be successfully used for monitoring the milling process.

EnViTSA: Ensemble of Vision Transformer with SpecAugment for Acoustic Event Classification.

Recent successes in deep learning have inspired researchers to apply deep neural networks to Acoustic Event Classification (AEC). While deep learning methods can train effective AEC models, they are susceptible to overfitting due to the models' high complexity. In this paper, we introduce EnViTSA, an innovative approach that tackles key challenges in AEC. EnViTSA combines an ensemble of Vision Transformers with SpecAugment, a novel data augmentation technique, to significantly enhance AEC performance. Raw acoustic signals are transformed into Log Mel-spectrograms using Short-Time Fourier Transform, resulting in a fixed-size spectrogram representation. To address data scarcity and overfitting issues, we employ SpecAugment to generate additional training samples through time masking and frequency masking. The core of EnViTSA resides in its ensemble of pre-trained Vision Transformers, harnessing the unique strengths of the Vision Transformer architecture. This ensemble approach not only reduces inductive biases but also effectively mitigates overfitting. In this study, we evaluate the EnViTSA method on three benchmark datasets: ESC-10, ESC-50, and UrbanSound8K. The experimental results underscore the efficacy of our approach, achieving impressive accuracy scores of 93.50%, 85.85%, and 83.20% on ESC-10, ESC-50, and UrbanSound8K, respectively. EnViTSA represents a substantial advancement in AEC, demonstrating the potential of Vision Transformers and SpecAugment in the acoustic domain.

Steep temporal integration for tone detection in a multi-tone, random-frequency masker.

In the detection of a tone burst, masking by several tones with random frequencies can produce steep temporal integration. This feature was evaluated for nine normal-hearing adults for 1000-Hz tone bursts presented in a continuous train of four-tone masker bursts. Masker frequencies were randomly selected (250-4000 Hz) for each burst, with the proviso that all tones were separated by ≥0.2 oct. Bursts were 80-ms in duration; when present, signal bursts were gated synchronously with masker bursts. The observed mean temporal-integration function was exceptionally steep-thresholds improved by 26 dB as signal duration increased from 1 to 8 bursts. The results also showed that the individual differences were large, and that the mean psychometric function was exceptionally shallow, spanning a range of 35 dB between 0.6 and 0.9 proportion correct responses, consistent with previous reports. These findings were interpreted in the context of three signal-detection models, one based on the absolute-level cue, and two based on the relative-level cue via template matching; all cues were derived from the excitation patterns of the stimuli. Template-matching models were able to predict the shallow psychometric functions as observed, but all models fall short in the steepness of the observed temporal integration.

Beyond noise levels: vehicle classification using psychoacoustic indicators from pass-by road traffic noise and their correlations with speed and temperature

Environmental noise control regulations typically employ noise level descriptors to set limits for noise exposure. However, other characteristics of noise, such as frequency content, temporal patterns and masking, have been proven to influence the perception of acoustic environments. In this sense, psychoacoustic indicators offer an objective means of establishing relationships between physical characteristics of noise and the human auditory sensation phenomena. This study explored psychoacoustic indicators of pass-by vehicle noise across different vehicle categories, driving speeds, and temperatures. Moreover, the indicators were exploited as features to train a classification algorithm to predict vehicle category. Over 2000 vehicle noise samples were collected using the Statistical Pass-By (SPB) method, categorized into three classes according to ISO 11819-1, besides an additional class for delivery vans. Correction coefficients were obtained for temperature and speed to noise levels, loudness, roughness, sharpness and fluctuation strength. The differences in these indicators based on vehicle category were then discussed. A vehicle-category predictive model using the three vehicle categories defined in ISO 11819-1 yielded 84% accuracy. Including vans as an extra vehicle category dropped accuracy to 72% due to their misclassification with passenger cars. Combining these two categories increased overall accuracy to 86%. These findings could enable a less visual-dependent vehicle categorization so that vehicle fleets worldwide are more consistently classified in terms of noise. Additionally, psychoacoustic indicators appear to be valuable features for vehicle classification systems aimed to resemble the human auditory experience.

Inadequate Sampling Frequency and Imprecise Taxonomic Identification Mask Results in Studies of Migratory Freshwater Fish Ichthyoplankton

In South America, knowledge of major spawning sites is crucial for maintaining migratory fish populations. In this study, we aimed to understand the spatio-temporal distribution of fish eggs in the upper São Francisco River using high sampling frequency and DNA metabarcoding identification. We evaluated the possible effects of the non-molecular identification of eggs and decreased sampling frequency on the determination of spawning sites and major breeding periods. Collections were carried out every three days from November 2019 to February 2020. We found that, if we had assumed that all of the free and non-adhesive sampled eggs belonged to migratory species, as is usual in the literature, this assumption would have been wrong for both the spawning sites and the breeding periods. Moreover, any decrease in the frequency of sampling could dramatically affect the determination of the major spawning rivers, and the spawning events of some of the migratory species may not have been detected. Therefore, without the proper identification and adequate sampling frequency of eggs, important spawning sites may be overlooked, leading to ineffective or inappropriate conservation measures.

Vocalization behavior response of captive bottlenose dolphins (Tursiops truncatus) to playbacks of pile driving noise

Anthropogenic noises in the ocean may cause significant impacts on the behavior of marine mammals, including stress, hearing impairment, and disruptions in social interactions. To investigate the potential effects of off-shore wind farm pile driving noise on dolphins, we conducted controlled experiments to 3 captive bottlenose dolphins (Tursiops truncatus) exposed to pile driving noise playbacks at four different sound pressure levels (Mean Lpeak 0,127,147,160 dB re 1 μPa) with a Lubell LL1424HP underwater transducer. Several vocalizational parameters of the underwater video recordings during the exposure were analyzed to assess the quantity of vocalizations and frequency modulations. Our findings revealed a notable increase in frequency modulation within whistle calls as the sound levels of pile driving noises increased. Changes in the number of calls at different noise levels can also be noticed. Additionally, we observed time-matching jaw claps to the pile driving sounds. These results suggest that pile driving activity several kilometres away from the habitat may still cause frequency masking, leading to altered behaviors of dolphins and potentially inducing population-level stress. Understanding the impact of anthropogenic noise including behavioral and acoustic responses, auditory masking, and stress on marine species is crucial for effective conservation and mitigation strategies.

Regularized Contrastive Masked Autoencoder Model for Machinery Anomaly Detection Using Diffusion-Based Data Augmentation

Unsupervised anomalous sound detection, especially self-supervised methods, plays a crucial role in differentiating unknown abnormal sounds of machines from normal sounds. Self-supervised learning can be divided into two main categories: Generative and Contrastive methods. While Generative methods mainly focus on reconstructing data, Contrastive learning methods refine data representations by leveraging the contrast between each sample and its augmented version. However, existing Contrastive learning methods for anomalous sound detection often have two main problems. The first one is that they mostly rely on simple augmentation techniques, such as time or frequency masking, which may introduce biases due to the limited diversity of real-world sounds and noises encountered in practical scenarios (e.g., factory noises combined with machine sounds). The second issue is dimension collapsing, which leads to learning a feature space with limited representation. To address the first shortcoming, we suggest a diffusion-based data augmentation method that employs ChatGPT and AudioLDM. Also, to address the second concern, we put forward a two-stage self-supervised model. In the first stage, we introduce a novel approach that combines Contrastive learning and masked autoencoders to pre-train on the MIMII and ToyADMOS2 datasets. This combination allows our model to capture both global and local features, leading to a more comprehensive representation of the data. In the second stage, we refine the audio representations for each machine ID by employing supervised Contrastive learning to fine-tune the pre-trained model. This process enhances the relationship between audio features originating from the same machine ID. Experiments show that our method outperforms most of the state-of-the-art self-supervised learning methods. Our suggested model achieves an average AUC and pAUC of 94.39% and 87.93% on the DCASE 2020 Challenge Task2 dataset, respectively.