Representation Of Acoustic Signals Research Articles

Self-supervised learning–based underwater acoustical signal classification via mask modeling

The classification of underwater acoustic signals has garnered a great deal of attention in recent years due to its potential applications in military and civilian contexts. While deep neural networks have emerged as the preferred method for this task, the representation of the signals plays a crucial role in determining the performance of the classification. However, the representation of underwater acoustic signals remains an under-explored area. In addition, the annotation of large-scale datasets for the training of deep networks is a challenging and expensive task. To tackle these challenges, we propose a novel self-supervised representation learning method for underwater acoustic signal classification. Our approach consists of two stages: a pretext learning stage using unlabeled data and a downstream fine-tuning stage using a small amount of labeled data. The pretext learning stage involves randomly masking the log Mel spectrogram and reconstructing the masked part using the Swin Transformer architecture. This allows us to learn a general representation of the acoustic signal. Our method achieves a classification accuracy of 80.22% on the DeepShip dataset, outperforming or matching previous competitive methods. Furthermore, our classification method demonstrates good performance in low signal-to-noise ratio or few-shot settings.

К формированию модели слуховой функции в задаче освещения окружающей обстановки

The large-scale-temporal representation of acoustic signals and technical solutions for their processing, allowing visualization of the environment in conditions of limited visibility using the operator's auditory analyzer, are considered. The principle of symmetry of physical laws is extended to the processing of signals from objects that are the purposes of detection and classification. Mod-els of scale-time representation of acoustic signals and their processing for visualization of the scene by the operator are proposed.

Acoustic detection of unmanned aerial vehicles using biologically inspired vision processing.

Robust detection of acoustically quiet, slow-moving, small unmanned aerial vehicles is challenging. A biologically inspired vision approach applied to the acoustic detection of unmanned aerial vehicles is proposed and demonstrated. The early vision system of insects significantly enhances signal-to-noise ratios in complex, cluttered, and low-light (noisy) scenes. Traditional time-frequency analysis allows acoustic signals to be visualized as images using spectrograms and correlograms. The signals of interest in these representations of acoustic signals, such as linearly related harmonics or broadband correlation peaks, essentially offer equivalence to meaningful image patterns immersed in noise. By applying a model of the photoreceptor stage of the hoverfly vision system, it is shown that the acoustic patterns can be enhanced and noise greatly suppressed. Compared with traditional narrowband and broadband techniques, the bio-inspired processing can extend the maximum detectable distance of the small and medium-sized unmanned aerial vehicles by between 30% and 50%, while simultaneously increasing the accuracy of flight parameter and trajectory estimations.

Impaired interaural correlation processing in people with schizophrenia.

Detection of transient changes in interaural correlation is based on the temporal precision of the central representations of acoustic signals. Whether schizophrenia impairs the temporal precision in the interaural correlation process is not clear. In both participants with schizophrenia and matched healthy-control participants, this study examined the detection of a break in interaural correlation (BIC, a change in interaural correlation from 1 to 0 and back to 1), including the longest interaural delay at which a BIC was just audible, representing the temporal extent of the primitive auditory memory (PAM). Moreover, BIC-induced electroencephalograms (EEGs) and the relationships between the early binaural psychoacoustic processing and higher cognitive functions, which were assessed by the Repeatable Battery for the Assessment of Neuropsychological Status (RBANS), were examined. The results showed that compared to healthy controls, participants with schizophrenia exhibited poorer BIC detection, PAM and RBANS score. Both the BIC-detection accuracy and the PAM extent were correlated with the RBANS score. Moreover, participants with schizophrenia showed weaker BIC-induced N1-P2 amplitude which was correlated with both theta-band power and inter-trial phase coherence. These results suggested that schizophrenia impairs the temporal precision of the central representations of acoustic signals, affecting both interaural correlation processing and higher-order cognitions.

Unsupervised Phoneme and Word Discovery From Multiple Speakers Using Double Articulation Analyzer and Neural Network With Parametric Bias.

This paper describes a new unsupervised machine-learning method for simultaneous phoneme and word discovery from multiple speakers. Phoneme and word discovery from multiple speakers is a more challenging problem than that from one speaker, because the speech signals from different speakers exhibit different acoustic features. The existing method, a nonparametric Bayesian double articulation analyzer (NPB-DAA) with deep sparse autoencoder (DSAE) only performed phoneme and word discovery from a single speaker. Extending NPB-DAA with DSAE to a multi-speaker scenario is, therefore, the research problem of this paper.This paper proposes the employment of a DSAE with parametric bias in the hidden layer (DSAE-PBHL) as a feature extractor for unsupervised phoneme and word discovery. DSAE-PBHL is designed to subtract speaker-dependent acoustic features and speaker-independent features by introducing parametric bias input to the DSAE hidden layer. An experiment demonstrated that DSAE-PBHL could subtract distributed representations of acoustic signals, enabling extraction based on the types of phonemes rather than the speakers. Another experiment demonstrated that a combination of NPB-DAA and DSAE-PBHL outperformed other available methods accomplishing phoneme and word discovery tasks involving speech signals with Japanese vowel sequences from multiple speakers.

Binaural unmasking of the accuracy of envelope-signal representation in rat auditory cortex but not auditory midbrain

Accurate neural representations of acoustic signals under noisy conditions are critical for animals’ survival. Detecting signal against background noise can be improved by binaural hearing particularly when an interaural-time-difference (ITD) disparity is introduced between the signal and the noise, a phenomenon known as binaural unmasking. Previous studies have mainly focused on the binaural unmasking effect on response magnitudes, and it is not clear whether binaural unmasking affects the accuracy of central representations of target acoustic signals and the relative contributions of different central auditory structures to this accuracy. Frequency following responses (FFRs), which are sustained phase-locked neural activities, can be used for measuring the accuracy of the representation of signals. Using intracranial recordings of local field potentials, this study aimed to assess whether the binaural unmasking effects include an improvement of the accuracy of neural representations of sound-envelope signals in the rat IC and/or auditory cortex (AC). The results showed that (1) when a narrow-band noise was presented binaurally, the stimulus-response (S-R) coherence of the FFRs to the envelope (FFRenvelope) of the narrow-band noise recorded in the IC was higher than that recorded in the AC. (2) Presenting a broad-band masking noise caused a larger reduction of the S-R coherence for FFRenvelope in the IC than that in the AC. (3) Introducing an ITD disparity between the narrow-band signal noise and the broad-band masking noise did not affect the IC S-R coherence, but enhanced both the AC S-R coherence and the coherence between the IC FFRenvelope and AC FFRenvelope. Thus, although the accuracy of representing envelope signals in the AC is lower than that in the IC, it can be binaurally unmasked, indicating a binaural-unmasking mechanism that is formed during the signal transmission from the IC to the AC.

Neural representation of interaural correlation in human auditory brainstem: Comparisons between temporal-fine structure and envelope

Central processing of interaural correlation (IAC), which depends on the precise representation of acoustic signals from the two ears, is essential for both localization and recognition of auditory objects. A complex soundwave is initially filtered by the peripheral auditory system into multiple narrowband waves, which are further decomposed into two functionally distinctive components: the quickly-varying temporal-fine structure (TFS) and the slowly-varying envelope. In rats, a narrowband noise can evoke auditory-midbrain frequency-following responses (FFRs) that contain both the TFS component (FFRTFS) and the envelope component (FFREnv), which represent the TFS and envelope of the narrowband noise, respectively. These two components are different in sensitivity to the interaural time disparity. In human listeners, the present study investigated whether the FFRTFS and FFREnv components of brainstem FFRs to a narrowband noise are different in sensitivity to IAC and whether there are potential brainstem mechanisms underlying the integration of the two components. The results showed that although both the amplitude of FFRTFS and that of FFREnv were significantly affected by shifts of IAC between 1 and 0, the stimulus-to-response correlation for FFRTFS, but not that for FFREnv, was sensitive to the IAC shifts. Moreover, in addition to the correlation between the binaurally evoked FFRTFS and FFREnv, the correlation between the IAC-shift-induced change of FFRTFS and that of FFREnv was significant. Thus, the TFS information is more precisely represented in the human auditory brainstem than the envelope information, and the correlation between FFRTFS and FFREnv for the same narrowband noise suggest a brainstem binding mechanism underlying the perceptual integration of the TFS and envelope signals.

Investigation of signal models and methods for evaluating structures of processing telecommunication information exchange systems under acoustic noise conditions

The paper considers models and methods for estimating signals during the transmission of information messages in telecommunication systems of audio exchange. One-dimensional probability distribution functions that can be used to isolate useful signals, and acoustic noise interference are presented. An approach to the estimation of the correlation and spectral functions of the parameters of acoustic signals is proposed, based on the parametric representation of acoustic signals and the components of the noise components. The paper suggests an approach to improving the efficiency of interference cancellation and highlighting the necessary information when processing signals from telecommunications systems. In this case, the suppression of acoustic noise is based on the methods of adaptive filtering and adaptive compensation. The work also describes the models of echo signals and the structure of subscriber devices in operational command telecommunications systems.

Redundant representation of acoustic signals using curvelet transform and its application to speech denoising

Redundant representation of acoustic signals using curvelet transform and its application to speech denoising

Differential conduction velocity regulation in ipsilateral and contralateral collaterals innervating brainstem coincidence detector neurons.

Information processing in the brain relies on precise timing of signal propagation. The highly conserved neuronal network for computing spatial representations of acoustic signals resolves microsecond timing of sounds processed by the two ears. As such, it provides an excellent model for understanding how precise temporal regulation of neuronal signals is achieved and maintained. The well described avian and mammalian brainstem circuit for computation of interaural time differences is composed of monaural cells in the cochlear nucleus (CN; nucleus magnocellularis in birds) projecting to binaurally innervated coincidence detection neurons in the medial superior olivary nucleus (MSO) in mammals or nucleus laminaris (NL) in birds. Individual axons from CN neurons issue a single axon that bifurcates into an ipsilateral branch and a contralateral branch that innervate segregated dendritic regions of the MSO/NL coincidence detector neurons. We measured conduction velocities of the ipsilateral and contralateral branches of these bifurcating axon collaterals in the chicken by antidromic stimulation of two sites along each branch and whole-cell recordings in the parent neurons. At the end of each experiment, the individual CN neuron and its axon collaterals were filled with dye. We show that the two collaterals of a single axon adjust the conduction velocities individually to achieve the specific conduction velocities essential for precise temporal integration of information from the two ears, as required for sound localization. More generally, these results suggest that individual axonal segments in the CNS interact locally with surrounding neural structures to determine conduction velocity.

Comparison of metrics for the evaluation of similarity in acoustic pressure signals

Determining if aeroacoustic sound predictions are accurate is difficult because the question of how to define ‘accurate’ remains open. This communication evaluates four metrics for comparing time-domain pressure signals, each implying its own definition of ‘accurate.’ An adaptation of a Structural Similarity Metric (originating from the image processing literature) to time–frequency representations of acoustic signals is shown to outperform typical metrics such as relative energy and mean square error.

Improving word recognition in noise among hearing-impaired subjects with a single-channel cochlear noise-reduction algorithm

A common complaint of the hearing impaired is the inability to understand speech in noisy environments even with their hearing assistive devices. Only a few single-channel algorithms have significantly improved speech intelligibility in noise for hearing-impaired listeners. The current study introduces a cochlear noise reduction algorithm. It is based on a cochlear representation of acoustic signals and real-time derivation of a binary speech mask. The contribution of the algorithm for enhancing word recognition in noise was evaluated on a group of 42 normal-hearing subjects, 35 hearing-aid users, 8 cochlear implant recipients, and 14 participants with bimodal devices. Recognition scores of Hebrew monosyllabic words embedded in Gaussian noise at several signal-to-noise ratios (SNRs) were obtained with processed and unprocessed signals. The algorithm was not effective among the normal-hearing participants. However, it yielded a significant improvement in some of the hearing-impaired subjects under different listening conditions. Its most impressive benefit appeared among cochlear implant recipients. More than 20% improvement in recognition score of noisy words was obtained by 12, 16, and 26 hearing-impaired at SNR of 30, 24, and 18 dB, respectively. The algorithm has a potential to improve speech intelligibility in background noise, yet further research is required to improve its performances.

Modeling cochlear dynamics: Interrelation between cochlea mechanics and psychoacoustics

A model of the cochlea was used to bridge the gap between model approaches commonly used to investigate phenomena related to otoacoustic emissions and more filter-based model approaches often used in psychoacoustics. In the present study, a nonlinear and active one-dimensional transmission line model was developed that accounts for several aspects of physiological data with a single fixed parameter set. The model shows plausible excitation patterns and an input-output function similar to the linear-compressive-linear function as hypothesized in psychoacoustics. The model shows realistic results in a two-tone suppression paradigm and a plausible growth function of the 2f(1)-f(2) component of distortion product otoacoustic emissions. Finestructure was found in simulated stimulus-frequency otoacoustic emissions (SFOAE) with realistic levels and rapid phase rotation. A plausible "threshold in quiet" including finestructure and spontaneous otoacoustic emissions (SOAE) could be simulated. It is further shown that psychoacoustical data of modulation detection near threshold can be explained by the mechanical dynamics of the modeled healthy cochlea. It is discussed that such a model can be used to investigate the representation of acoustic signals in healthy and impaired cochleae at this early stage of the auditory pathway for both, physiological as well as psychoacoustical paradigms.

Gibbs sampling for modal arrival time and amplitude estimation from time-frequency representations of acoustic signals.

A Gibbs sampling-maximum a posteriori approach is developed for modal arrival time and amplitude estimation from time-frequency representations of broadband acoustic signals propagating in underwater media. The goal is to obtain accurate estimates of arrival times of propagating modes and corresponding amplitudes, which can then be employed for source localization and geoacoustic inversion. The method provides uncertainty information on both modal arrival time and amplitude estimates, typically unavailable when traditional methods are used. Estimates and uncertainty are propagated through an inversion process, generating posterior probability distributions of source range and geoacoustic properties. [Work supported by ONR.]

Microelectronic package characterisation using scanning acoustic microscopy

In a highly competitive market, reliable techniques for manufacturing quality control of electronic devices are demanded. Characterisation of modern microelectronic package integrity becomes more difficult due to the continued miniaturisation of electronic device and the complexity of advanced micro-assembling technologies such as chip-scale packages and 3D IC stacks. In this paper, sparse representations of acoustic signals are sought to improve the scanning acoustic microscopy (SAM), a common non-destructive tool for failure analysis of microelectronic packages. Sparse representation of an ultrasonic signal is obtained by decomposing it in an overcomplete dictionary. Detection and location of ultrasonic echoes are then performed on the basis of the resulting redundant representation. The method offers a solution to the deconvolution problem for restoration of the ultrasonic reflectivity function. It can restore closely space overlapping echoes beyond the resolution of the conventional SAM system. It also produces high resolution and accurate estimates for ultrasonic echo parameters, i.e., time-of-flight, amplitude, centre frequency, and bandwidth. These merits of the proposed method are explored in various potential applications for microelectronic package characterisation.

Representation of acoustic signals in the human cochlea in presence of a cochlear implant electrode

Background: In subjects with remaining low frequency hearing, combined electric-acoustic stimulation (EAS) of the auditory system is a new therapeutic perspective. Intracochlear introduction of a cochlear implant electrode, however, may alter the biomechanical properties of the inner ear and thus affect perception of acoustic stimuli. Study design: Based on histological observations of morphologic changes after cochlear implantation in cadaveric and post mortem studies the effects of basilar membrane (BM) stiffening in the ascending basal and middle turns of the cochlea due to close contact of the BM with the electrode were simulated in a 3D-computational finite element model of the inner ear. To verify our simulated results, pre- and postoperative pure-tone audiograms of 13 subjects with substantial residual hearing, who underwent cochlear implantation, were evaluated. Results: In the scenario of partial BM-fixation, acoustic energy of middle (2 kHz) and high (6 kHz) frequency was focused basally and apically to the fixed section, increasing BM displacement amplitudes up to 6 dB at a stimulation level of 94 dB (SPL). Lower frequencies were not affected by fixation in the basal and middle turn of the cochlea. In implanted subjects, a small but significant decrease of thresholds was observed at 1.5 kHz, a place in tonotopy adjacent to the tip region of the implanted electrode. Conclusion: Our model suggests that stiffening of the basilar membrane adjacent to an implanted electrode into the basal and middle cochlear turn did not affect BM movement in the low frequency area. Focussing of acoustic energy may increase perception in regions adjacent to the fixed section. Observations in implanted subjects were concordant with our model predictions. High frequencies, however, should not be amplified in patients using EAS to avoid disturbances in discrimination due to tonotopically incorrect frequency representation.

Representation of acoustic signals in the eighth nerve of the tokay gecko II. Masking of pure tones with noise

Acoustic signals are generally encoded in the peripheral auditory system of vertebrates by a duality scheme. For frequency components that fall within the excitatory tuning curve, individual eighth nerve fibers can encode the effective spectral energy by a spike-rate code, while simultaneously preserving the signal waveform periodicity of lower frequency components by phase-locked spike-train discharges. To explore how robust this duality of representation may be in the presence of noise, we recorded the responses of auditory fibers in the eighth nerve of the Tokay gecko to tonal stimuli when masking noise was added simultaneously. We found that their spike-rate functions reached plateau levels fairly rapidly in the presence of noise, so the ability to signal the presence of a tone by a concomitant change in firing rate was quickly lost. On the other hand, their synchronization functions maintained a high degree of phase-locked firings to the tone even in the presence of high-intensity masking noise, thus enabling a robust detection of the tonal signal. Critical ratios (CR) and critical bandwidths showed that in the frequency range where units are able to phaselock to the tonal periodicity, the CR bands were relatively narrow and the bandwidths were independent of noise level. However, to higher frequency tones where phaselocking fails and only spike-rate codes apply, the CR bands were much wider and depended upon noise level, so that their ability to filter tones out of a noisy background degraded with increasing noise levels. The greater robustness of phase-locked temporal encoding contrasted with spike-rate coding verifies a important advantage in using lower frequency signals for communication in noisy environments.

Speech intelligibility prediction in hearing-impaired listeners based on a psychoacoustically motivated perception model.

Sensorneural hearing-impaired listeners suffer severely from deterioration in the processing and internal representation of acoustic signals. In order to understand this deterioration in detail, a numerical perception model was developed which is based on current functional models of the signal processing in the auditory system. To test this model, the individual's speech intelligibility in quiet and in noise was predicted. The primary input parameter of the model is the precisely measured audiogram of each listener. In a refined version of the model, additional input parameters are derived from predicting the individual's temporal forward masking and notched-noise measurements with the same model assumptions. The predictions of the perception model were compared with those of the articulation index (AI) and the speech transmission index (STI). The accuracy of prediction with the perception model is in the same range as with the AI and the STI. The model does not require a calibration function and has the advantage of a greater flexibility in including different processing deficits associated with hearing impairment. However, it requires more time for computation. For the hearing-impaired listeners examined so far the individually measured psychoacoustical parameters have only a secondary effect on the prediction as compared to the audiogram. Nevertheless, the underlying model is a first step toward a quantitative understanding of speech intelligibility and helps to distinguish between the influence of the "attenuation" and the "distortion" component of the hearing loss.

A wavelet auditory model and noise suppression

Based on a wavelet auditory model (WAM) of the mammalian auditory system, discrete time‐scale representations of acoustic signals which are inherently robust to ‘‘noise’’ emerge. The model consists of the two main processes of (i) a ‘‘continuous’’ wavelet transform with a causal analyzing wavelet determined by a ‘‘shark‐fin’’‐shaped frequency response (cochlear filter), and (ii) a specific and signal‐dependent sampling of the wavelet transform. In general the pattern of samples in the wavelet domain is irregular and determined at lower frequencies by activity at higher frequencies. By analyzing the model in terms of the theory of mathematical frames, it is shown that acoustic signals may be fully recovered from their WAM representation through iterative reconstruction algorithms. Noise is thought of as that portion of an acoustic signal which is ‘‘incoherent’’ with respect to the underlying WAM frame functions. Because coherent energy is highly localized and incoherent (noise) energy is necessarily scattered in the time‐scale plane by the WAM frame representation, wavelet shrinkage techniques provide powerful algorithms for noise suppression. Several numerical examples of the noise suppression abilities of these algorithms are presented in the paper.

Representation of acoustic signals in the eighth nerve of the Tokay gecko: I. Pure tones

A systematic study of the encoding properties of 146 auditory nerve fibers in the Tokay gecko ( Gekko gecko, L) was conducted with respect to pure tones and two-tone rate suppression. Our aim was a comprehensive understanding of the peripheral encoding of simple tonal stimuli and their representation by temporal synchronization and spike rate codes as a prelude to subsequent studies of more complex signals. Auditory nerve fibers in the Tokay gecko have asymmetrical, V-shaped excitatory tuning curves with best excitatory frequencies that range from 200–5100 Hz and thresholds between 4–35 dB SPL. A low-frequency excitatory ‘tail’ extends far into the low-frequency range and two-tone suppression is present only on the high frequency side of the tuning curve. The response properties to pure tones at different loci within a tuning curve can differ greatly, due to evident interactions between the representations of temporal, spectral and intensity stimulus features. For frequencies below 1250 Hz, pure tones are encoded by both temporal synchronization and spike rate codes, whereas above this frequency a fiber's ability to encode the waveform periodicity is lost and only a rate code predominates. These complimentary representations within a tuning curve raise fundamental issues which need to be addressed in interpreting how more complex, bioacoustic communication signals are represented in the peripheral and central auditory system. And since auditory nerve fibers in the Tokay gecko exhibit tonal sensitivity, selective frequency tuning, and iso-intensity and iso-frequency contours that seem comparable to similar measures in birds and mammals, these issues likely apply to most higher vertebrates in general. The simpler wiring diagram of the reptilian auditory system, coupled with the Tokay gecko's remarkable vocalizations, make this animal a good evolutionary model in which to experimentally explore the encoding of more complex sounds of communicative significance.