Single Audio Research Articles

Vehicle classification based on audio-visual feature fusion with low-quality images and noise

In Intelligent Transport Systems (ITS), vision is the primary mode of perception. However, vehicle images captured by low-cost traffic cameras under challenging weather conditions often suffer from poor resolution and insufficient detail representation. On the other hand, vehicle noise provides complementary auditory features that offer advantages such as environmental adaptability and a large recognition distance. To address these limitations and enhance the accuracy of low-quality traffic surveillance classification and identification, an effective audio-visual feature fusion method is crucial. This paper presents a research study that establishes an Urban Road Vehicle Audio-visual (URVAV) dataset specifically designed for low-quality images and noise recorded in complex weather conditions. For low-quality vehicle image classification, the paper proposes a simple Convolutional Neural Network (CNN)-based model called Low-quality Vehicle Images Net (LVINet). Additionally, to further enhance classification accuracy, a spatial channel attention-based audio-visual feature fusion method is introduced. This method converts one-dimensional acoustic features into a two-dimensional audio Mel-spectrogram, allowing for the fusion of auditory and visual features. By leveraging the high correlation between these features, the representation of vehicle characteristics is effectively enhanced. Experimental results demonstrate that LVINet achieves a classification accuracy of 93.62% with reduced parameter count compared to existing CNN models. Furthermore, the proposed audio-visual feature fusion method improves classification accuracy by 7.02% and 4.33% when compared to using single audio or visual features alone, respectively.

Forest Sound Event Detection with Convolutional Recurrent Neural Network-Long Short-Term Memory

Sound event detection tackles an audio environment's complex sound analysis and recognition problem. The process involves localizing and classifying sounds mainly to estimate the start point and end points of the separate sounds and describe each sound. Sound event detection capability relies on the type of sound. Although detecting sequences of distinct temporal sounds is straightforward, the situation becomes complex when the sound is multiple overlapping of much single audio. This situation usually occurs in the forest environment. Therefore, this aim of the paper is to propose a Convolution Recurrent Neural Network-Long Short-Term Memory algorithm to detect an audio signature of intruders in the forest environment. The audio is extracted in the Mel-frequency cepstrum coefficient and fed into the algorithm as an input. Six sound categories are chainsaw, machete, car, hatchet, ambiance, and bike. They were tested using several epochs, batch size, and filter of the layer in the model. The proposed model can achieve an accuracy of 98.52 percent in detecting the audio signature with a suitable parameter selection. In the future, additional types of audio signatures of intruders and further aspects of evaluation can be added to make the algorithm better at detecting intruders in the forest environment.

Self-supervised speech denoising using only noisy audio signals

In traditional speech denoising tasks, clean audio signals are often used as the training target, but absolutely clean signals are collected from expensive recording equipment or in studios with the strict environments. To overcome this drawback, we propose an end-to-end self-supervised speech denoising training scheme using only noisy audio signals, named Only-Noisy Training (ONT), without extra training conditions. The proposed ONT strategy constructs training pairs only from each single noisy audio, and it contains two modules: training audio pairs generated module and speech denoising module. The first module adopts a random audio sub-sampler on each noisy audio to generate training pairs. The sub-sampled pairs are then fed into a novel complex-valued speech denoising module. Experimental results show that the proposed method not only eliminates the high dependence on clean targets of traditional audio denoising tasks, but also achieves on-par or better performance than other training strategies. Availability—ONT is available at https://github.com/liqingchunnnn/Only-Noisy-Training

How does materiality ‘bite back’? Investigating cassette tapes in local, translocal and virtual music scenes

Despite its status as an analogue sound carrier, the cassette has shown remarkable resilience in the digital era. Drawing on qualitative data gathered in three significant markets for cassettes, Japan, Australia and the USA, during 2018 and 2019, this article explores how the cassette tape’s material significance in the 21st century manifests itself in a complex network of interrelated local, translocal and virtual practices of music creation, distribution and consumption. The article draws on Magaudda’s ‘circuit of practice’ concept and Peterson and Bennett’s three tier model of scenes (local, trans-local and virtual). Taking the cassette’s hybrid occurrence and usage throughout a plethora of highly distinctive music scenes in Japan, Australia and the United States as a basis, we argue that Magaudda’s ‘circuit of practice’ theory needs to be structurally extended to grasp the multifaceted circuits of music in the digital age. This requires that the occurrence of a single audio format such as the cassette is recurringly analysed within different cultural contexts in order to map and delineate the format’s overall significance for contemporary music practices.

Multimodal Music Emotion Recognition Method Based on the Combination of Knowledge Distillation and Transfer Learning

The main difficulty of music emotion recognition is the lack of sufficient labeled data. Only the labeled data with unbalanced categories are used to train the emotion recognition model. Not only is accurate labeling of emotion categories costly and time-consuming, but it also requires extensive musical background for labelers At the same time, the emotion of music is often affected by many factors. Singing methods, music styles, arrangement methods, lyrics, and other factors will affect the expression of music emotions. This paper proposes a multimodal method based on the combination of knowledge distillation and music style transfer learning and verifies the effectiveness of the method on 20,000 songs. Experiments show that compared with traditional methods, such as single audio, single lyric, and single audio with multimodal lyric methods, the method proposed in this paper has significantly improved the accuracy of emotion recognition, and the generalization ability has been significantly improved.

TRAINING PODCASTS IN THE TEACHING PROCESS UKRAINIAN AS A FOREIGN LANGUAGE

The article reveals the possibilities of using podcasting in the process of teaching foreign students of preparatory departments of the Ukrainian language as a foreign language, shows its didactic potential (resources Studyukrainian and Ukrainianlessons). The article analyzes the advantages and disadvantages of podcasting in the organization of the educational process and presents the types of work with podcasts. The modern educational process is inconceivable without the widespread use of such technology as “podcasting”. A podcast is a single audio or video file or a series of such files that are published at the same address on the Internet and have a certain frequency of publication, which depends on the creator of a particular podcast. One of the most productive platforms for online learning of Ukrainian as a foreign language is the Studyukrainian resource. The site already has 5 seasons of podcasts (about 160 recordings), and the resource is actively replenished. In the first season the podcasts are written in English, in the following seasons the share of English in the teacher’s explanations decreases significantly, in the fifth season the podcasts are written exclusively in Ukrainian. Each podcast lasts about 20 minutes. Foreign students have the opportunity to receive by e-mail a “script” in PDF format of the podcast and additional materials on the topic (grammar tables, explanations, examples, diagrams, a list of phrases and dialogues, exercises for more practice and vocabulary on the topic, etc.). In addition, the foreign student is given additional opportunities: sets of cards for learning words, videos of various songs and clips, which explain the Ukrainian realities or features of word usage (for example, national holidays, greetings etc.). The site provides foreign students with a dictionary, a list of phrases, a grammar block, a blog for communication with the teacher. In the process of learning the Ukrainian language as a foreign language, the fullest possible disclosure of the possibilities of podcasting technology is possible, because the language as a means of transmitting information in this context is also the object of study. The use of podcasts in teaching various activities helps to increase students’ motivation to learn the national language; the emergence of choice in the selection of interesting phonetic material; promotes the formation of listening skills and the ability to perceive different speech styles with different accents and intonations; improving speaking skills. Podcast as a media carrier of information is, along with other resources, an integral technical means of distance learning of Ukrainian as a foreign language, which allows to solve complex problems of foreign language education. Key words: podcasting, iPod, listening training, new information technologies in teaching Ukrainian as a foreign language, Studyukrainian and Ukrainianlessons resources.

Brain Computer Interface for Communication and Control of Peripherals and Appliances

The concept is designed to improve upon the recent developed system, utilizing auditory steady state response (ASSR) as a basis for the Brain Computer Interface (BCI) paradigm. It utilizes the classification of signals through a discrete wavelet transform (DWT) before the actual transmission to reduce overhead at the processing system. The electroencephalogram (EEG) obtained from the subject is through a p300 based EEG receivers. A compression algorithm is used to reduce the bandwidth usage and provide a quicker transmission of the large and continuous EEG. An Arduino board along with a proximity sensor is used to detect the presence and distance of the subject and consequently control playback of a single frequency audio signal, which as received by the user, is used for producing the EEG signals. A continuous focus of the user is required on the playback of the single frequency sound to produce a sizeable reading. At the receiving end, another Arduino board is installed with an SD card module, which contains the commands, responsible for the actual control of the devices. The concept can be utilized for various purposes from controlling IoT based systems to wheelchairs and hospital beds as well as bionic limbs, which however are limited due to the overall bulk of all the equipment currently required. The main aim of this paper is to propose an improvement in the transmission, reduction the latency of the signals and to provide a concept for utilization by the handicapped or physically impaired patients. Since the EEG is obtained through the inner ear of the subject, it completely eliminates any need for invasive surgery and provides a simplified solution. Developments have shown to be able to achieve over 95% of accuracy in the domain, currently limited by length of the EEG required in order to process the actual commands from the subject’s brain.

Detection of AMR double compression using compressed-domain speech features

The adaptive multi-rate (AMR) audio codec is an established standard of speech signal compression, for either transmitting speech signal over mobile networks or storing digital audio on handheld devices. The widespread use of AMR codec and the high availability of tampering software have increased authentication cases in court. AMR double compression detection is a challenging engineering problem and a topic of multimedia forensics. As a general rule, a double compressed AMR file cannot be considered an original file. In this paper, a new method based on support vector machine (SVM) is proposed to classify single and double compressed AMR digital audio. Instead of using the decoded speech waveform, the proposed method uses only compressed-domain speech features. Specific parameters are extracted from encoded AMR files and used to create a set of statistical features. After applying robust scaling to features and selecting the SVM model, recursive feature elimination with correlation bias reduction (RFE-CBR) is used to determine the best number of features to maximize accuracy in SVM classification. The experiments reveal that the proposed algorithm can discriminate single and double compressed AMR speech, outperforming the published methods. The average accuracies using TIMIT database and CARIOCA1 database, the last recorded from landline phone calls, are about 99%. Other experiments, including frame offset attack and noise addition, found that the method is robust and reliable.

Audio Signal Detection and Enhancement Based on Linear CMOS Array and Multi-Channel Data Fusion

An audio signal detection system based on laser speckle and multi-channel data fusion is presented. A linear CMOS array is used as the detector, which owns a fast line rate and suitable sensing size. The signals from the pixels are selected and fused to enhance the reconstructed signal. The reconstructed audio signals are evaluated with a segmental SNR (SegSNR) algorithm. The experimental results of three categories of audio sources (single voice audio, conversation and music) show that data fusion can improve the SegSNR scores. Especially, direct phase-error based filtering (pbf) fusion gives a nearly 3.0 dB increase and obtains another 1.0 dB increase with the combination of single channel process. The experimental results show that the fusion algorithms are not sensitive to audio types and the performance of multi-channel data fusion is not weakened with the increase of measuring distance. This feature has potential applications in remote sensing. The intelligibility of the fused audio signals is evaluated with normalized subband envelope correlation (NSEC) algorithm and the evaluation results shows that fusion can also enhance the intelligibility of the recovered signal.

Joint learning for voice based disease detection

Voice analysis provides a non-invasive way for disease detection, in which most methods only consider a single audio, although different audios contain complementary information and a fusion of them is beneficial. In this paper, a novel model JOLL4R (JOint Learning based on Label Relaxed low-Rank Ridge Regression) is proposed to fuse audios for voice based disease detection. First, the model couples the regression losses from two audios together to jointly learn a transformation matrix for each audio. Secondly, the conventional zero-one regression targets are relaxed by the ϵ-dragging technique so that the margins between different classes are enlarged. Third, low-rank constraint is imposed to exploit the correlation structure among different classes. The proposed algorithm not only enables to consider multiple audios, but also adjusts the weight of each audio adaptively. Due to the design of losses coupling, ϵ-dragging technique, and low rank constraint, high performance is achieved. Experiments conducted on two disease detection tasks, each with six types of fusion, show that our fusion approach outperforms the case of using a single audio and another two fusion methods. Finally, key factors in JOLL4R are analyzed.

Neural decoding of attentional selection in multi-speaker environments without access to clean sources

Objective. People who suffer from hearing impairments can find it difficult to follow a conversation in a multi-speaker environment. Current hearing aids can suppress background noise; however, there is little that can be done to help a user attend to a single conversation amongst many without knowing which speaker the user is attending to. Cognitively controlled hearing aids that use auditory attention decoding (AAD) methods are the next step in offering help. Translating the successes in AAD research to real-world applications poses a number of challenges, including the lack of access to the clean sound sources in the environment with which to compare with the neural signals. We propose a novel framework that combines single-channel speech separation algorithms with AAD. Approach. We present an end-to-end system that (1) receives a single audio channel containing a mixture of speakers that is heard by a listener along with the listener’s neural signals, (2) automatically separates the individual speakers in the mixture, (3) determines the attended speaker, and (4) amplifies the attended speaker’s voice to assist the listener. Main results. Using invasive electrophysiology recordings, we identified the regions of the auditory cortex that contribute to AAD. Given appropriate electrode locations, our system is able to decode the attention of subjects and amplify the attended speaker using only the mixed audio. Our quality assessment of the modified audio demonstrates a significant improvement in both subjective and objective speech quality measures. Significance. Our novel framework for AAD bridges the gap between the most recent advancements in speech processing technologies and speech prosthesis research and moves us closer to the development of cognitively controlled hearable devices for the hearing impaired.

Neural decoding of attentional selection in multi-speaker environments without access to separated sources.

People who suffer from hearing impairments can find it difficult to follow a conversation in a multi-speaker environment. Modern hearing aids can suppress background noise; however, there is little that can be done to help a user attend to a single conversation without knowing which speaker is being attended to. Cognitively controlled hearing aids that use auditory attention decoding (AAD) methods are the next step in offering help. A number of challenges exist, including the lack of access to the clean sound sources in the environment with which to compare with the neural signals. We propose a novel framework that combines single-channel speech separation algorithms with AAD. We present an end-to-end system that 1) receives a single audio channel containing a mixture of speakers that is heard by a listener along with the listener's neural signals, 2) automatically separates the individual speakers in the mixture, 3) determines the attended speaker, and 4) amplifies the attended speaker's voice to assist the listener. Using invasive electrophysiology recordings, our system is able to decode the attention of a subject and detect switches in attention using only the mixed audio. We also identified the regions of the auditory cortex that contribute to AAD. Our quality assessment of the modified audio demonstrates a significant improvement in both subjective and objective speech quality measures. Our novel framework for AAD bridges the gap between the most recent advancements in speech processing technologies and speech prosthesis research and moves us closer to the development of cognitively controlled hearing aids.

Sparse $\ell _{1}$-Optimal Multiloudspeaker Panning and Its Relation to Vector Base Amplitude Panning

Panning techniques, such as vector base amplitude panning (VBAP), are a widely used practical approach for spatial sound reproduction using multiple loudspeakers. Although limited to a relatively small listening area, they are very efficient and offer good localization accuracy, timbral quality, as well as a graceful degradation of quality outside the sweet spot. The aim of this paper is to investigate optimal sound reproduction techniques that adopt some of the advantageous properties of VBAP, such as the sparsity and the locality of the active loudspeakers for the reproduction of a single audio object. To this end, we state the task of multiloudspeaker panning as an ${\ell _{1}}$ optimization problem. We demonstrate and prove that the resulting solutions are exactly sparse. Moreover, we show the effect of adding a nonnegativity constraint on the loudspeaker gains in order to preserve the locality of the panning solution. Adding this constraint, ${\ell _{1}}$ -optimal panning can be formulated as a linear program. Using this representation, we prove that unique ${\ell _{1}}$ -optimal panning solutions incorporating a nonnegativity constraint are identical to VBAP using a Delaunay triangulation for the loudspeaker setup. Using results from linear programming and duality theory, we describe properties and special cases, such as solution ambiguity, of the VBAP solution.

A Novel Audio Aggregation Watermarking Algorithm Based on Copyright Protection

AbstractA novel algorithm of audio aggregation watermarking was proposed in this paper for copyright protection. The algorithm not only protects copyright of a single audio in audio aggregation but also protects copyright of the whole audio aggregation. The experimental results showed the watermark of aggregate audio can resist to some attacks, for example, deleting audio, and substituting audio; at the same time, watermark of every single audio is robust to some attacks such as low-filtering, resampling, requantization, noise addition and mp3 compression. The proposed algorithm has good imperceptibility, strong robustness and low complexity.

Reliability of an interactive sport-specific choice reaction time device

Methods Twelve recreationally trained males participated in Part a, which consisted of two visits (mean +/SD, 3.7 +/1.3 days); a familiarization testing day (V1a), followed by a subsequent testing day (V1b), and was conducted over a three week investigation period (28 +/5 yr, 178 +/9 cm, 79.15 +/15.7 kg, 17.5 +/6.6 % body fat). Part a was composed of nine choice reaction time testing protocols, including single step audio (CRA); single step visual (CRV); 15/30s single tower unidirectional [CRS(15s) (30s)]; 15/30s two tower lateral-directional [CRL(15s), (30s)]; 15/30s three tower multi-directional [CRM (15s), (30s)]; and a three tower, 2-minute stick hit test (stick hits). Seventeen recreationally trained males participated in Part b, which consisted of two visits (4.9 +/1.9 days) following a familiarization day (V1b and V2b), and was conducted over a two week investigational period (21.5 +/4.7 y, 181.1 +/6.1 cm, 85.2 +/17 kg, 14.5 +/11 % body fat). Part b comprised the same choice reaction time testing protocols as Part a. Part c consisted of a pooled mean of 62 tests taken from Part a and Part b, which examined data within choice reaction testing days between V1a, V2a, V1b, and V2b, except the 2-minute Stick Hits data. Results Mean (+/SD) time (seconds) values for Part a, Part b, and Part c were 0.87, 0.91 and 0.86 for Day/Trial 1 respectively, and 0.81, 0.89, and 0.85 for Day/Trial 2 which resulted in no significant differences from Day/Trial 1 to Day/Trial 2 for Part a, b, and c (p > 0.05). However, all times between testing days/trials decreased (a: -0.071 sec, b: -0.021 sec, c: -0.010). Differences in days from Part b (-0.02 sec) and Trials for Part c (-0.01 sec) resulted in similar findings, suggesting a familiarization session between testing days may result in similar reliability to that of within-day trials (p = 1.00). Two testing batteries showed a significant decrease in time between Day 1 and Day 2 after familiarization: CRL15 (Mean difference = -0.07, p = 0.036) and CRM30 (Mean difference = -0.05, p = 0.022).

Sonification as Sensemaking in Control Room Applications

Traditional analog alarms are nearly always exclusively binary, resulting in a single audio signal display sounding in order to inform users of a particular system parameter which has varied from within its predefined limits. During emergency situations, users can be faced with multiple simultaneous alarms, each relaying information concerning different, but usually related, system parameters. Using advanced sonification principles, achieved real-time with digital sound manipulation according to established human factors principles, new multi-parameter alarms can be created to better inform users of suboptimal conditions. These high-tech alarms also serve to help users maintain high levels of situational awareness, while aiding in sensemaking and decreasing stress caused by alarm flooding and cognitive overload.

USB 다중 신호 전송 시스템

최근 컴퓨터에서 DVI(Digital Visual Interface), 오디오, 비디오, USB(Universal Serial Bus), 랜(LAN) 등의 다양한 신호를 동시에 전송할 필요성이 요구됨에 따라 케이블이 복잡해지고 확장성 등의 문제점이 제기 되고 있다. 이에, 본 논문에서는 하나의 USB 케이블을 이용하여 컴퓨터의 영상, 음성, USB, 랜 등의 신호를 분배 할 수 있는 하나의 USB 다중 신호 전송 시스템을 설계 및 구현하였다. 따라서, USB 다중 신호 전송을 위해 단일 USB 신호를 DVI, 오디오, 다중 USB, 랜으로 변환하는 변환기 모듈들을 구현하였다. 이는 USB 방식으로 DVI 포트를 지원하며, 최대 1920 * 1080 해상도를 지원한다. USB 다중 신호 전송 시스템은 하나의 케이블로 다수의 신호를 전송하여 각종 케이블의 설치 비용이 절감되고, 화면 복제 기능을 이용하여 학교 및 학원 등에서 강의 편의 제공 및 컴퓨터 주변 포트의 확장성을 제공할 것이다. In recent years, the need to simultaneously transmit a variety of signals, such as DVI(Digital Visual Interface), audio, video, USB(Universal Serial Bus), LAN from the computer is required. So the cable complexity and scalability issues have been raised. In this paper, this signal can be distributed using a single USB cable, computer, video, audio, USB, LAN, one USB multi-signal transmission system was designed and implemented. USB multi-signal transmission was implemented in order to convert a single DVI, audio, and multiple USB, LAN, USB signal converter modules. This USB DVI port supports up to 1920 * 1080 resolution. USB multi-signal transmission system by sending multiple signals into a single cable installation costs of the various cable and using the replication feature of the screen, will provide schools and institutes, etc., providing the convenience of the river, and the scalability of computer peripheral ports.

Syllable Congruency of Audio-Visual Speech Stimuli Facilitates the Spatial Ventriloquism Only with Bilateral Visual Presentations

Spatial ventriloquism refers to a shift of perceptual location of a sound toward a synchronized visual stimulus. It has been assumed to reflect early processes uninfluenced by cognitive factors such as syllable congruency between audio-visual speech stimuli. Conventional experiments have examined compelling situations which typically entail pairs of single audio and visual stimuli to be bound. However, for natural environments our multisensory system is designed to select relevant sensory signals to be bound among adjacent stimuli. This selection process may depend upon higher (cognitive) mechanisms. We investigated whether a cognitive factor affects the size of the ventriloquism when an additional visual stimulus is presented with a conventional audio-visual pair. Participants were presented with a set of audio-visual speech stimuli, comprising one or two bilateral movies of a person uttering single syllables together with recordings of this person speaking the same syllables. One of movies and the speec...

Communication Device With Single Output Audio Transducer

A communication device 100 includes a first audio output 105; a second audio output 110; an audio transducer 115 for dual mode use; a shuttered acoustic path 120 coupled between the audio transducer 115 and the first audio output 105; and an attenuated acoustic path 125 coupled between the audio transducer 115 and the second audio output 110.

A Multichannel Sinusoidal Model Applied to Spot Microphone Signals for Immersive Audio

In this paper, a multichannel version of the sinusoids plus noise model (also known as deterministic plus stochastic decomposition) is proposed and applied to spot microphone signals of a music recording. These are the recordings captured by the various microphones placed in a venue, before the mixing process produces the final multichannel audio mix. Coding these microphone signals makes them available to the decoder, allowing for interactive audio reproduction which is a necessary component in immersive audio applications. The proposed model uses a single reference audio signal in order to derive a noise signal per spot microphone. This noise signal can significantly enhance the sinusoidal representation of the corresponding spot signal. The reference can be one of the spot signals or a downmix, depending on the application. Thus, for a collection of multiple spot signals, only the reference is fully encoded (e.g., as an MP3 monophonic signal). For the remaining spot signals, their sinusoidal parameters and corresponding noise spectral envelopes are retained and coded, resulting in bitrates for this side information in the order of 15 kb/s for perceptual performance above the 4.0 grade on the mean opinion score (MOS) scale.