Audio Quality Research Articles

Construction and validation of an audio-visual tool to instruct caregivers of patients with pemphigus in photographing skin and oral cavity lesions during tele-consultation: A cross-sectional study.

Background Owing to the lack of standardised audio-visual (A-V) instructions to take photographs, patients with pemphigus faced difficulties during tele-consultation in COVID-19 pandemic. Objective To construct and validate an A-V instruction tool to take photographs of skin and oral cavity lesions of pemphigus. Methods In this observational study, we included patients with pemphigus of either gender seeking tele-consultation, aged 18 years or older. A-V instructions demonstrating skin and oral cavity photography were designed and shared with the patients. They were requested to send pictures of lesions that complied with the instructions. They were then required to complete a 10-item questionnaire for face validation in the two following rounds. The videos were content validated by 14 experts in the field of clinical dermatology. Results Forty-eight patients took part in face validation. A majority of patients, 47 (97.9%) and 45 (93.8%); rated the audio and video quality as being above average, respectively. Forty-seven patients (97.9%) said the instructional videos were useful, and 42 patients (87.5%) said they did not need to take any further images to show how severe their disease was. The average scale content validity index for the instructions on skin imaging and the oral cavity imaging during round 1 of content validation was 0.863 and 0.836, respectively. Limitation Validated instruction videos are in Hindi language and need to be further translated and validated in other local languages for use in non-Hindi speaking regions. Conclusion A-V instructions were useful to take photographs during tele-consultation.

Beyond Words

Beyond Words

PEER-TO-PEER AUDIO STREAMING USING THE TOR NETWORK: PROBLEMS, OPPORTUNITIES AND DEVELOPMENT PROSPECTS

This article analyses the potential and challenges associated with the implementation of audio streaming in the anonymous Tor network and the use of peer-to-peer (P2P) technologies. The main focus is on discussing the technical and conceptual issues that arise when integrating audio streaming services with the Tor network, including delays, limited bandwidth and other limitations that affect audio quality. The aim of the study is to find practical solutions to the problem of anonymous audio streaming using Tor technologies or other means to preserve anonymity. By analysing different approaches and tools, it becomes apparent that peer-to-peer broadcasting offers a flexible solution that allows audio to be transmitted over P2P networks, reducing the load on individual nodes and optimising bandwidth usage. This solution offers opportunities for scalability, fault tolerance, bandwidth optimisation and latency reduction, making it an attractive option for audio streaming setups. Meanwhile, the Utopia ecosystem, with its decentralised structure and high level of privacy protection, is another potential solution. Not only does it allow bypassing censorship and firewalls, but it also ensures anonymity in communication and data exchange. With its built-in audio and video communication tools, as well as the ability to create private channels and share files, Utopia can become a reliable platform for streaming audio. Thus, based on the analysis, the solution to the problem of anonymous audio streaming may lie in the integration of technologies such as Peercasting and Utopia, which provide not only the technical capability for streaming, but also a high level of privacy protection for users. It is also important to consider the flexibility of these solutions and the ability to adapt to the specifics of anonymous networks and user requirements. The practical significance of the study of audio streaming over Tor using P2P technologies is to develop methods to increase the anonymity and security of audio communications, which is critical in the face of growing threats to privacy on the Internet. This allows to overcome censorship and Internet restrictions in repressive regimes, providing users with freedom of thought and access to information.

The Style Transformation of Gu-Zheng Music Based on the Improved Model of Cyclegan

In recent years, generative adversarial networks have excelled in the field of image style migration, however their performance in the field of music has been mediocre. Existing music style migration does not work well for style migration of gu-zheng music. In order to solve these problems, we first extract the features of gu-zheng music and the Mel-spectrum features, then use CycleGAN to do style transformation on the combined features and Mel-spectrum features, and then use WaveNet vocoder to decode the migrated spectrograms, and finally achieve the style migration with gu-zheng music. The proposed model was evaluated on the publicly available dataset FMA, and the average style migration rate of the compliant music reached 94.07%. Compared to other algorithms, the music produced by this method outperformed other algorithms in terms of style migration rate and audio quality.

Learning Audio and Video Bitrate Selection Strategies via Explicit Requirements

Mobile video streaming dominates today's network traffic, and adaptive bitrate (ABR) algorithms have been routinely adopted for transmitting media content across dynamic mobile networks. State-of-the-art ABR algorithms mainly alter video bitrate without considering audio bitrate as they consider the impact on the video negligible due to their small size. However, to bring users an immersive experience, recent content providers have applied high-quality audio with large sizes, like stereophonic sound. Therefore, improper audio bitrate selection will adversely affect video bitrate selection, leading to undesirable audio/video combinations (the highest video quality with the lowest audio quality, and vice versa) and frequent playback interruptions. To address these inefficiencies, we propose a Self-Play reinforcement learning-based Audio-aware ABR algorithm named SPA to learn strategies for audio and video bitrate selections. By learning from explicit goals, SPA can match the actual requirements and attain good performance. By conducting trace-driven and testbed-based experiments, we observe SPA's considerable superiority compared to existing approaches, including reducing the undesirable combinations by up to 34.17× and achieving zero stall time across 88.57% of traces. We also invite 35 volunteers to join a subjective test, and the result shows that 33/35 people consider SPA provides them with a satisfactory viewing experience.

Imperceptible and Reversible Acoustic Watermarking Based on Modified Integer Discrete Cosine Transform Coefficient Expansion

This paper aims to explore an alternative reversible digital watermarking solution to guarantee the integrity of and detect tampering with data of probative importance. Since the payload for verification is embedded in the contents, algorithms for reversible embedding and extraction, imperceptibility, payload capacity, and computational time are issues to evaluate. Thus, we propose a reversible and imperceptible audio information-hiding algorithm based on modified integer discrete cosine transform (intDCT) coefficient expansion. In this work, the original signal is segmented into fixed-length frames, and then intDCT is applied to each frame to transform signals from the time domain into integer DCT coefficients. Expansion is applied to DCT coefficients at a higher frequency to reserve hiding capacity. Objective evaluation of speech quality is conducted using listening quality objective mean opinion (MOS-LQO) and the segmental signal-to-noise ratio (segSNR). The audio quality of different frame lengths and capacities is evaluated. Averages of 4.41 for MOS-LQO and 23.314 [dB] for segSNR for 112 ITU-T test signals were obtained with a capacity of 8000 bps, which assured imperceptibility with the sufficient capacity of the proposed method. This shows comparable audio quality to conventional work based on Linear Predictive Coding (LPC) regarding MOS-LQO. However, all segSNR scores of the proposed method have comparable or better performance in the time domain. Additionally, comparing histograms of the normalized maximum absolute value of stego data shows a lower possibility of overflow than the LPC method. A computational cost, including hiding and transforming, is an average of 4.884 s to process a 10 s audio clip. Blind tampering detection without the original data is achieved by the proposed embedding and extraction method.

StyleSinger: Style Transfer for Out-of-Domain Singing Voice Synthesis

Style transfer for out-of-domain (OOD) singing voice synthesis (SVS) focuses on generating high-quality singing voices with unseen styles (such as timbre, emotion, pronunciation, and articulation skills) derived from reference singing voice samples. However, the endeavor to model the intricate nuances of singing voice styles is an arduous task, as singing voices possess a remarkable degree of expressiveness. Moreover, existing SVS methods encounter a decline in the quality of synthesized singing voices in OOD scenarios, as they rest upon the assumption that the target vocal attributes are discernible during the training phase. To overcome these challenges, we propose StyleSinger, the first singing voice synthesis model for zero-shot style transfer of out-of-domain reference singing voice samples. StyleSinger incorporates two critical approaches for enhanced effectiveness: 1) the Residual Style Adaptor (RSA) which employs a residual quantization module to capture diverse style characteristics in singing voices, and 2) the Uncertainty Modeling Layer Normalization (UMLN) to perturb the style attributes within the content representation during the training phase and thus improve the model generalization. Our extensive evaluations in zero-shot style transfer undeniably establish that StyleSinger outperforms baseline models in both audio quality and similarity to the reference singing voice samples. Access to singing voice samples can be found at https://stylesinger.github.io/.

V2Meow: Meowing to the Visual Beat via Video-to-Music Generation

Video-to-music generation demands both a temporally localized high-quality listening experience and globally aligned video-acoustic signatures. While recent music generation models excel at the former through advanced audio codecs, the exploration of video-acoustic signatures has been confined to specific visual scenarios. In contrast, our research confronts the challenge of learning globally aligned signatures between video and music directly from paired music and videos, without explicitly modeling domain-specific rhythmic or semantic relationships. We propose V2Meow, a video-to-music generation system capable of producing high-quality music audio for a diverse range of video input types using a multi-stage autoregressive model. Trained on 5k hours of music audio clips paired with video frames mined from in-the-wild music videos, V2Meow is competitive with previous domain-specific models when evaluated in a zero-shot manner. It synthesizes high-fidelity music audio waveforms solely by conditioning on pre-trained general-purpose visual features extracted from video frames, with optional style control via text prompts. Through both qualitative and quantitative evaluations, we demonstrate that our model outperforms various existing music generation systems in terms of visual-audio correspondence and audio quality. Music samples are available at tinyurl.com/v2meow.

UniCATS: A Unified Context-Aware Text-to-Speech Framework with Contextual VQ-Diffusion and Vocoding

The utilization of discrete speech tokens, divided into semantic tokens and acoustic tokens, has been proven superior to traditional acoustic feature mel-spectrograms in terms of naturalness and robustness for text-to-speech (TTS) synthesis. Recent popular models, such as VALL-E and SPEAR-TTS, allow zero-shot speaker adaptation through auto-regressive (AR) continuation of acoustic tokens extracted from a short speech prompt. However, these AR models are restricted to generate speech only in a left-to-right direction, making them unsuitable for speech editing where both preceding and following contexts are provided. Furthermore, these models rely on acoustic tokens, which have audio quality limitations imposed by the performance of audio codec models. In this study, we propose a unified context-aware TTS framework called UniCATS, which is capable of both speech continuation and editing. UniCATS comprises two components, an acoustic model CTX-txt2vec and a vocoder CTX-vec2wav. CTX-txt2vec employs contextual VQ-diffusion to predict semantic tokens from the input text, enabling it to incorporate the semantic context and maintain seamless concatenation with the surrounding context. Following that, CTX-vec2wav utilizes contextual vocoding to convert these semantic tokens into waveforms, taking into consideration the acoustic context. Our experimental results demonstrate that CTX-vec2wav outperforms HifiGAN and AudioLM in terms of speech resynthesis from semantic tokens. Moreover, we show that UniCATS achieves state-of-the-art performance in both speech continuation and editing. Audio samples are available at https://cpdu.github.io/unicats.

TrumpetNet: A Convolutional Neural Network with Self-Attention Mechanisms for visual detection of trumpet fingering

Visual detection of fingering on the trumpet is an increasingly interesting topic in music research. The ability to recognize and track the movements of the trumpet player’s fingers during the performance of a musical piece can provide valuable information for analyzing and improving instrument technique. However, this is a largely unexplored task, as most works focus on audio quality rather than instrument fingering techniques. Developing techniques for identifying essential finger positions on a musical instrument is crucial, as poor fingering techniques can harm instrument performance. In this work, we propose the visual detection of this fingering using convolutional neural networks with a proprietary dataset created for this purpose. Additionally, to improve the results and focus on the essential parts of the instrument, we use self-attention mechanisms by extracting these features automatically.

FACTORS AFFECTING EFL STUDENTS’ ORAL TRANSLATION EXAM PERFORMANCE: A CASE IN SOUTHERN VIETNAM

Despite the rapid development of technology and digital applications, interpreters play an indispensable role in assisting effective communication and connecting speakers from different cultures. To fulfil the global demand, more oral translation programs and courses are offered in higher education institutions. While there has been more attention to process-based teaching and learning, little has been done to examine what affects students’ performance during summative assessment. This present study explores (1) EFL students' perceptions of the factors influencing their oral translation exam performance, (2) strategies students use and expectations on lecturers to enhance interpreting competence. This study employed a mixed approach. Online questionnaire and interviews were used to collect data from 72 English majors at a university in the south of Vietnam. The results revealed that audio quality, testing-taking anxiety, memory, vocabulary shortage, and listening skills had a greater impact on students' oral translation exam performance than other factors. Students have various strategies and expectations of lecturers concerning a positive testing environment.

Application of Convolutional Neural Network for Decoding of 12-Lead Electrocardiogram from a Frequency-Modulated Audio Stream (Sonified ECG).

Research of novel biosignal modalities with application to remote patient monitoring is a subject of state-of-the-art developments. This study is focused on sonified ECG modality, which can be transmitted as an acoustic wave and received by GSM (Global System for Mobile Communications) microphones. Thus, the wireless connection between the patient module and the cloud server can be provided over an audio channel, such as a standard telephone call or audio message. Patients, especially the elderly or visually impaired, can benefit from ECG sonification because the wireless interface is readily available, facilitating the communication and transmission of secure ECG data from the patient monitoring device to the remote server. The aim of this study is to develop an AI-driven algorithm for 12-lead ECG sonification to support diagnostic reliability in the signal processing chain of the audio ECG stream. Our methods present the design of two algorithms: (1) a transformer (ECG-to-Audio) based on the frequency modulation (FM) of eight independent ECG leads in the very low frequency band (300-2700 Hz); and (2) a transformer (Audio-to-ECG) based on a four-layer 1D convolutional neural network (CNN) to decode the audio ECG stream (10 s @ 11 kHz) to the original eight-lead ECG (10 s @ 250 Hz). The CNN model is trained in unsupervised regression mode, searching for the minimum error between the transformed and original ECG signals. The results are reported using the PTB-XL 12-lead ECG database (21,837 recordings), split 50:50 for training and test. The quality of FM-modulated ECG audio is monitored by short-time Fourier transform, and examples are illustrated in this paper and supplementary audio files. The errors of the reconstructed ECG are estimated by a popular ECG diagnostic toolbox. They are substantially low in all ECG leads: amplitude error (quartile range RMSE = 3-7 μV, PRD = 2-5.2%), QRS detector (Se, PPV > 99.7%), P-QRS-T fiducial points' time deviation (<2 ms). Low errors generalized across diverse patients and arrhythmias are a testament to the efficacy of the developments. They support 12-lead ECG sonification as a wireless interface to provide reliable data for diagnostic measurements by automated tools or medical experts.

Broadcast Media Sensitization on Violence Against Women and Children in Port Harcourt

The study examined broadcast media sensitization on violence against women and children in Port Harcourt. The broadcast media have been sensitizing the members of audience on violence against women and children that happens in form of rape, domestic violence, among others. The social cognitive theory and aggressive cues media violence theory were adopted for the study. Three objectives which include to determine the frequency of broadcast media sensitization on violence against women and children in Port Harcourt and to examine the perception of residents of Port Harcourt towards broadcast media sensitization on violence against women and children and three research questions were adopted. Research methodology used was the survey design with the Taro Yamane formula used in deriving the sample size of 400 from the population size of 1,148,665. The convenience sampling technique was used and questionnaire was the instrument for data collection. Simple frequency distribution tables and percentages were used in the interpretation of data. Findings revealed that reports and programmes on violence against women and children in Port Harcourt are not broadcast regularly. Findings also revealed that residents of Port Harcourt find reports on violence against women and children to be scary, unreal and far. In conclusion, the broadcast media -radio and tv are great tools to influence sensitization because of their audio and audio-visual qualities. The study recommended that members of the audience will appreciate the adequacy of sensitization on violence against women and children when perpetrators are made to face the full weight of the law to serve as deterrent to others.

Muddy, muddled, or muffled? Understanding the perception of audio quality in music by hearing aid users.

Previous work on audio quality evaluation has demonstrated a developing convergence of the key perceptual attributes underlying judgments of quality, such as timbral, spatial and technical attributes. However, across existing research there remains a limited understanding of the crucial perceptual attributes that inform audio quality evaluation for people with hearing loss, and those who use hearing aids. This is especially the case with music, given the unique problems it presents in contrast to human speech. This paper presents a sensory evaluation study utilising descriptive analysis methods, in which a panel of hearing aid users collaborated, through consensus, to identify the most important perceptual attributes of music audio quality and developed a series of rating scales for future listening tests. Participants (N = 12), with a hearing loss ranging from mild to severe, first completed an online elicitation task, providing single-word terms to describe the audio quality of original and processed music samples; this was completed twice by each participant, once with hearing aids, and once without. Participants were then guided in discussing these raw terms across three focus groups, in which they reduced the term space, identified important perceptual groupings of terms, and developed perceptual attributes from these groups (including rating scales and definitions for each). Findings show that there were seven key perceptual dimensions underlying music audio quality (clarity, harshness, distortion, spaciousness, treble strength, middle strength, and bass strength), alongside a music audio quality attribute and possible alternative frequency balance attributes. We outline how these perceptual attributes align with extant literature, how attribute rating instruments might be used in future work, and the importance of better understanding the music listening difficulties of people with varied profiles of hearing loss.

Echo cancellation using Artificial Intelligence technique

To improve the audio quality in telecommunication, echo cancellation was proposed to prevent signals from re-appearing. Generally speaking, the echo canceler first recognizes the original transmitted signal that appears again due to delay in the transmitted or recieved signal. Then, the echo canceler subtracts the echo in the signals. Commonly, echo cancelers are calledacoustic echo cancellation (AEC) in general, whether they aim for acoustic echo, line echo, or both. However, in modern times, new problems have been emerging which require more innovative, more advanced methods to tackle the echo issue. Thus, the paper reviews methods regarding Artificial Intelligence (AI) to improve the current means for echo cancellation. In this paper, we mainly discuss the approaches to echo cancellation, the identification of echo using adaptive AI and the usage of adaptive filters and neural network for echo cancellation. For instance, deep learning approaches can facilitate the issues of AEC (active echo control) and ANC (active noise control),the optimization of LMS algorithm can be achieved by combining Swarm Intelligence (SI), combining adaptive digital filter and recurrent neural network for acoustic echo cancellation etc. Using AI in echo cancellation can help us achieve high-quality, full-duplex telecommunication in modern telephony system.

Speech Inpainting Based on Multi-Layer Long Short-Term Memory Networks

Audio inpainting plays an important role in addressing incomplete, damaged, or missing audio signals, contributing to improved quality of service and overall user experience in multimedia communications over the Internet and mobile networks. This paper presents an innovative solution for speech inpainting using Long Short-Term Memory (LSTM) networks, i.e., a restoring task where the missing parts of speech signals are recovered from the previous information in the time domain. The lost or corrupted speech signals are also referred to as gaps. We regard the speech inpainting task as a time-series prediction problem in this research work. To address this problem, we designed multi-layer LSTM networks and trained them on different speech datasets. Our study aims to investigate the inpainting performance of the proposed models on different datasets and with varying LSTM layers and explore the effect of multi-layer LSTM networks on the prediction of speech samples in terms of perceived audio quality. The inpainted speech quality is evaluated through the Mean Opinion Score (MOS) and a frequency analysis of the spectrogram. Our proposed multi-layer LSTM models are able to restore up to 1 s of gaps with high perceptual audio quality using the features captured from the time domain only. Specifically, for gap lengths under 500 ms, the MOS can reach up to 3~4, and for gap lengths ranging between 500 ms and 1 s, the MOS can reach up to 2~3. In the time domain, the proposed models can proficiently restore the envelope and trend of lost speech signals. In the frequency domain, the proposed models can restore spectrogram blocks with higher similarity to the original signals at frequencies less than 2.0 kHz and comparatively lower similarity at frequencies in the range of 2.0 kHz~8.0 kHz.

Research on sound signal filtering and processing delay based on multiple receivers

With the rapid development of modern communication and audio technology, sound signal processing has become more and more important. This paper deeply explores the potential of multi-receiver audio signal processing technology based on practical application scenarios, and studies the effect of beamforming technology and adaptive noise cancellation algorithm on audio signal quality improvement. Experimental results show that with proper technology selection and framework design, the audio signal processing effect in complex environments can be significantly improved. In addition, this paper also predicts the trend of signal filtering and processing in the future and puts forward suggestions for the application of deep learning in this field, the research of adaptive algorithms, the fusion of multi-sensor information, the optimization of computational efficiency, and the establishment of real scene simulation. Overall, sound signal processing has great potential and opportunities in practical applications, which are worthy of further research and exploration.

MFCC-based Audio Quality Classification Method for Forensics

MFCC-based Audio Quality Classification Method for Forensics

Auto Annotation of Linguistic Features for Audio Deepfake Discernment

We present an innovative approach to auto-annotate Expert Defined Linguistic Features (EDLFs) as subsequences in audio time series to improve audio deepfake discernment. In our prior work, these linguistic features – namely pitch, pause, breath, consonant release bursts, and overall audio quality, labeled by experts on the entire audio signal – have been shown to improve detection of audio deepfakes with AI algorithms. We now expand our approach to pilot a way to auto annotate subsequences in the time series that correspond to each EDLF. We developed an ensemble of discords, i.e. anomalies in time series, detected using matrix profiles across multiple discord lengths to identify multiple types of EDLFs. Working closely with linguistic experts, we evaluated where discords overlapped with EDLFs in the audio signal data. Our ensemble method to detect discords across multiple discord lengths achieves much higher accuracy than using individual discord lengths to detect EDLFs. With this approach and domain validation we establish the feasibility of using time series subsequences to capture EDLFs to supplement annotation by domain experts, for improved audio deepfake detection.

Analyzing the Influence of Diverse Background Noises on Voice Transmission: A Deep Learning Approach to Noise Suppression

This paper presents an approach to enhancing the clarity and intelligibility of speech in digital communications compromised by various background noises. Utilizing deep learning techniques, specifically a Variational Autoencoder (VAE) with 2D convolutional filters, we aim to suppress background noise in audio signals. Our method focuses on four simulated environmental noise scenarios: storms, wind, traffic, and aircraft. The training dataset has been obtained from public sources (TED-LIUM 3 dataset, which includes audio recordings from the popular TED-TALK series) combined with these background noises. The audio signals were transformed into 2D power spectrograms, upon which our VAE model was trained to filter out the noise and reconstruct clean audio. Our results demonstrate that the model outperforms existing state-of-the-art solutions in noise suppression. Although differences in noise types were observed, it was challenging to definitively conclude which background noise most adversely affects speech quality. The results have been assessed with objective (mathematical metrics) and subjective (listening to a set of audios by humans) methods. Notably, wind noise showed the smallest deviation between the noisy and cleaned audio, perceived subjectively as the most improved scenario. Future work should involve refining the phase calculation of the cleaned audio and creating a more balanced dataset to minimize differences in audio quality across scenarios. Additionally, practical applications of the model in real-time streaming audio are envisaged. This research contributes significantly to the field of audio signal processing by offering a deep learning solution tailored to various noise conditions, enhancing digital communication quality.