Audio Quality Assessment Research Articles

A Computationally Efficient Model for Combined Assessment of Monaural and Binaural Audio Quality

A Computationally Efficient Model for Combined Assessment of Monaural and Binaural Audio Quality

Non-intrusive method for audio quality assessment of lossy-compressed music recordings using convolutional neural networks

Most of the existing algorithms for the objective audio quality assessment are intrusive, as they require access both to an unimpaired reference recording and an evaluated signal. This feature excludes them from many practical applications. In this paper, we introduce a non-intrusive audio quality assessment method. The proposed method is intended to account for audio artefacts arising from the lossy compression of music signals. During its development, 250 high-quality uncompressed music recordings were collated. They were subsequently processed using the selection of five popular audio codecs, resulting in the repository of 13,000 audio excerpts representing various levels of audio quality. The proposed non-intrusive method was trained with the data obtained employing a well-established intrusive model (ViSQOL v3). Next, the performance of the trained model was evaluated utilizing the quality scores obtained in the subjective listening tests undertaken remotely over the Internet. The listening tests were carried out in compliance with the MUSHRA recommendation (ITU-R BS.1534-3). In this study, the following three convolutional neural networks were compared: (1) a model employing 1D convolutional filters, (2) an Inception-based model, and (3) a VGG-based model. The last-mentioned model outperformed the model employing 1D convolutional filters in terms of predicting the scores from the listening tests, reaching a correlation value of 0.893. The performance of the Inceptionbased model was similar to that of the VGG-based model. Moreover, the VGG-based model outperformed the method employing a stacked gated-recurrent-unit-based deep learning framework, recently introduced by Mumtaz et al. (2022).

Towards Improved Objective Perceptual Audio Quality Assessment - Part 1: A Novel Data-Driven Cognitive Model

Towards Improved Objective Perceptual Audio Quality Assessment - Part 1: A Novel Data-Driven Cognitive Model

Nonintrusive Perceptual Audio Quality Assessment for User-Generated Content Using Deep Learning

With the boom of social media communication, teleconferencing, and online classes, audiovisual communication over bandwidth strained networks has become an integral part of our lives. Consequently, the growing demand for the quality of experience necessitates developing algorithms to measure and enrich user experience. Prior studies have mainly focused on assessing speech quality and intelligibility with reference to audio quality assessment, while other categories in user-generated multimedia (UGM) are less explored. Moreover, frequency-domain properties of speech and UGM audio are significantly different from each other. Furthermore, there is a lack of a standard dataset for the quality assessment of UGM. Considering these limitations, in this article, we first develop the IIT-JMU-UGM audio dataset consisting of 1150 audio clips, with diverse context, content, and types of degradation commonly observed in real-world scenarios and annotated with the subjective quality scores. Finally, we propose a non-intrusive audio quality assessment metric using a stacked gated-recurrent-unit-based deep learning framework. The proposed model outperforms several baseline methods, including state-of-the-art non-intrusive and intrusive approaches. The resulting Pearson’s correlation coefficient of 0.834 indicates that the proposed method efficiently mirrors human auditory perception.

A Benchmark of Non-intrusive Parametric Audio Quality Estimation Models for Broadcasting Systems and Web-casting Applications

Due to the rising usage of various broadcasting systems and web-casting applications, a measurement of audio quality has become an essential task. This paper presents a benchmark of the parametric models for non-intrusive estimation of the audio quality perceived by the end user. The proposed solution is based on machine learning techniques for broadcasting systems and web-casting applications. The main goal of this study is to assess the performance of the non-intrusive parametric models as well as to evaluate a statistical significance of the performance differences between those models. The paper provides a comparison of several models based on the Support Vector Regression, Genetic Programming, Multigene Symbolic Regression, Neural Networks and Random Forest. The obtained results indicate that among the investigated models the most accurate, although not the fastest ones, are the model based on Random Forest (a broadcast scenario) and the SVR-based model (a web-cast scenario). These models represent promising candidates for non-intrusive parametric audio quality assessment in the context of broadcasting systems and web-casting applications.

Study of Subjective and Objective Quality Assessment of Audio-Visual Signals.

The topics of visual and audio quality assessment (QA) have been widely researched for decades, yet nearly all of this prior work has focused only on single-mode visual or audio signals. However, visual signals rarely are presented without accompanying audio, including heavy-bandwidth video streaming applications. Moreover, the distortions that may separately (or conjointly) afflict the visual and audio signals collectively shape user-perceived quality of experience (QoE). This motivated us to conduct a subjective study of audio and video (A/V) quality, which we then used to compare and develop A/V quality measurement models and algorithms. The new LIVE-SJTU Audio and Video Quality Assessment (A/V-QA) Database includes 336 A/V sequences that were generated from 14 original source contents by applying 24 different A/V distortion combinations on them. We then conducted a subjective A/V quality perception study on the database towards attaining a better understanding of how humans perceive the overall combined quality of A/V signals. We also designed four different families of objective A/V quality prediction models, using a multimodal fusion strategy. The different types of A/V quality models differ in both the unimodal audio and video quality prediction models comprising the direct signal measurements and in the way that the two perceptual signal modes are combined. The objective models are built using both existing state-of-the-art audio and video quality prediction models and some new prediction models, as well as quality-predictive features delivered by a deep neural network. The methods of fusing audio and video quality predictions that are considered include simple product combinations as well as learned mappings. Using the new subjective A/V database as a tool, we validated and tested all of the objective A/V quality prediction models. We will make the database publicly available to facilitate further research.

A novel two-layer model for overall quality assessment of multichannel audio

With the development of multichannel audio systems, corresponding audio quality assessment techniques, especially the objective prediction models, have received increasing attention. Existing methods, such as PEAQ (Perceptual Evaluation of Audio Quality) recommended by ITU, usually lead to poor results when assessing multichannel audio, which have little correlation with subjective scores. In this paper, a novel two-layer model based on Multiple Linear Regression (MLR) and Neural Network (NN) is proposed. Through the first layer, two indicators of multichannel audio, Audio Quality Score (AQS) and Spatial Perception Score (SPS) are derived, and through the second layer the overall score is output. The final results show that this model can not only improve the correlation with the subjective test score by 30.7% and decrease the Root Mean Square Error (RMSE) by 44.6%, but also add two new indicators: AQS and SPS, which can help reflect the multichannel audio quality more clearly.

Headphone-based immersive audio for virtual reality headsets

In this paper, we propose a binaural system for virtual auditory space synthesis and reproduction. We introduce a subjective listening test for head-related transfer function (HRTF) customization and real-time implementation of ambisonic surround sound over headphones. In this test, a listener first selects the closest-matching HRTF dataset from an existing database. The selected HRTF dataset must fulfill the criteria of front-back discrimination and up-down discrimination. Secondly, the selected HRTF dataset is utilized to build a binaural ambisonic system composed of an encoder for synthesizing multiple sound sources, an image-source model for listening-room simulation, an ambisonic rotator responding to head motion in three axes, and a binaural ambisonic decoder. Additionally, we propose a system optimization technique that minimizes computational cost and enhances audio quality. Finally, we present both subjective and objective measurements of the perceived audio quality that were conducted to validate this approach. The audio quality assessment examined the frequency hearing impairment, localization error, and localization blur. The proposed system is especially suitable for mixing and rendering immersive sound in virtual reality.

A user-centered taxonomy of factors contributing to the listener experience of reproduced audio

The traditional paradigm for the assessment of audio quality is that of a listener positioned in the geometric center of a standardized loudspeaker setup, fully attending to the reproduced sound scene. However, this is not how listeners generally interact with audio technology. Audio is consumed in a variety of environments and situations, over devices with varying quality, and by listeners with different expectations and needs. Drawing on research from soundscapes, human computer interaction, and multimedia quality of experience, this paper proposes a user-centered taxonomy of factors that influence the listener experience of reproduced audio. The taxonomy is supported by data from recent research into the perception of complex reproduced sound scenes, and by new data from a web-based survey investigating the structure of experiences with audio technology. In this survey, participants were asked to consider previous experiences with audio technology, and data were collected on the experience itself (psychological need fulfillment and affect), perceptual attributes related to the reproduced audio, and the importance of audio quality to the experience. Results point toward a model of listener experience that can be used to profile listener experiences in different contexts and can be used as a measurement tool in future controlled experiments.

Subjective and Objective Assessment of Perceived Audio Quality of Current Digital Audio Broadcasting Systems and Web-Casting Applications

This paper investigates the impact of different audio codecs typically deployed in current digital audio broadcasting (DAB) systems and web-casting applications, which represent a main source of quality impairment in these systems and applications, on the quality perceived by the end user. Both subjective and objective assessments are used. Two different audio quality prediction models, namely Perceptual Evaluation of Audio Quality (PEAQ) and Perceptual Objective Listening Quality Assessment (POLQA) Music, are evaluated by comparing the predictions with subjectively obtained grades. The results show that the degradations introduced by the typical lossy audio codecs deployed in current DAB systems and web-casting applications operating at the lowest bit rate typically used in these distribution systems and applications seriously impact the subjective audio quality perceived by the end user. Furthermore, it is shown that a retrained POLQA Music provides the best overall correlations between predicted objective measurements and subjective scores allowing to predict the final perceived quality with good accuracy when scores are averaged over a small set of musical fragments (R = 0.95).

Categorization of Sound Attributes for Audio Quality Assessment—A Lexical Study

For the evaluation of perceived quality in audio coding, two well-known subjective test methods, both of which are based on Basic Audio Quality (BAQ), are recommended by the International Telecommunication Union. Although a predictor of quality, BAQ is likely to be multidimensional. Listening tests can be used to evaluate other attributes that contribute to impairments created by coding. The goal of this study is to define categories of additional attributes, thereby providing a complement to the single BAQ metric. When quality attributes are sorted, there appears to be three groups: one related to space, a second related to defects, and a third split into timbre and quality.

Identifying Compression History of Wave Audio and Its Applications

Audio signal is sometimes stored and/or processed in WAV (waveform) format without any knowledge of its previous compression operations. To perform some subsequent processing, such as digital audio forensics, audio enhancement and blind audio quality assessment, it is necessary to identify its compression history. In this article, we will investigate how to identify a decompressed wave audio that went through one of three popular compression schemes, including MP3, WMA (windows media audio) and AAC (advanced audio coding). By analyzing the corresponding frequency coefficients, including modified discrete cosine transform (MDCT) and Mel-frequency cepstral coefficients (MFCCs), of those original audio clips and their decompressed versions with different compression schemes and bit rates, we propose several statistics to identify the compression scheme as well as the corresponding bit rate previously used for a given WAV signal. The experimental results evaluated on 8,800 audio clips with various contents have shown the effectiveness of the proposed method. In addition, some potential applications of the proposed method are discussed.

Subjective quality assessment of multichannel audio accompanied with video in representative broadcasting genres

Immersive broadcasting applications have received a lot of attention in the last years. In this context, the development of advanced HDTV and 3DTV formats is being successfully adopted by the consumer market, having a strong impact in the way that traditional broadcasting contents are displayed to final users. Together with the above advances in video technology, multichannel spatial audio has also experienced a considerable impulse within the audiovisual industry. However, the need for specific production tools and loudspeaker setups corresponding to multiple competing audio formats seems to be an important factor affecting their adoption by the consumer community. Moreover, it is well-known that the perceived audio quality is highly influenced by the reproduction context, where the existing multimodal interaction between audio and video plays a very important role. This paper presents a formal evaluation of the perceived sound quality provided by several spatial audio formats accompanied with video in the context of television broadcasting. Stereo, advanced surround formats and 3D Binaural sound are evaluated considering a set of representative broadcasting contents (sports, movies, music and animation) to assess their impact on the perceptual attributes contemplated within the international recommendations.

A blind digital audio watermarking scheme based on EMD and UISA techniques

In this paper, a new perceptual spread spectrum audio watermarking scheme is discussed. The watermark embedding process is performed in the Empirical Mode Decomposition (EMD) domain, and the hybrid watermark extraction process is based on the combination of EMD and ISA (Independent Subspace Analysis) techniques, followed by the generic detection system, i.e. inverse perceptual filter, predictor filter and correlation based detector. Since the EMD decomposes the audio signal into several oscillating components---the intrinsic mode functions (IMF)---the watermark information can be inserted in more than one IMF, using spread spectrum modulation, allowing hence the increase of the insertion capacity. The imperceptibility of the inserted data is ensured by the use of a psychoacoustical model. The blind extraction of the watermark signal, from the received watermarked audio, consists in the separation of the watermark from the IMFs of the received audio signal. The separation is achieved by a new proposed under-determined ISA method, here referred to as UISA. The proposed hybrid watermarking system was applied to the SQAM (Sound Quality Assessment Material) audio database (Available at http://sound.media.mit.edu/mpeg4/audio/sqam/ ) and proved to have efficient detection performances in terms of Bit Error Rate (BER) compared to a generic perceptual spread spectrum watermarking system. The perceptual quality of the watermarked audio was objectively assessed using the PEMO-Q (Tool for objective perceptual assessment of audio quality) algorithm. Also, using our technique, we can extract the different watermarks without using any information of original signal or the inserted watermark. Experimental results exhibit that the transparency and high robustness of the watermarked audio can be achieved simultaneously with a substantial increase of the amount of information transmitted. A reliability of 1.8 10Â?Â?Â?4 (against 1.5 10Â?Â?Â?2 for the generic system), for a bit rate of 400 bits/s, can be achieved when the channel is not disturbed.

Perceptual-based quality assessment for audio–visual services: A survey

Accurate measurement of the perceived quality of audio–visual services at the end-user is becoming a crucial issue in digital applications due to the growing demand for compression and transmission of audio–visual services over communication networks. Content providers strive to offer the best quality of experience for customers linked to their different quality of service (QoS) solutions. Therefore, developing accurate, perceptual-based quality metrics is a key requirement in multimedia services. In this paper, we survey state-of-the-art signal-driven perceptual audio and video quality assessment methods independently, and investigate relevant issues in developing joint audio–visual quality metrics. Experiments with respect to subjective quality results have been conducted for analyzing and comparing the performance of the quality metrics. We consider emerging trends in audio–visual quality assessment, and propose feasible solutions for future work in perceptual-based audio–visual quality metrics.

Standardization activities in the ITU for a QoE assessment of IPTV

This article gives an overview of the state of the art of objective quality assessment of audio and visual media and its standardization activities in the ITU. IPTV services are becoming one of the most promising applications over next generation networks. To provide end users with comfortable, stable, and economical services, QoE assessment methodologies for quality design and management are indispensable.

Objective Assessment of Speech and Audio Quality—Technology and Applications

In the past few years, objective quality assessment models have become increasingly used for assessing or monitoring speech and audio quality. By measuring perceived quality on an easily-understood subjective scale, such as listening quality (excellent, good, fair, poor, bad), these methods provide a quick and repeatable way to estimate customer experience. Typical applications include audio quality evaluation, selection of codecs or other equipment, and measuring the quality of telephone networks. To introduce this special issue, this paper provides an overview of the field, outlining the main approaches to intrusive, nonintrusive and parametric models and discussing some of their limitations and areas of future work

PEMO-Q—A New Method for Objective Audio Quality Assessment Using a Model of Auditory Perception

A new method for the objective assessment and prediction of perceived audio quality is introduced. It represents an expansion of the speech quality measure q <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">C</sub> , introduced by Hansen and Kollmeier, and is based on a psychoacoustically validated, quantitative model of the "effective" peripheral auditory processing by Dau et al. To evaluate the audio quality of a given distorted signal relative to a corresponding high-quality reference signal, the auditory model is employed to compute "internal representations" of the signals, which are partly assimilated in order to account for assumed cognitive aspects. The linear cross correlation coefficient of the assimilated internal representations represents the perceptual similarity measure (PSM). PSM shows good correlations with subjective quality ratings if different types of audio signals are considered separately, whereas a better accuracy of signal-independent quality prediction is achieved by a second quality measure PSM <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">t</sub> represented by the fifth percentile of the sequence of instantaneous audio quality PSM(t). The new measures were evaluated using a large database of subjective listening tests that were originally carried out on behalf of the International Telecommunication Union (ITU) and Moving Pictures Experts Group (MPEG) for the evaluation of various low bit-rate audio codecs. Additional tests with data unknown in the development phase of the model were carried out. Except for linear distortions, the new method shows a higher prediction accuracy than the ITU-R recommendation BS.1387 ("PEAQ") for the tested data

Introduction to the Special Section on Objective Quality Assessment of Speech and Audio

Introduction to the Special Section on Objective Quality Assessment of Speech and Audio

Special issue on objective quality assessment of speech and audio

Special issue on objective quality assessment of speech and audio