Gradient Ascent Subjective Multimedia Quality Testing

Stephen Voran,Andrew Catellier

doi:10.1155/2011/472185

Abstract

Subjective testing is the most direct means of assessing multimedia quality as experienced by users. When multiple dimensions must be evaluated, these tests can become slow and costly. We present gradient ascent subjective testing (GAST) as an efficient way to locate optimizing sets of coding or transmission parameter values. GAST combines gradient ascent optimization techniques with subjective test trials. As a proof-of-concept, we used GAST to search a two-dimensional parameter space for the known region of maximal audio quality, using paired-comparison listening trials. That region was located accurately and much more efficiently than use of an exhaustive search. We also used GAST to search a two-dimensional quantizer design space for a point of maximal image quality, using side-by-side paired-comparison trials. The point of maximal image quality was efficiently located, and the corresponding quantizer shape and deadzone agree closely with the quantizer specifications for JPEG 2000, Part 1.

Highlights

Subjective testing is arguably the most basic and direct way to assess the user-perceived quality of image, video, audio, and multimedia presentations
We present gradient ascent subjective testing (GAST) as an efficient alternative to exhaustive search (ES) absolute category rating (ACR) testing
The starting points are randomly distributed across the search space, and the ending points are mostly clustered near the center of the search space

Summary

Introduction

Subjective testing is arguably the most basic and direct way to assess the user-perceived quality of image, video, audio, and multimedia presentations. It may be necessary to find an optimal partitioning of bits between different signal components in a multimedia program In each of these cases one is seeking a point in a multidimensional parameter space that produces maximal perceived quality. The adaptive psychometric testing method in [12] uses subject responses to modify stimulus levels so that they efficiently converge to the threshold of perception This is a powerful univariate threshold locating technique but it does not address multidimensional optimization.

Gradient Ascent Subjective Testing Algorithm

Objective function

GAST Experiments

Starting point

Discussion and Observations