The Axiomatic Usability Evaluation Method

Abstract

This chapter introduces a new usability evaluation method, the axiomatic evaluation method, which is developed based on the axiomatic design theory – a formalized design methodology that can be used to solve a variety of design problems. This new evaluation method examines three domains of a product: customer domain, functional domain, and control domain. This method investigates not only usability problems reported by the users, but also usability problems related to customer requirements and usability problems related to control through checking the mapping matrix between the three domains. To determine how well this new usability evaluation method works, a between-subject experiment was conducted to compare the axiomatic evaluation method with the think aloud method. Sixty participants were randomly assigned to use either method to evaluate three popular consumer electronic devices (music player, digital camera, mobile phone) that represented different levels of complexity. Number of usability problems discovered, completion time, and overall user satisfaction were collected. Results show that the axiomatic evaluation method performed better in finding usability problems for the mobile phone. The axiomatic evaluation method was also better at finding usability problems about user expectation and control than the think aloud method. Benefits and drawbacks of using the axiomatic evaluation method are discussed.This chapter introduces a new usability evaluation method, the axiomatic evaluation method, which is developed based on the axiomatic design theory – a formalized design methodology that can be used to solve a variety of design problems. This new evaluation method examines three domains of a product: customer domain, functional domain, and control domain. This method investigates not only usability problems reported by the users, but also usability problems related to customer requirements and usability problems related to control through checking the mapping matrix between the three domains. To determine how well this new usability evaluation method works, a between-subject experiment was conducted to compare the axiomatic evaluation method with the think aloud method. Sixty participants were randomly assigned to use either method to evaluate three popular consumer electronic devices (music player, digital camera, mobile phone) that represented different levels of complexity. Number of usability problems discovered, completion time, and overall user satisfaction were collected. Results show that the axiomatic evaluation method performed better in finding usability problems for the mobile phone. The axiomatic evaluation method was also better at finding usability problems about user expectation and control than the think aloud method. Benefits and drawbacks of using the axiomatic evaluation method are discussed.