Chapter 9 - Special Topics

Abstract

Chapter 9 introduces a number of topics related to the measurement or analysis of user experience data not traditionally thought of as part of “mainstream” UX data. These include information you can glean from live data on a production website, data from card-sorting studies, data related to the accessibility of a website, and UX return on investment (ROI). If you’re dealing with a live website, you should be studying what your users are doing on the site as much as you can. Don’t just look at page views. Look at click-through rates and drop-off rates. Whenever possible, conduct live A/B tests to compare alternative designs (typically with small differences). Use appropriate statistics (e.g., χ2) to make sure any differences you’re seeing are statistically significant. Card sorting can be immensely helpful in learning how to organize some information or an entire website. Consider starting with an open sort and then following up with one or more closed sorts. Hierarchical cluster analysis and multidimensional scaling are useful techniques for summarizing and presenting the results. Closed card sorts can be used to compare how well different information architectures work for the users. Tree-testing tools can also be a useful way to test a candidate organization. Accessibility is just usability for a particular group of users. Whenever possible, try to include older users and users with various kinds of disabilities in your usability tests. In addition, you should evaluate your product against published accessibility guidelines or standards, such as the Web Content Accessibility Guidelines or Section 508. Calculating ROI data for usability work is sometimes challenging, but it usually can be done. If the users are employees of your company, it’s generally easy to convert usability metrics such as reductions in task times into dollar savings. If the users are external customers, you generally have to extrapolate usability metrics such as improved task completion rates or improved overall satisfaction to decreases in support calls, increases in sales, or increases in customer loyalty.