Adopting Maximum Pupil Diameter to Detect Subtle Usability Issues of a Smartphone Application, Conflict Solver

Abstract

With the increasing popularity and unparalleled ubiquity of smartphones, researchers in various fields are designing and developing novel applications for this platform as part of the solution to the challenging problems they face. Usability lies at the core of the user experience of such applications. However, the assessments used are often limited to summative and post-event methods, which can overlook subtle yet impactful issues. Objective and instantaneous measures of cognitive workloads provide a solution to this shortcoming. Our previous research has established the reliability of maximum pupil dilation, measured with Tobii Pro Nano, as a preeminent indicator of cognitive workload surges in mobile application users. In this study, we used this measure to locate user cognitive workload peaks while using Conflict Solver and discovered subtle user interface issues that were not reported in the post-usability interview. A total of 30 participants completed a Conflict Solver usability experiment with two phases. In phase 1, the participants performed two “Add a Term” tasks on the original Conflict Solver, followed by a semi-structured interview about their experience with the application. A few subtle usability issues with a drop-down menu were detected through identifying user cognitive workload peaks. In phase 2, the same participants completed the same tasks on Conflict Solver with a redesigned and extended drop-down menu. The results showed that the new design solved the usability issues, and the participants became more favor the drop-down menu over the input box. In conclusion, including maximum pupil dilation into the usability assessment toolkit would provide a more objective and comprehensive usability assessment of a smartphone application. It can also be used to verify the successfulness of a user interface design solution.