Task-based Usability Testing Research Articles

Remote Usability Assessment of Topic Visualization Interfaces with Public Participation Data: A Case Study

Citizen participation often faces challenges of transparency and accountability. Visualizations’s usability becomes key for public consultation activities. The tree map is frequently used to disseminate data and to give it back to the population. The purpose of this study is to understand how tree maps and stacked barcharts differ in terms of effectiveness, efficiency, and satisfaction in tasks, like solving topic categorization and comparison analysis tasks. An experimental design was used to examine user performance based on a task-based usability test. 34 participants interacted remotely with data visualizations from an open 2016 participatory constitution-making process. The ANOVA showed that stacked barcharts work significantly better for comparison tasks than the tree map, but there are no significant differences in regards to categorization tasks. Public participation initiatives should first determine what cognitive operations their users perform before deciding which visualization interfaces will be more useful for the intended public.

Feasibility-Usability Study of a Tablet App Adapted Specifically for Persons with Cognitive Impairment-SMART4MD (Support Monitoring and Reminder Technology for Mild Dementia).

Population ageing within Europe has major social and economic consequences. One of the most devastating conditions that predominantly affects older people is dementia. The SMART4MD (Support Monitoring and Reminder Technology for Mild Dementia) project aims to develop and test a health application specifically designed for people with mild dementia. The aim of this feasibility study was to evaluate the design of the SMART4MD protocol, including recruitment, screening, baseline examination and data management, and to test the SMART4MD application for functionality and usability before utilization in a full-scale study. The feasibility study tested the protocol and the app in Spain and Sweden. A total of nineteen persons with cognitive impairment, and their informal caregivers, individually performed a task-based usability test of the SMART4MD app model in a clinical environment, followed by four-week testing of the app in the home environment. By employing a user-centered design approach, the SMART4MD application proved to be an adequate and feasible interface for an eHealth intervention. In the final usability test, a score of 81% satisfied users was obtained. The possibility to test the application in all the procedures included in the study generated important information on how to present the technology to the users and how to improve these procedures.

Usability and feasibility of consumer-facing technology to reduce unsafe medication use by older adults.

Mobile health technology can improve medication safety for older adults, for instance, by educating patients about the risks associated with anticholinergic medication use. This study's objective was to test the usability and feasibility of Brain Buddy, a consumer-facing mobile health technology designed to inform and empower older adults to consider the risks and benefits of anticholinergics. Twenty-three primary care patients aged ≥60 years and using anticholinergic medications participated in summative, task-based usability testing of Brain Buddy. Self-report usability was assessed by the System Usability Scale and performance-based usability data were collected for each task through observation. A subset of 17 participants contributed data on feasibility, assessed by self-reported attitudes (feeling informed) and behaviors (speaking to a physician), with confirmation following a physician visit. Overall usability was acceptable or better, with 100% of participants completing each Brain Buddy task and a mean System Usability Scale score of 78.8, corresponding to "Good" to "Excellent" usability. Observed usability issues included higher rates of errors, hesitations, and need for assistance on three tasks, particularly those requiring data entry. Among participants contributing to feasibility data, 100% felt better informed after using Brain Buddy and 94% planned to speak to their physician about their anticholinergic related risk. On follow-up, 82% reported having spoken to their physician, a rate independently confirmed by physicians. Consumer-facing technology can be a low-cost, scalable intervention to improve older adults' medication safety, by informing and empowering patients. User-centered design and evaluation with demographically heterogeneous clinical samples uncovers correctable usability issues and confirms the value of interventions targeting consumers as agents in shared decision making and behavior change.

Academic E-book Usability from the Student’s Perspective

Abstract
 Objective – This article describes how librarians systematically compared different e-book platforms to identify which features and design impact usability and user satisfaction.
 Methods – This study employed task-based usability testing, including the “think-aloud protocol.” Students at the University of Colorado Boulder completed a series of typical tasks to compare the usability and measure user satisfaction with academic e-books. For each title, five students completed the tasks on three e-book platforms: the publisher platform and two aggregators. Thirty-five students evaluated seven titles on nine academic e-book platforms.
 Results – This study identified each platform’s strengths and weaknesses based on students’ experiences and preferences. The usability tests indicated that students preferred Ebook Central over EBSCO and strongly preferred the aggregators over publisher platforms.
 Conclusions – Librarians can use student expectations and preferences to guide e-book purchasing decisions. Preferences may vary by institution, but variations in e-book layout and functionality impact students’ ability to successfully complete tasks and influences their affinity for or satisfaction with any given platform. Usability testing is a useful tool for gauging user expectations and identifying preferences for features, functionality, and layout.

A Patient-Facing Diabetes Dashboard Embedded in a Patient Web Portal: Design Sprint and Usability Testing.

BackgroundHealth apps and Web-based interventions designed for patients with diabetes offer novel and scalable approaches to engage patients and improve outcomes. However, careful attention to the design and usability of these apps and Web-based interventions is essential to reduce the barriers to engagement and maximize use.ObjectiveThe aim of this study was to apply design sprint methodology paired with mixed-methods, task-based usability testing to design and evaluate an innovative, patient-facing diabetes dashboard embedded in an existing patient portal and integrated into an electronic health record.MethodsWe applied a 5-day design sprint methodology developed by Google Ventures (Alphabet Inc, Mountain View, CA) to create our initial dashboard prototype. We identified recommended strategies from the literature for using patient-facing technologies to enhance patient activation and designed a dashboard functionality to match each strategy. We then conducted a mixed-methods, task-based usability assessment of dashboard prototypes with individual patients. Measures included validated metrics of task performance on 5 common and standardized tasks, semistructured interviews, and a validated usability satisfaction questionnaire. After each round of usability testing, we revised the dashboard prototype in response to usability findings before the next round of testing until the majority of participants successfully completed tasks, expressed high satisfaction, and identified no new usability concerns (ie, stop criterion was met).ResultsThe sample (N=14) comprised 5 patients in round 1, 3 patients in round 2, and 6 patients in round 3, at which point we reached our stop criterion. The participants’ mean age was 63 years (range 45-78 years), 57% (8/14) were female, and 50% (7/14) were white. Our design sprint yielded an initial patient-facing diabetes dashboard prototype that displayed and summarized 5 measures of patients’ diabetes health status (eg, hemoglobin A1c). The dashboard used graphics to visualize and summarize health data and reinforce understanding, incorporated motivational strategies (eg, social comparisons and gamification), and provided educational resources and secure-messaging capability. More than 80% of participants were able to successfully complete all 5 tasks using the final prototype. Interviews revealed usability concerns with design, the efficiency of use, and content and terminology, which led to improvements. Overall satisfaction (0=worst and 7=best) improved from the initial to the final prototype (mean 5.8, SD 0.4 vs mean 6.7, SD 0.5).ConclusionsOur results demonstrate the utility of the design sprint methodology paired with mixed-methods, task-based usability testing to efficiently and effectively design a patient-facing, Web-based diabetes dashboard that is satisfying for patients to use.

User-Centered Evaluations with Older Adults: Testing the Usability of a Mobile Health System for Heart Failure Self-Management.

Many older adults living with heart failure struggle to follow recommended self-management routines. To help older adults with heart failure more effectively and efficiently self-manage their disease, we developed Engage, a mobile health application promoting the performance, logging, and sharing of routine self-management behaviors. This paper reports on the usability evaluation of the Engage system with 15 older adults with heart failure and informal caregivers. In two phases, participants used Engage during a task-based usability test (n=5) and a scenario-based usability test (n=10). Usability and performance data were assessed through video-recorded observation and the administration of the system usability scale (SUS) and NASA Task Load Index (TLX). We found that task-based testing was useful in quickly identifying problems within our application, but scenario-based testing elicited more valuable feedback from older adults. A comparison of the different evaluation methods used and the discussion of the challenges encountered provide multiple implications for the practice of usability testing of mobile health products with older adults.

Improved usability of a multi-infusion setup using a centralized control interface: A task-based usability test.

The objective of this study was to assess the usability benefits of adding a bedside central control interface that controls all intravenous (IV) infusion pumps compared to the conventional individual control of multiple infusion pumps. Eighteen dedicated ICU nurses volunteered in a between-subjects task-based usability test. A newly developed central control interface was compared to conventional control of multiple infusion pumps in a simulated ICU setting. Task execution time, clicks, errors and questionnaire responses were evaluated. Overall the central control interface outperformed the conventional control in terms of fewer user actions (40±3 vs. 73±20 clicks, p<0.001) and fewer user errors (1±1 vs. 3±2 errors, p<0.05), with no difference in task execution times (421±108 vs. 406±119 seconds, not significant). Questionnaires indicated a significant preference for the central control interface. Despite being novice users of the central control interface, ICU nurses displayed improved performance with the central control interface compared to the conventional interface they were familiar with. We conclude that the new user interface has an overall better usability than the conventional interface.

Card Sorting in an Online Environment: Key to Involving Online-Only Student Population in Usability Testing of an Academic Library Web Site?

ABSTRACTBased on in-person, task-based usability testing and interviews, the authors' library Web site was recently overhauled in order to improve user experience. This led to the authors' interest in additional usability testing methods and test environments that would most closely fit their library's goals and situation. The appeal of card sorting methods became evident: learning more than users' points of confusion interacting with the site, but learning users' preferences for grouping pages or concepts and naming various library links. The appeal of the online venue for card sorting was first that testing could incorporate input from a larger base of users than in-person testing alone, and, additionally, that testing could include the university's online-only student population.

The Perfect Storm: Examining User Experience and Conducting a Usability Test to Investigate a Disruptive Academic Library Web Site Redevelopment

The researchers conducted a task-based usability test of the effectiveness of online research beginning on the library Web site homepage. The participants included five university faculty members, six graduate students, and six undergraduate students. All participants reported feeling satisfied with their overall research experience, though most were unable to effectively complete all the research tasks of the test. The researchers identified weaknesses in the approach and process of many participants, and overall usability issues of the library discovery tool and other library Web site pages and research interfaces. Findings indicate the need to strategically incorporate self-service information literacy and research skills help into the library Web site, and to implement navigation and design changes to the library homepage, discovery tool interface, online catalog, and across all the library's Web services.

Evaluating Mobile Phones and Web Sites for Academic Information Needs

In the last decade, there has been an exponential growth in use of mobile phones among people. Smart phone invention has digitized life of a common man especially after introduction of 3G/4G technology. People are used to use Internet on the move because of this advancement in technology. This advancement has also motivated usability design researchers to propose more usable designs for both smart phones and web sites. This work focuses on evaluation of web usability of mobile phones as well as usability of university web sites. Evaluation is performed on the most popular mobile phones required by the most common mobile users. Selection of the most popular mobile devices, the most common mobile users and their web usage is done by conducting a very detailed survey in the local market. Survey concludes that students and labors are the most common buyers of mobile phones and we choose three mobiles phones from the category of most popular phones that are iPhone (iPhone 4 precisely), Q-Mobile (Q Mobile A35) and Windows phone (Lumia 535). Six participants (three male and three females) are selected for fully detailed and rigorous task-based usability testing with “think aloud” technique. Task scenarios are defined to evaluate the usability of both i.e. smart phones and chosen university web sites. From results of usability testing, we find out that iPhone has better usability design as far as its response time is concerned while Q Mobile ranks second and Microsoft Windows phone takes last position in this ranking. Usability evaluation of university web sites on these mobile phones concludes that web site of Islamia university of Bahawalpur (I.U.B, Bahawalpur) has better mobile usability design and Bahauddin Zakariya university (BZU, Multan) and NFC Institute of Engineering and Technology NFCIET, Multan) second and third respectively and while web site of Institute of Southern Punjab (ISP, Multan) comes last when measured in terms of task completion time. All tests are performed on wireless network when internet download speed is between 3MBPS to 3.2MBPS.

Student Searching with EBSCO Discovery: A Usability Study

The researchers conducted task-based usability testing of the EBSCO Discovery Service (EDS) with 20 students. While most participants felt positively about EDS, a few usability issues emerged. Confusion arose regarding source types and icons, facets/limiters, relevancy ranking, integrated search connectors, and more. Students encountered difficulty with spelling corrections and a failure to understand terminology, and they demonstrated an unwillingness to view additional pages of search results. Findings indicate the need for changes to the local implementation of EDS as well as to those features and structures under EBSCO's control.

Merits of usability testing for PACS selection

ObjectivesTo compare the usability of different Picture Archiving and Communication System (PACS) workstations, determine whether a usability test has added value with respect to the traditional way of comparing PACSs based on functional requirements, and to evaluate the appropriateness of a task-based methodology for a PACS usability test. MethodsA task-based usability test of four PACS workstations was performed. Radiologists’ subjective responses to the PACSs and their performance on the tasks were measured. To mimic the traditional PACS selection process, two functional requirements were defined which the PACSs met in varying degrees. The focus of the usability test was on the aspects of the PACS related to these requirements. The usability results were compared to the PACSs’ ability to meet the functional requirements. ResultsOne PACS outperformed the other PACSs both in terms of subjective preference and task performance, indicating its superior usability. There were differences in usability between PACSs with identical functionality. Also, a PACS with theoretically advantageous functionality for a given task did not necessarily have better usability for this task than a PACS without this functionality. There was a discrepancy between participants’ subjective preferences and their task performance, which indicates that it is vital to include performance measures in the usability assessment so that it accurately reflects the efficiency of interaction. ConclusionsThe differences in usability between PACSs with identical functionality indicate that functional requirements alone are insufficient to determine a PACS's overall quality. A usability test should therefore be used in addition to a functional requirement list in a PACS selection process to ensure that a hospital buys the PACS with the highest quality. A task-based usability evaluation methodology, which yields both subjective preference data and objective performance data of radiologists interacting with the PACS, is very suitable for such a usability test.

Reverse engineering of content to find usability problems

For tools that involve the creation of an artifact or document, reverse engineering potentially provides an interesting alternative to task-based usability testing. In this case study, participants...

Comparative User Experiences of Next-Generation Catalogue Interfaces

One of the presumed advantages of next-generation library catalogue interfaces is that the user experience is improved—that it is both richer and more intuitive. Often the interfaces come with little or no user-facing documentation or imbedded "help" for patrons based on an assumption of ease of use and familiarity of the experience, having followed best practices in use on the Web. While there has been much gray literature (published on library Web sites, etc.) interrogating these implicit claims and contrasting the new interfaces to traditional Web-based catalogues, this article details a consistent and formal comparison of whether users can actually accomplish common library tasks, unassisted, using these interfaces. The author has undertaken a task-based usability test of vendor-provided next-generation catalogue interfaces and Web-scale discovery tools (Encore Synergy, Summon, WorldCat Local, Primo Central, EBSCO Discovery Service). Testing was done with undergraduates across all academic disciplines. The resulting qualitative data, noting any demonstrated trouble using the software as well as feedback or suggested improvements that the users may have about the software, will assist academic libraries in making or validating purchase and subscription decisions for these interfaces as well as help vendors make data-driven decisions about interface and experience enhancements.