Usability Testing Study Research Articles

Getting a technology-based diabetes intervention ready for prime time: a review of usability testing studies.

Consumer health technologies can educate patients about diabetes and support their self-management, yet usability evidence is rarely published even though it determines patient engagement, optimal benefit of any intervention, and an understanding of generalizability. Therefore, we conducted a narrative review of peer-reviewed articles published from 2009 to 2013 that tested the usability of a web- or mobile-delivered system/application designed to educate and support patients with diabetes. Overall, the 23 papers included in our review used mixed (n = 11), descriptive quantitative (n = 9), and qualitative methods (n = 3) to assess usability, such as documenting which features performed as intended and how patients rated their experiences. More sophisticated usability evaluations combined several complementary approaches to elucidate more aspects of functionality. Future work pertaining to the design and evaluation of technology-delivered diabetes education/support interventions should aim to standardize the usability testing processes and publish usability findings to inform interpretation of why an intervention succeeded or failed and for whom.

European Public Assessment Report (EPAR) summaries for the public: are they fit for purpose? A user-testing study

ObjectivesApply ‘user testing’ methodology to test the readability of a European Public Assessment Report (EPAR) summary—which describes how the decision was made by the European Medicines Agency to approve a...

Usability of an Atrial Fibrillation Anticoagulation Decision-Support Tool

In individuals with nonvalvular atrial fibrillation, anticoagulant therapy with warfarin reduces the rate of thromboembolic events but increases the risk of bleeding. Treatment decisions frequently are inconsistent with guidelines. A new web-based atrial fibrillation decision-support tool (AF-DST) provides patient-specific information on the risk-benefit tradeoff of anticoagulation. The authors performed a pilot usability testing study of the AF-DST with 4 medical house officers and 4 attending physicians by simulating 9 outpatient clinical encounters involving tradeoffs between risks and benefits of anticoagulation. They recorded positive and negative critical incidents in the simulations and assessed satisfaction with use of the AF-DST by the Computer System Usability Questionnaire (CSUQ; score range on each item: 1 = strongly disagree to 7 = strongly agree). Users found the AF-DST to be helpful and had high CSUQ scores (mean item score, 6.3). Usability testing identified 6 positive and 14 negative critical incidents. Participants felt that the AF-DST guided them toward the correct decision. Nevertheless, they desired more information on the "black box" calculations and ignored alerts. Training level appeared to affect how the AF-DST was used, in particular, how users interacted with the AF-DST. Overall satisfaction with the AF-DST was high and the tool effectively communicated recommendations and uncertainty. Usability testing identified design issues and potential errors caused by decision-support tool use; these gaps should be addressed prior to clinical implementation.

Patient-Centered Design of an Information Management Module for a Personally Controlled Health Record

BackgroundThe development of health information technologies should be informed by iterative experiments in which qualitative and quantitative methodologies provide a deeper understanding of the abilities, needs, and goals of the target audience for a personal health application.ObjectiveOur objective was to create an interface for parents of children with attention-deficit hyperactivity/disorder (ADHD) to enter disease-specific information to facilitate data entry with minimal task burden.MethodsWe developed an ADHD-specific personal health application to support data entry into a personally controlled health record (PCHR) using a three-step, iterative process: (1) a needs analysis by conducting focus groups with parents of children with ADHD and an heuristic evaluation of a prerelease version of a PCHR, (2) usability testing of an initial prototype personal health application following a “think aloud” protocol, (3) performance testing of a revised prototype, and (4) finalizing the design and functionality of the ADHD personal health application. Study populations for the three studies (focus groups and two usability testing studies) were recruited from organizations in the greater Boston area. Study eligibility included being an English- or Spanish-speaking parent who was the primary caretaker of a school-age child with ADHD. We determined subjects’ health literacy using the Test of Functional Health Literacy in Adults (TOFHLA). We assessed subjects’ task burden using the National Aeronautics and Space Administration (NASA) Task Load Index. To assess the impact of factors associated with the time spent entering data, we calculated Pearson correlation coefficients (r) between time on task and both task burden and subject characteristics. We conducted t tests to determine if time on task was associated with successful task completion.ResultsThe focus groups included three cohorts: 4 Spanish-speaking parents with diverse health literacy, 4 English-speaking parents with lower health literacy, and 7 English-speaking parents with higher health literacy. Both the initial usability testing cohort (n = 10) and the performance-testing cohort (n = 7) included parents of diverse health literacy and ethnicity. In performance testing, the prototype PCHRs captured patient-specific data with a mean time on task of 11.9 minutes (SD 6.5). Task burden experienced during data entry was not associated with successful task completion (P = .92). Subjects’ past computer experience was highly correlated with time on task (r = .86, P = .01), but not with task burden (r = .18, P = .69). The ADHD personal health application was finalized in response to these results by (1) simplifying the visual environment, (2) including items to support users limited by health literacy or technology experience, and (3) populating the application’s welcome screen with pictures of culturally diverse families to establish a personal family-oriented look and feel.ConclusionsOur patient-centered design process produced a usable ADHD-specific personal health application that minimizes the burden of data entry.

Searching for Information: Applying Usability Testing Methods to a Study of Information Retrieval and Relevance Assessment

ABSTRACTThis paper discusses research in progress that explores the relevance assessments of researchers using networked information systems. To date, two academics are participating in an intensive qualitative exploration of their search for information related to personal research projects. This paper discusses the application of usability testing techniques and a formal usability laboratory to the observation of searching and relevance assessments. Early findings suggest integrating usability testing and information studies research can help build our understanding of individual retrieval experiences.