Usability Engineering Methods Research Articles

User-Centered Design of a Preference-Driven Patient Activation Tool for Optimizing Depression Treatment in Integrated Primary Care Settings (The Transform DepCare Study).

Few patient engagement tools incorporate thecomplex patient experiences, contexts, and workflows that limit depression treatment implementation. Describe a user-centered design (UCD) process for operationalizing a preference-driven patient activation tool. Informed by UCD and behavior change/implementation science principles, we designed a preference-driven patient activation prototype for engaging patients in depression treatment. We conducted three usability cycles using different recruitment/implementation approaches: near live/live testing in primary care waiting rooms (V1-2) and lab-based think aloud testing (V3) oversampling older, low-literacy, and Spanish-speaking patients in the community and via EHR algorithms. We elicited clinician and "heuristic" expert input. We administered the system usability scale (SUS) all three cycles and pre-post V3, the patient activation measure, decisional conflict scale, and depression treatment barriers. We employed descriptive statistics and thematically analyzed observer notes and transcripts for usability constructs. Overall, 43 patients, 3 clinicians, and 5 heuristic (a usability engineering method for identifying usability problems) experts participated. Among patients, 41.9% were ≥ 65years old, 79.1% female, 23.3% Black, 62.8% Hispanic, and 55.8% Spanish-speaking and 46.5% had ≤ high school education. We described V1-3 usability (67.2, 77.3, 81.8), treatment seeking (92.3%, 87.5%, 92.9%), likelihood/comfort discussing with clinician (76.9%, 87.5%, 100.0%), and pre vs. post decisional conflict (23.7 vs. 15.2), treatment awareness (71.4% vs. 92.9%), interest in antidepressants (7.1% vs. 14.3%), and patient activation (66.8 vs. 70.9), with fewer barriers pertaining to cost/insurance, access/coordination, and self-efficacy/stigma/treatment efficacy. Key themes included digital literacy, understandability, high acceptability for aesthetics, high usefulness of patient/clinician videos, and workflow limitations. We adapted manual entry/visibility/content; added patient activation and a personalized algorithm; and proposed flexible, care manager delivery leveraging clinic screening protocols. We provide an example of leveraging UCD to design/adapt a real-world, patient experience and workflow-aligned patient activation tool in diverse populations.

Application of Anti-Stigma Design Heuristics for Usability Inspection.

User interface evaluation has become important in developing usable health care technologies. Although usability engineering methods have been applied in the design and evaluation of health care software, available heuristics focus on task-work aspects and do not address stigma associated with many health conditions. We used a previous set of heuristics and propose a new set of anti-stigma heuristics to evaluate stigmatization in health care websites. The extended set of heuristics were concurrently applied in a heuristic evaluation and a cognitive walkthrough to evaluate an endometriosis and sexual pain website. The walkthrough involved 5 tasks that required 21 actions to execute. Twenty-six usability problems were identified and recommendations for re-design were made to the design team before end-user testing. The anti-stigma heuristics received worse ratings than the traditional heuristics, resulting in several design changes that might otherwise have been missed. Thus, the new anti-stigma heuristics were a valuable contribution.

A Review of Usability Methods Used in the Evaluation of Mobile Health Applications for Diabetes.

A literature review was conducted to identify applicable studies in the databases ACM Digital Library, Cinahl and Pubmed between the years 2015 and 2020. After the inclusion and exclusion criteria were applied, 32 articles remained that were included in the final review. Most of the studies included one established usability engineering method such as an expert-based and/or user-based method or a validated questionnaire/instrument. Some also included a combination of these. Others used methods of their own design; commonly questionnaires and interviews either on their own or in combination. To achieve an adequate level of evidence and quality in the evaluation, it is important that at least one is an established usability engineering method or a validated instrument. This to assure and continue to build the evidence base in this area.

Heuristic Evaluation on Usability of Educational Games: A Systematic Review

This work presents a systematic review whose objective was to identify heuristics applicable to the evaluation of the usability of educational games. Heuristics are usability engineering methods that aim to detect problems in the use of a system during its development and / or when its interface is in interaction with the user. Therefore, applying heuristics is an essential part of developing digital educational games. Search sources were articles available in all the databases present in the Capes / MEC / Brazil periodicals portal, in the available languages. The descriptors adopted were "educational games", "heuristic" and "usability" in Boolean search in titles, abstracts and keywords, with AND operator, for publications starting in 2014. The inclusion criteria were: (a) articles with a clear description of the methodology used in the usability analysis; (b) studies presenting primary data and (c) articles whose focus corresponds to the investigated question. Two examiners conducted the searches in the databases and a third the evaluation and general review of the data. Initially, 93 articles were identified, of which 19 were repeated, 5 were literature reviews. Of the 69 that remained, 57 were elected as not eligible with only 12 selected for full studies, of which 6 entered the final review. With this review we can deduce that the field of heuristics and usability for educational games is still little explored, with few specific evaluations validated or in the process of validation, requiring greater investment in the area. Through this review, we found at least one heuristic that meets the usability evaluation of educational software: Game User Experience Satisfaction Scale (GUESS).

EHealth literacy issues, constructs, models, and methods for health information technology design and evaluation

The concept of eHealth literacy is beginning to be recognized as a being of key importance in the design and adoption of effective and efficient health information systems and applications targeted to lay people and patients. Indeed, many systems such as patient portals and personal health records have not been adopted due to a mismatch between the level of eHealth literacy demanded by a system and the level of eHealth literacy possessed by end users. The purpose of this paper is to present an overview of important concepts related to eHealth literacy, as well as how the notion of eHealth literacy can be applied to improve the design and adoption of consumer health information systems. This paper begins with describing the importance of eHealth literacy with respect to design of health applications for the general public paired with examples of consumer health information systems whose limited success and adoption has been attributed to the lack of consideration for eHealth literacy. This is followed by definitions of what eHealth literacy is and how it emerged from the related concept of health literacy. A model for conceptualizing the importance of aligning consumers’ eHealth literacy skills and the demands systems place on their skills is then described. Next, current tools for assessing consumers’ eHealth literacy levels are outlined, followed by an approach to systematically incorporating eHealth literacy in the deriving requirements for new systems is presented. Finally, a discussion of evolving approaches for incorporating eHealth literacy into usability engineering methods is presented.

Usability Evaluation of Public Web Mapping Sites

Abstract. Web mapping sites are interactive maps that are accessed via Webpages. With the rapid development of Internet and Geographic Information System (GIS) field, public web mapping sites are not foreign to people. Nowadays, people use these web mapping sites for various reasons, in that increasing maps and related map services of web mapping sites are freely available for end users. Thus, increased users of web mapping sites led to more usability studies. Usability Engineering (UE), for instance, is an approach for analyzing and improving the usability of websites through examining and evaluating an interface. In this research, UE method was employed to explore usability problems of four public web mapping sites, analyze the problems quantitatively and provide guidelines for future design based on the test results. Firstly, the development progress for usability studies were described, and simultaneously several usability evaluation methods such as Usability Engineering (UE), User-Centered Design (UCD) and Human-Computer Interaction (HCI) were generally introduced. Then the method and procedure of experiments for the usability test were presented in detail. In this usability evaluation experiment, four public web mapping sites (Google Maps, Bing maps, Mapquest, Yahoo Maps) were chosen as the testing websites. And 42 people, who having different GIS skills (test users or experts), gender (male or female), age and nationality, participated in this test to complete the several test tasks in different teams. The test comprised three parts: a pretest background information questionnaire, several test tasks for quantitative statistics and progress analysis, and a posttest questionnaire. The pretest and posttest questionnaires focused on gaining the verbal explanation of their actions qualitatively. And the design for test tasks targeted at gathering quantitative data for the errors and problems of the websites. Then, the results mainly from the test part were analyzed. The success rate from different public web mapping sites was calculated and compared, and displayed by the means of diagram. And the answers from questionnaires were also classified and organized in this part. Moreover, based on the analysis, this paper expands the discussion about the layout, map visualization, map tools, search logic and etc. Finally, this paper closed with some valuable guidelines and suggestions for the design of public web mapping sites. Also, limitations for this research stated in the end.

Validation process model for product lifecycle management

Nowadays people demand outstanding products: easy-to-use, easy-to-understand, and worry-free. This objective can be ensured only through validation processes that are managed in a quality management system. Manufacturing companies often confuse validation with or reduces it to verification procedures. This is usually due to the unfamiliarity of these companies with managing the complex heterogeneous information necessary for integrating customer voice into testing activities. This paper proposes a model for integrating validation processes into a product lifecycle management strategy, which allows customers to actually feedback on the design. The model achieves a human-centred design through usability-engineering methods and has been tested on the case study of a board game for visually impaired people.

Designing and conducting low-cost in-situ clinical simulations: a methodological approach.

The need to develop more effective and usable systems in healthcare has led to increasing adoption of usability engineering methods. Primary among these methods has been usability testing, which involves observing representative users while they carry out representative tasks using health information systems. However, such testing may not capture the cognitive and social contexts of use of systems in real settings, necessitating the use of clinical simulations. Such simulations involve testing representative users carrying out representative tasks, systems in realistic and representative environments. In this paper we describe our work in both reducing the cost of simulation testing while at the same time increasing the fidelity of the simulations. We describe an approach which involves conducting simulations within real work environments and contexts as a way to achieve a high level of ecological validity. The stages involved in setting up such low-cost in-situ clinical simulations are detailed in this paper.

Investigating and promoting UX practice in industry: An experimental study

The efforts of addressing user experience (UX) in product development keep growing, as demonstrated by the proliferation of workshops and conferences bringing together academics and practitioners, who aim at creating interactive software able to satisfy their users. This special issue focuses on “Interplay between User Experience Evaluation and Software Development”, stating that the gap between human-computer interaction and software engineering with regard to usability has somewhat been narrowed. Unfortunately, our experience shows that software development organizations perform few usability engineering activities or none at all. Several authors acknowledge that, in order to understand the reasons of the limited impact of usability engineering and UX methods, and to try to modify this situation, it is fundamental to thoroughly analyze current software development practices, involving practitioners and possibly working from inside the companies. This article contributes to this research line by reporting an experimental study conducted with software companies. The study has confirmed that still too many companies either neglect usability and UX, or do not properly consider them. Interesting problems emerged. This article gives suggestions on how they may be properly addressed, since their solution is the starting point for reducing the gap between research and practice of usability and UX. It also provides further evidence on the value of the research method, called Cooperative Method Development, based on the collaboration of researchers and practitioners in carrying out empirical research; it has been used in a step of the performed study and has revealed to be instrumental for showing practitioners why to improve their development processes and how to do so.

Usability Methods for Ensuring Health Information Technology Safety: Evidence-Based Approaches Contribution of the IMIA Working Group Health Informatics for Patient Safety

Issues related to lack of system usability and potential safety hazards continue to be reported in the health information technology (HIT) literature. Usability engineering methods are increasingly used to ensure improved system usability and they are also beginning to be applied more widely for ensuring the safety of HIT applications. These methods are being used in the design and implementation of many HIT systems. In this paper we describe evidence-based approaches to applying usability engineering methods. A multi-phased approach to ensuring system usability and safety in healthcare is described. Usability inspection methods are first described including the development of evidence-based safety heuristics for HIT. Laboratory-based usability testing is then conducted under artificial conditions to test if a system has any base level usability problems that need to be corrected. Usability problems that are detected are corrected and then a new phase is initiated where the system is tested under more realistic conditions using clinical simulations. This phase may involve testing the system with simulated patients. Finally, an additional phase may be conducted, involving a naturalistic study of system use under real-world clinical conditions. The methods described have been employed in the analysis of the usability and safety of a wide range of HIT applications, including electronic health record systems, decision support systems and consumer health applications. It has been found that at least usability inspection and usability testing should be applied prior to the widespread release of HIT. However, wherever possible, additional layers of testing involving clinical simulations and a naturalistic evaluation will likely detect usability and safety issues that may not otherwise be detected prior to widespread system release. The framework presented in the paper can be applied in order to develop more usable and safer HIT, based on multiple layers of evidence.

From Usability Testing to Clinical Simulations: Bringing Context into the Design and Evaluation of Usable and Safe Health Information Technologies

The objective of this paper is to explore human factors approaches to understanding the use of health information technology (HIT) by extending usability engineering approaches to include analysis of the impact of clinical context through use of clinical simulations. Methods discussed are considered on a continuum from traditional laboratory-based usability testing to clinical simulations. Clinical simulations can be conducted in a simulation laboratory and they can also be conducted in real-world settings. The clinical simulation approach attempts to bring the dimension of clinical context into stronger focus. This involves testing of systems with representative users doing representative tasks, in representative settings/environments. Application of methods where realistic clinical scenarios are used to drive the study of users interacting with systems under realistic conditions and settings can lead to identification of problems and issues with systems that may not be detected using traditional usability engineering methods. In conducting such studies, careful consideration is needed in creating ecologically valid test scenarios. The evidence obtained from such evaluation can be used to improve both the usability and safety of HIT. In addition, recent work has shown that clinical simulations, in particular those conducted in-situ, can lead to considerable benefits when compared to the costs of running such studies. In order to bring context of use into the testing of HIT, clinical simulation, involving observing representative users carrying out tasks in representative settings, holds considerable promise.

HCI Practices for Building Usable Software

The integration of usability engineering methods into software development life cycles is seldom realized in industrial settings.

Designing Interactive Applications to Support Novel Activities

R&D in media-related technologies including multimedia, information retrieval, computer vision, and the semantic web is experimenting on a variety of computational tools that, if sufficiently matured, could support many novel activities that are not practiced today. Interactive technology demonstration systems produced typically at the end of their projects show great potential for taking advantage of technological possibilities. These demo systems or “demonstrators” are, even if crude or farfetched, a significant manifestation of the technologists’ visions in transforming emerging technologies into novel usage scenarios and applications. In this paper, we reflect on design processes and crucial design decisions made while designing some successful, web-based interactive demonstrators developed by the authors. We identify methodological issues in applying today’s requirement-driven usability engineering method to designing this type of novel applications and solicit a clearer distinction between designing mainstream applications and designing novel applications. More solution-oriented approaches leveraging design thinking are required, and more pragmatic evaluation criteria is needed that assess the role of the system in exploiting the technological possibilities to provoke further brainstorming and discussion. Such an approach will support a more efficient channelling of the technology-to-application transformation which are becoming increasingly crucial in today’s context of rich technological possibilities.

Research of Applying Usability Engineering to Design of Distance Education System

It is discussed how to apply usability engineering theory and method to the design of distance education system, it aims to design a distance education system with good usability in order to resolve such issues as operating difficulties and poor user experience resulted by traditional function design method. The design of Engineering Drawing distance education system is illustrated as an example, the method of applying usability engineering theory and methods to the design of distance education system has universal guiding significance and broad application prospects.

Comparison of alphabetical versus categorical display format for medication order entry in a simulated touch screen anesthesia information management system: an experiment in clinician-computer interaction in anesthesia

BackgroundAnesthesia information management system (AIMS) records should be designed and configured to facilitate the accurate and prompt recording of multiple drugs administered coincidentally or in rapid succession.MethodsWe proposed two touch-screen display formats for use with our department’s new EPIC touch-screen AIMS. In one format, medication “buttons” were arranged in alphabetical order (i.e. A-C, D-H etc.). In the other, buttons were arranged in categories (Common, Fluids, Cardiovascular, Coagulation etc.). Both formats were modeled on an iPad screen to resemble the AIMS interface. Anesthesia residents, anesthesiologists, and Certified Registered Nurse Anesthetists (n = 60) were then asked to find and touch the correct buttons for a series of medications whose names were displayed to the side of the entry screen. The number of entries made within 2 minutes was recorded. This was done 3 times for each format, with the 1st format chosen randomly. Data were analyzed from the third trials with each format to minimize differences in learning.ResultsThe categorical format had a mean of 5.6 more drugs entered using the categorical method in two minutes than the alphabetical format (95% confidence interval [CI] 4.5 to 6.8, P < 0.0001). The findings were the same regardless of the order of testing (i.e. alphabetical-categorical vs. categorical - alphabetical) and participants’ years of clinical experience. Most anesthesia providers made no (0) errors for most trials (N = 96/120 trials, lower 95% limit 73%, P < 0.0001). There was no difference in error rates between the two formats (P = 0.53).ConclusionsThe use of touch-screen user interfaces in healthcare is increasingly common. Arrangement of drugs names in a categorical display format in the medication order-entry touch screen of an AIMS can result in faster data entry compared to an alphabetical arrangement of drugs. Results of this quality improvement project were used in our department’s design of our final intraoperative electronic anesthesia record. This testing approach using cognitive and usability engineering methods can be used to objectively design and evaluate many aspects of the clinician-computer interaction in electronic health records.

Collaboration in Open Source Domains

Free and open source software (F/OSS) developers have a tendency to build feature-centric projects rather than following a user-centered design, ignoring the necessity of usability in the resulting product. While there are many reasons behind this, the main cause can be stated as the lack of awareness of usability from developers’ point of view and little interaction of project stakeholders with Human-Computer Interaction (HCI) studies. This chapter examines different types of collaboration methods of usability experts and developers focusing particularly on open source projects, together with potential issues envisaged during the communication phases. The chapter also focuses on the collaboration trends and patterns of HCI experts, developers and users with an emphasis on concerns related to inefficient exploitation of current tools and technologies and provide an open usability engineering method which could be exploited in distributed projects

Library as Laboratory: Computer Science Students Practice Usability Engineering in an Academic Library.

This article presents a project-based collaboration between a usability engineering (UE) course and the campus library. Student teams developed UE projects based on usability issues they identified in the library's web-based and information technology services. Using the library as a focal point, students practiced common UE methods such as ethnomethodological observation, web usability evaluation, and an iterative prototype process based on cycles of design, testing and evaluation. At the end of the term, teams of students presented their novel solutions to library staff as part of their final grade. Other academic librarians may find inspiration to pursue similar partnerships on their own campuses.

Usability methods’ familiarity among map application developers

Developments in hardware and software have led to new innovative methods for visualising geospatial data. At the same time user-centred design (UCD) and usability engineering methods have a fundamental role in designing applications for new technical environments, which involve entirely new ways of interacting. However, applying methods from other research disciplines may not always be straightforward, as the product developers have to operate in a challenging interdisciplinary field. The aim of this study was to find out how usability engineering is currently included in the development of map services. Seven companies developing different types of map applications in Finland were interviewed. The results support the suitability of usability engineering for map application design, since by including the usability approach into the product design, while simultaneously taking into account the individuality and diversity of users and their tasks together with the characteristics of the maps, application developers are more likely to design products that have a higher quality of use. This study identifies the main occasions when the usability approach could be most beneficial. Furthermore, the benefits and challenges of including usability approaches in map application design are discussed. Preliminary ideas on what usability means in the context of map applications are also given. Finally, the importance for providing a basis for the further development of application-specific guidelines and techniques is addressed.

Heuristic evaluation quality score (HEQS)

Heuristic Evaluation is a discount usability engineering method involving three or more evaluators who evaluate the compliance of an interface based on a set of heuristics. Because the quality of t...

Usability engineering methods for software developers

The human-computer interaction community aims to increase the awareness and acceptance of established methods among software practitioners. Indeed, awareness of the basic usability methods will drive an Information Society for all.