Usability Inspection Research Articles

Live or Let Die: MILSTD 1472 Type Usability Checklists in an Agile Development World

Traditional Human System Integration (HSI) processes are at risk of being eliminated given the industry driven velocity and continuous delivery methods of Agile and Development to Operations Engineering. One such process at risk is usability standard checklists like MIL-STD 1472. The DoD Design Criteria Standard—Human Engineering, or MIL-STD 1472 is a military standard that provides guidelines for the ergonomic design of military systems and equipment. It is primarily focused on ensuring that military systems are designed to optimize human performance, safety, and comfort. However, it can be time-consuming to use given that it contains over 4,400 compliance items in its entirety. In an Agile DevOps environment, the typical delivery of products is 2 weeks with some software products boasting deployments every 11 min. Incorporation of usability and rigorous human factors checklists in the design of products using a DevOps environment is a challenge given the shorter timelines and smaller increments between product development and deployment combined with the expertise and hours needed to execute the checklists. The need for regression testing, the act of re-checking items to make sure new features have not created unforeseen usability issues, exacerbates the problem. Software quality testing for each release demands automatic methods to reduce the efforts of recursive testing. Industry standard methods in agile engineering utilize heuristics for quick turn low-cost usability inspections. This paper describes the derivation of a set of usability heuristics based on MIL-STD 1472 which can be used to identify potential human factors issues more quickly, with less cost, and reduced amount of expertise required. The paper also gives an example of a case study which incorporated automatic checking to support usability experts keeping pace with the high velocity pipelines of software development in an agile DevOps environment.

Development and expert inspections of the user interface for a primary care decision support system

BackgroundGeneral practitioners play a unique key role in diagnosing patients with unclear diseases. Decision support systems in primary care can assist with diagnosis provided that they are efficient and user-friendly. ObjectivesThe objective of this study is to develop a high-fidelity prototype of the user interface of a clinical decision support system for primary care, particularly for diagnosis support in unclear diseases, using expert inspections at an early stage of development to ensure a high level of usability. MethodsThe user interface prototype was iteratively developed based on previous research, design principles, and usability guidelines. During the development phase, three usability inspections were carried out by all experts at four-week intervals as heuristic walkthrough. Each inspection consisted of two parts: 1) Task-based inspection 2) Free exploration and evaluation based on usability heuristics. Five domain experts assessed the current status of development.The tasks in the inspections were based on the task model derived in the requirements analysis: perform data entry, review and discuss results, schedule further diagnostics, refer to specialists and close case. ResultsAs a result of this iterative development, a high-fidelity, clickable user interface prototype was created that is able to fulfil all six tasks of our task model. The usability inspections identified a total of 196 usability issues (for all 3 inspections; Part 1: 90 issues, Part 2: 106 issues), ranging in severity from minor to severe. These served the continuous adjustment and improvement of the prototype. All main tasks were completed successfully despite these problems. ConclusionUsability inspections through heuristic walkthroughs can support and optimise the development of a user-centred decision support system in order to ensure its suitability for performing relevant tasks.

Usability Assessment of OpenEMR: Enhancing Healthcare Interactions

Electronic medical record (EMR) systems are pivotal in modern healthcare for digitizing patient records and improving clinical workflows. However, persistent usability challenges hinder their effective adoption and user satisfaction. This study employs a comprehensive user-centered design approach to identify and address usability issues in one such EHR, an open-source system named openEMR. Our main objective was to assess openEMR’s usability through several usability metrics. In addition to a summative usability test with 10 participants, we employed usability inspection (i.e., a heuristic evaluation) to generate additional findings using multiple methods. The results from the summative usability test provide a comprehensive perspective on openEMR’s performance. In addition to task-specific observations, participants offered insights into openEMR’s overall usability through a debrief interview. Through quantitative surveys, usability testing, qualitative interviews, and a heuristic evaluation, this study identified critical usability issues, proposes targeted solutions, and discusses implications for enhancing EMR systems in healthcare settings.

Evaluasi Usability dan User Experience Pada E-Learning Universitas Sriwijaya Menggunakan Metode TUXEL 2.0

The era of technology that continues to grow makes all work in life easier and more efficient. These developments have an impact on the world of education which innovates to create a learning style that can do learning whenever and wherever we want without the limitations of place and time with the help of technology called E-Learning. A website that functions as an E-Learning supporter is called a Learning Management System (LMS). Sriwijaya University already has an LMS called Sriwijaya University E-Learning. There are some problems and difficulties encountered in using E-Learning of Sriwijaya University. This is proven after the researcher conducted a pre-survey. The average answer stated strongly agree that Sriwijaya University E-Learning needs to be evaluated for usability and user experience. Furthermore, from the interview with the Coordinator of Learning Development Center (LP3MP UNSRI), the information was obtained that the need for input from users to make E-Learning better and to know the level of usability of E-Learning itself to continue to be used. The method used in this research is TUXEL 2.0 which consists of three stages namely Usability Inspection, General LMS/Pedagogical Usability Inspection, User Experience Evaluation. The sample used in this study was 150 students consisting of 15 people each per faculty. The result of usability inspection found 50 problems reported. while in General LMS/Pedagogical Usability Inspection there are total 33 problems found and in User Experience Evaluation stage, it is known that all aspect scales are > 0.8 means all aspects have positive level which indicates Sriwjaya University E-Learning is very good

Preliminary Usability Evaluation of Personal Medical Devices towards Better Home Healthcare

Portable medical devices have been playing an important role in improving health outcomes across the socio-economic spectrum. In the recent decades, substantial population have started using such devices to monitor important vital signs. Significant proportion of these users are older patients with chronic diseases, declined physical and/or cognitive capabilities. Furthermore, these portable devices are often used in unguided environments, that can present unpredictable use contexts. In this preliminary study, we conducted usability inspection on five portable medical devices. Cognitive walkthrough followed by Nielson’s ten Usability Heuristic evaluation method were utilized for the purpose of usability inspection. Sixteen usability concerns were uncovered in such evaluation study. We also recorded the elapsed time of measuring vital signs with all five devices on a daily basis in a month. The average total completion time after familiarization (after the first week) was 5.17 minutes. Results suggest that ‘non-expert’, independent users may face challenges when using portable medical devices in home care settings. Further, the study results also indicate that unsatisfactory usability of portable medical devices could pose hinderances in technology adoption among low eHealth literate users.

User Experience in Learning Management System: Edmodo versus Google Classroom

Nowadays, Learning Management System (LMS) is widely used to support the learning process. LMS becomes a medium of interaction between students and teachers. Poor usability and user experience (UX) of an LMS can make students spend more time learning the system than learning the educational content. Therefore, usability and user experience evaluation are important for e-learning. Edmodo and Google Classroom are two widely used LMS. This study involved students as inspectors to compare the user experience of Edmodo versus Google Classroom using Technique for User Experience Evaluation in E-learning (TUXEL). Inspectors divided into two groups: one group inspected Edmodo and another group inspected Google Classroom. After completing the inspection, the inspectors were asked to fill three questionnaires: usability inspection, general LMS / pedagogical usability inspection, and user experience. The data then processed and analyzed per category per dimension. The results of the overall general usability and pedagogical usability inspections show fewer problematic aspects in Edmodo (22 items) than Google Classroom (30 items). However, Google Classroom seems to be preferred by students because it is simpler, practical and, more comfortable. Edmodo is recommended for distance learning since it has more features. While Google Classroom is more suitable to be used as a support or complement for a learning course.

Understanding Pitfalls and Opportunities of Applying Heuristic Evaluation Methods to VR Training Systems: An Empirical Study

The usability of virtual reality (VR) training applications is crucial for their success, but examining the usability in the early development stages remains challenging. A realistic and plausible solution would be revisiting and reconciling Heuristics Evaluation (HE) methods among the most widely used usability inspection methods in the human-computer interaction (HCI) domain. While research on studying and using HE methods is growing within the VR domain, few studies have considered the novel VR environment challenges new requirements for fitting HE methods to the context and applying them effectively. To this end, we conducted a user study with 14 evaluators using the standard HE methods to complete two HE sessions for a VR training application. We identified five critical challenges that evaluators encountered in the HE process by observing and interviewing them. Based on our findings, we discuss the importance of considering an easy-to-use heuristic set, how we can facilitate the HE procedures in the VR context, and the opportunities for developing HE-supporting tools.

Effects of Heuristic Type, User Interaction Level, and Evaluator’s Characteristics on Usability Metrics of Augmented Reality (AR) User Interfaces

Heuristic evaluation is a famous usability inspection method often used in the assessment of Augmented Reality (AR) systems. Usability heuristics can be characterized as general or specific to a domain. During comparative studies between a new and a traditional set of heuristics, the focus is often around performance metrics (number of problems, time, errors), and little attention is given to the experience itself (i.e., evaluator’s overall satisfaction toward utilization of the set of heuristics). In addition, limited research has been done on the application of heuristics to evaluate interfaces with different levels of interaction, along with how a longer set of low-level heuristics compares against a traditionally-sized set of high-level heuristics. In this study, recruited participants (mostly university students) performed a heuristic evaluation using either a set of 110 specific heuristics (SH) or a set of 23 general heuristics (GH) to evaluate two versions of an AR application, designed with a low and a high level of interactivity. Big Five personality traits, prior experience with AR or Virtual Reality systems, and various usability data (e.g., subjective ratings related to the usability of heuristics and AR application, number of unique problems, and severity ratings) were collected through questionnaires and utilized during statistical analysis. Results show that even though the employment of the set of GH produced higher overall satisfaction of evaluators, the set of SH resulted in a significantly larger number of unique usability problems. Furthermore, the more interactive version of the AR application was perceived by evaluators as the harder version to inspect, regardless of the set of heuristics utilized. Neuroticism and agreeableness traits were found with a significant impact on subjective usability ratings.

Game development software engineering: digital educational game promoting algorithmic thinking

<span>The purpose of this study is to create a digital educational game that promotes algorithmic thinking for elementary school students. However, the processes of game development differ from traditional software development which cannot guarantee its effectiveness in terms of human-machine interfaces. In this article, we propose a new game development software engineering (GDSE) as a model for game development. This new model aims to complement and mitigate the shortcomings of traditional software development. The principles of human-computer interaction are now incorporated in the new model. The GDSE includes design, development, usability inspection, game experience evaluation, educational value evaluation and release. It was used as a research method to develop a game that promotes algorithmic thinking for children. The results of this study are not only a digital educational game that promotes algorithmic thinking for children but also a new game development life cycle that guarantees the performance of digital games in terms of usability enhancement, game experience and educational value.</span>

Performance, Usability, and User Experience of Rayyan for Systematic Reviews

ABSTRACTThis study investigated performance, usability, and user experience with Rayyan, a systematic review (SR) tool. A remote usability inspection was conducted via the institutional Zoom platform. Each participant was randomly assigned to two groups and completed two citation screening sessions independently. Data were collected through pre‐ and post‐test questionnaires, screening sessions, and Systems Usability Scale (SUS) surveys. Thirteen participants completed the study. We found significant accuracy difference between Web Rayyan and its mobile app (t[12] = 1.877 p = .042), but no statistically sequential effect on screening performance. The average SUS scores of both Rayyan versions were below 68. More participants (77%) preferred Web Rayyan to the mobile app. The mobile Rayyan has potential if its user interface design can prevent errors or allow easy action reversal. As the first controlled experiment investigating both Web and mobile Rayyan, this study provides empirical evidence to guide the design, development, and selection of SR tools.

Usability Inspection of a Mobile Cancer Telerehabilitation System.

Cancer rehabilitation (CR) has been shown to address physical impairments and progressive disablement experienced by patients with cancer. Access to CR is limited by multiple barriers. Previous studies demonstrated that telemedicine approaches may facilitate access to rehabilitation services however usability and acceptance of cancer telerehabilitation has not been systematically evaluated. This goal of this study was usability inspection of a mobile cancer telerehabilitation system based on cognitive walkthroughs and heuristics evaluations, supplemented by surveys to capture health literacy, patient engagement, and acceptance. The System Usability Scale provided a standardized reference for usability and satisfaction, and the mean result of 83.2±16.9 placed this mobile interface in the top 10th percentile. A semi-structured qualitative interview provided actionable feedback, which informed the next iteration of this project. Overall, this mobile telerehabilitation system was considered by cancer patients easy-to-use, satisfying, and engaging with 91% of participants planning to use it in the future.

Cognitive Testing of an Electronic Consent Platform: Researcher Perspectives.

This study focuses on feedback from domain experts to assess usability and acceptance of the E-Consent electronic consent platform. Quantitative and qualitative data were captured throughout the usability inspection, which was structured around a cognitive walkthrough with heuristics evaluation. Additional surveys measured biobanking knowledge and attitudes and familiarity with informed consent. A semi-structured qualitative interview captured open-ended feedback. 23 researchers of various ages and job titles were included for analysis. The System Usability Scale (SUS) provided a standardized reference for usability and satisfaction, and the mean result of 86.7 corresponds with an 'above average' usability rating in the >90th percentile. Overall, participants believe that electronic consenting using this platform will be faster than previous workflows while enhancing patient understanding, and human rapport is still a key component of the consent process. Expert review has provided valuable insight and actionable information that will be used to further enhance this maturing platform.

Application of Anti-Stigma Design Heuristics for Usability Inspection.

User interface evaluation has become important in developing usable health care technologies. Although usability engineering methods have been applied in the design and evaluation of health care software, available heuristics focus on task-work aspects and do not address stigma associated with many health conditions. We used a previous set of heuristics and propose a new set of anti-stigma heuristics to evaluate stigmatization in health care websites. The extended set of heuristics were concurrently applied in a heuristic evaluation and a cognitive walkthrough to evaluate an endometriosis and sexual pain website. The walkthrough involved 5 tasks that required 21 actions to execute. Twenty-six usability problems were identified and recommendations for re-design were made to the design team before end-user testing. The anti-stigma heuristics received worse ratings than the traditional heuristics, resulting in several design changes that might otherwise have been missed. Thus, the new anti-stigma heuristics were a valuable contribution.

Usability inspection: Novice crowd inspectors versus expert

Objective:This research study aims to investigate the use of novice crowd inspectors for usability inspection with respect to time spent and the cost incurred. This study compares the results of the novice crowd usability inspection guided by a single expert’s heuristic usability inspection (novice crowd usability inspection henceforth) with the expert heuristic usability inspection. Background:Traditional usability evaluation methods are time-consuming and expensive. Crowdsourcing has emerged as a cost-effective and quick means of software usability evaluation. Method:In this regard, we designed an experiment to evaluate the usability of two websites and a web dashboard. Results:The results of the experiment show that novice crowd usability inspection guided by a single expert’s heuristic usability inspection:a) Finds the same usability issues (w.r.t. content & quantity) as expert heuristic usability inspection.b) Is cost-effective than expert heuristic usability inspection employing less time duration. Conclusion:Based on the findings of this research study, we can conclude that the novice crowd usability inspection guided by a single expert’s heuristic usability inspection and expert heuristic usability inspection, on average, gives the same results in terms of issues identified.

Mobile Health for Smoking Cessation Among Disadvantaged Young Women During and After Pregnancy: User-Centered Design and Usability Study.

BackgroundSmoking prevalence during and after pregnancy remains high among socioeconomically disadvantaged women. Mobile health (mHealth) apps with game and social support elements seem promising to support smoking cessation.ObjectiveThis study aims to describe the user-centered design and usability evaluation of Kindle, an mHealth app with game and social support elements, to support disadvantaged young women during and after pregnancy through the first stages of smoking cessation.MethodsDisadvantaged women (n=9), members of their social networks (n=4), and nurses supporting these women (n=51) were informants throughout the iterative prototype development of Kindle according to the International Organization for Standardization 9241-11:2018. Specific phases included understanding the context of use through secondary analysis of qualitative interview data (phase 1), establishing the user and organizational requirements (phase 2), production of design solutions (phase 3), and usability inspection of the prototype through a heuristic evaluation (3 experts) along with user testing by a think aloud method (5 disadvantaged women and 5 nurses; phase 4). Usability problems were categorized according to the principles of the Healthcare Information and Management Systems Society.ResultsPhase 1 resulted in an understanding of the VoorZorg program and the needs of VoorZorg nurses and clients (eg, focus on early stages of change and building new supportive networks to aid clients in smoking cessation). In phase 2, we established requirements (n=22; eg, mHealth app, secure communication between nurses and clients, easy-to-use interfaces, inclusion of game elements, and tailoring at early stages of change in smoking cessation). Phase 3 resulted in a prototype of Kindle, combining the interface for nurses and clients, including the following functionalities: personal goal setting with earning points; secured chat function between nurses and other clients; and tips, diary, and profile creation. The heuristic evaluation and thinking aloud method in phase 4 revealed 78 usability problems in the interfaces. Most usability problems concerned simplicity (eg, unclear clickable button) and naturalness (eg, unclear icon).ConclusionsThe user-centered design and usability testing of the mHealth app Kindle yielded useful insights. The involvement of end users, specifically socioeconomically disadvantaged women during and after their pregnancy, resulted in a prototype that met their needs and requirements (eg, mHealth app, secure communication between nurses and clients, easy-to-use interfaces, inclusion of game elements, and tailoring to the early stages of change in smoking cessation) to achieve readiness for smoking cessation. Moreover, the usability evaluation by end users and experts revealed unique usability problems for this population. These insights allow for further optimization of Kindle and encourage future studies to engage disadvantaged populations in all phases of mHealth intervention design and usability testing.

Social Virtual Reality Platform Comparison and Evaluation Using a Guided Group Walkthrough Method

As virtual reality (VR) headsets become more commercially accessible, a range of social platforms have been developed that exploit the immersive nature of these systems. There is a growing interest in using these platforms in social and work contexts, but relatively little work into examining the usability choices that have been made. We developed a usability inspection method based on cognitive walkthrough that we call guided group walkthrough. Guided group walkthrough is applied to existing social VR platforms by having a guide walk the participants through a series of abstract social tasks that are common across the platforms. Using this method we compared six social VR platforms for the Oculus Quest. After constructing an appropriate task hierarchy and walkthrough question structure for social VR, we ran several groups of participants through the walkthrough process. We undercover usability challenges that are common across the platforms, identify specific design considerations and comment on the utility of the walkthrough method in this situation.

A Review of Website Measurement for Website Usability Evaluation

There are many website usability measurement. Therefore, researcher that focus on usability measurement need to select the best method for usability evaluation to get the best result. This paper review website measurement that has classify to usability inspection, usability inquiry and usability testing. Some of usability measurement focus on experts only to measure the evaluation and some of measurement focus on real users. Each of website measurement have advantage and disadvantage. Researcher need to determine the scope, context and the reason when evaluate usability for website. This paper found that survey is common method that used by researcher because it save cost and can gather a population easily.

A Heuristic Evaluative Framework for Self-Regulated Learning Design

Researchers and educators have explored a variety of technologies to facilitate self-regulated learning (SRL). Drawing from contemporary perspectives on SRL, this paper articulates two fundamental design principles for SRL-promoting technologies: the platform principle and the support principle. This paper then discusses how usability inspection methods, such as heuristic evaluations and cognitive walkthroughs, can readily assess whether and how these needs are met. This framework can assist researchers and educators in evaluating technologies to make strategic design and implementation decisions aligned with self-regulation.

Usability of Remote Assessment of Exercise Capacity for Pulmonary Telerehabilitation Program.

Pulmonary rehabilitation [PR] has been successfully carried out via telemedicine however initial patient assessment has been traditionally conducted in PR centers. The first step in PR is assessment of patient's exercise capacity which allows individualized prescription of safe and effective exercise program. With COVID-19 pandemics assessment of patients in PR centers has been limited resulting in significant reduction of patients undergoing life-saving PR. The goal of this pilot study was to introduce approaches for remote assessment of exercise capacity using videoconferencing platforms and provide initial usability assessment of this approach by conducing cognitive walkthrough testing. We developed a remote assessment system that supports comprehensive physical therapy assessment necessary for prescription of a personalized exercise program tailored to individual fitness level and limitations in gait and balance of the patient under evaluation. Usability was assessed by conducting cognitive walkthrough and system usability surveys. The usability inspection of the remote exercise assessment demonstrated overall high acceptance by all study participants. Our next steps in developing user-centered interface should include usability evaluation in different subgroups of patients with varying socio-economic background, different age groups, computer skills, literacy and numeracy.

Heuristic Evaluation of the User Interface for a Semi-Autonomous Agricultural Robot Sprayer

This study presents the heuristic evaluation, as a usability inspection method for Human-Robot Interaction (HRI) systems. First, the methodology to engineer a semi-autonomous agricultural robot sprayer is presented, and then findings from heuristic usability evaluation studies that were carried out on a human-robot interface for a semi-autonomous agricultural vineyard robot sprayer. The following research-based heuristics for the design of robot teleoperation were used: Platform architecture and scalability, Error prevention and recovery, Visual design, Information presentation, Robot state awareness, Interaction effectiveness and efficiency, Robot environment/surroundings awareness, and Cognitive factors. In each evaluation study, usability problems were identified, and specific suggestions were documented for HRI usability improvement. In each design iteration, a smaller number of usability issues were identified.. Results of the final heuristic evaluation showed that the system is at a good level of usability and is expected to provide satisfactory services to its typical users.